a cyk variant for scfg decoding without a dot chart
play

A CYK+ Variant for SCFG Decoding Without a Dot Chart Rico Sennrich - PowerPoint PPT Presentation

Feb 21, 2023 •28 likes •408 views

A CYK+ Variant for SCFG Decoding Without a Dot Chart Rico Sennrich Institute for Language, Cognition and Computation University of Edinburgh October 25 2014 R. Sennrich Recursive CYK+ 1 / 15 Outline CYK+ and the role of the dot chart

  1. A CYK+ Variant for SCFG Decoding Without a Dot Chart Rico Sennrich Institute for Language, Cognition and Computation University of Edinburgh October 25 2014 R. Sennrich Recursive CYK+ 1 / 15

  2. Outline CYK+ and the role of the dot chart Recursive variant Evaluation R. Sennrich Recursive CYK+ 2 / 15

  3. Problem CYK+ parsing CYK+ and Earley-style variants are popular parsers for decoding with SCFGs (Moses, cdec, SAMT, Jane, ...). alternative: binarization and decoding with plain CYK. problem CYK+ parsing [with syntactic models] takes a lot of memory. n = 20 n = 40 n = 80 0.32 GB 2.63 GB 51.64 GB most of the memory is consumed by the dot chart. R. Sennrich Recursive CYK+ 3 / 15

  4. Problem CYK+ parsing CYK+ and Earley-style variants are popular parsers for decoding with SCFGs (Moses, cdec, SAMT, Jane, ...). alternative: binarization and decoding with plain CYK. problem CYK+ parsing [with syntactic models] takes a lot of memory. n = 20 n = 40 n = 80 0.32 GB 2.63 GB 51.64 GB most of the memory is consumed by the dot chart. solution in this talk, we present a variant of CYK+ without a dot chart. our variant requires less memory and is faster, with same result. R. Sennrich Recursive CYK+ 3 / 15

  5. CYK+ The CYK+ algorithm bottom-up chart parser generalization of CYK to n -ary rules two data structures: main chart: non-terminal symbols dot chart: rule prefix applications (dotted items) difference to Earley: dotted item represents all rules with same prefix dot chart allows dynamic binarization: rules that match span (i,j) are found by combining dotted item in (i,k) and (non-)terminal symbol in span (k,j). R. Sennrich Recursive CYK+ 4 / 15

  6. CYK+ it is a trap it is a trap 1 1 → S NP V NP 2 2 NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  7. CYK+ it is a trap it is a trap 1 1 NP → S NP V NP 2 2 NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  8. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  9. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  10. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  11. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  12. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 NN → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  13. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 NN → NP it 5 NP V • 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  14. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 NN → NP it 5 NP V • 5 V → is 6 6 → ART a 7 7 NP 8 8 NN → trap 9 9 10 10 dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  15. CYK+ it is a trap it is a trap 1 NP • 1 NP → S NP V NP 2 2 V NP → ART NN 3 3 ART • ART 4 4 NN → NP it 5 NP V • 5 V → is 6 6 → ART a 7 7 NP 8 8 NN → trap 9 9 10 10 S dot chart main chart grammar CYK+ steps search for terminal rule of size 1. 1 combine dotted item and (non-)terminal of two subspans. 2 create new dotted item from (non-)terminal in cell. 3 R. Sennrich Recursive CYK+ 5 / 15

  16. CYK+: complexity considerations monolingual 1-best parser main chart: O ( n 2 ) dot chart: O ( n 2 ) parsing steps: O ( n 3 ) SCFG decoding Non-locality of LM scores restricts recombination of dotted items [Hopkins and Langmead, 2010] main chart: O ( n 2 ) (with beam search) on the fast jet ski of mr smith the JJ NPB of NNP dot chart: O ( n scope ( G ) ) the JJ NPB of NNP the JJ NPB of NNP parsing steps: O ( n scope ( G ) ) the JJ NPB of NNP choice point choice point rule scope: number of choice points in rule R. Sennrich Recursive CYK+ 6 / 15

  17. The dot chart in SCFG decoding purpose of dot chart allows recombination of different dotted items → does not apply to SCFG decoding allows re-use of same dotted item for different spans it is a trap it is a trap 1 NP • 1 NP 2 2 V 3 3 ART • ART 4 4 NN 5 NP V • 5 6 6 7 7 NP 8 8 9 9 10 10 S dot chart main chart R. Sennrich Recursive CYK+ 7 / 15

  18. The dot chart in SCFG decoding purpose of dot chart allows recombination of different dotted items → does not apply to SCFG decoding allows re-use of same dotted item for different spans it is a trap it is a trap 1 NP • 1 NP 2 2 V 3 3 ART • ART 4 4 NN 5 NP V • 5 6 6 7 7 NP 8 8 9 9 10 10 S dot chart main chart R. Sennrich Recursive CYK+ 7 / 15

  19. The dot chart in SCFG decoding purpose of dot chart allows recombination of different dotted items → does not apply to SCFG decoding allows re-use of same dotted item for different spans it is a trap it is a trap 1 NP • 1 NP 2 2 V 3 3 ART • ART 4 4 NN 5 NP V • 5 6 6 7 7 NP 8 8 9 9 10 10 S dot chart main chart R. Sennrich Recursive CYK+ 7 / 15

  20. The dot chart in SCFG decoding purpose of dot chart allows recombination of different dotted items → does not apply to SCFG decoding allows re-use of same dotted item for different spans it is a trap it is a trap 1 NP • 1 NP 2 2 V 3 3 ART • ART 4 4 NN 5 NP V • 5 6 6 7 7 NP 8 8 9 9 10 10 S dot chart main chart R. Sennrich Recursive CYK+ 7 / 15

  21. A recursive variant Core idea we do not initially know if rule prefix application can be extended. → dotted items are re-visited throughout time. we can change chart traversal order to guarantee that when span (i,k) is visited, all spans (k,j) have been visited before. this eliminates need to store dotted items; instead, they are extended recursively, then discarded. it is a trap it is a trap 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 traditional proposed R. Sennrich Recursive CYK+ 8 / 15

  22. Recursive CYK+ it is a trap it is a trap 1 1 → S NP V NP 2 2 NP → ART NN 3 3 4 4 → NP it 5 5 V → is 6 6 → ART a 7 7 8 8 NN → trap 9 9 10 10 dot chart main chart grammar recursive CYK+ steps search for terminal rule of size 1. 1 initial call to rule consume function. 2 recursive call to rule consume function. 3 R. Sennrich Recursive CYK+ 9 / 15

Recommend

Chris Hopkins, PhD, MBA and CSO patient variant drug p.R292H pathogenicity Is this variant in

Chris Hopkins, PhD, MBA and CSO patient variant drug p.R292H pathogenicity Is this variant in

Chris Hopkins, PhD, MBA and CSO patient variant drug p.R292H pathogenicity Is this variant in my patient pathogenic or benign? "The Doctor" by Sir Luke Fildes (1891) Drop in genome price drives increase in patient variant

500 views • 22 slides
<? <?php php $ret = $ret = foo foo($a ($a); ); // C++ // C++ Variant v_ret Variant

<? <?php php $ret = $ret = foo foo($a ($a); ); // C++ // C++ Variant v_ret Variant

<? <?php php $ret = $ret = foo foo($a ($a); ); // C++ // C++ Variant v_ret Variant v_ret; ; Variant Variant v_a v_a; ; v_ret v_ret = = f_foo f_foo(v_a (v_a); ); Facebook 2010 (confidential) <?php <?

287 views • 12 slides
IDN Variant TLDs Program Update IDN Variant TLDs Program Origins No variants of gTLDs will be

IDN Variant TLDs Program Update IDN Variant TLDs Program Origins No variants of gTLDs will be

IDN Variant TLDs Program Update IDN Variant TLDs Program Origins No variants of gTLDs will be delegated until appropriate variant management solutions are developed http://www.icann.org/en/groups/board/do

533 views • 31 slides
Variant Effect Predictor Demo: The Variant Effect Predictor (VEP)

Variant Effect Predictor Demo: The Variant Effect Predictor (VEP)

Variant Effect Predictor Demo: The Variant Effect Predictor (VEP) We have identified five variants on human chromosome nine, an A deletion at 128328461, C->A

301 views • 7 slides
Working with Charts Objectives Understand and plan a chart Create a chart Move and

Working with Charts Objectives Understand and plan a chart Create a chart Move and

Working with Charts Objectives Understand and plan a chart Create a chart Move and resize charts and chart objects Apply chart layouts and styles Microsoft Office 2013-Illustrated Fundamentals 2 Objectives Customize chart

621 views • 26 slides
An Extended GHKM Algorithm for Inducing -SCFG Peng Li, Yang Liu and Maosong Sun THUNLP&CSS

An Extended GHKM Algorithm for Inducing -SCFG Peng Li, Yang Liu and Maosong Sun THUNLP&CSS

An Extended GHKM Algorithm for Inducing -SCFG Peng Li, Yang Liu and Maosong Sun THUNLP&CSS Tsinghua University, China Outline l Background l Rule extraction algorithm l Modeling l Experiments l Conclusion 2 Outline l

664 views • 40 slides
A General-Purpose Rule Extractor for SCFG-Based Machine Translation Greg Hanneman , Michelle

A General-Purpose Rule Extractor for SCFG-Based Machine Translation Greg Hanneman , Michelle

A General-Purpose Rule Extractor for SCFG-Based Machine Translation Greg Hanneman , Michelle Burroughs , and Alon Lavie Language Technologies Institute Carnegie Mellon University Fifth Workshop on Syntax and Structure in Statistical Translation

312 views • 28 slides
Observation Decoding with Sensor Models: Recognition Tasks via Classical Planning Diego Aineto,

Observation Decoding with Sensor Models: Recognition Tasks via Classical Planning Diego Aineto,

Observation Decoding with Sensor Models: Recognition Tasks via Classical Planning Diego Aineto, Sergio Jimenez, Eva Onaindia October 16, 2020 Universitat Polit` ecnica de Val` encia 1 What is decoding? What is decoding? Decoding : finding

848 views • 26 slides
Variant-Related Issues Variant TLD Issues Project Goal

Variant-Related Issues Variant TLD Issues Project Goal

Variant-Related Issues Variant TLD Issues Project Goal Iden0fy and agree on issues related to variants that should be included in the report 2

218 views • 10 slides
Chapter 6 Decoding Statistical Machine Translation Decoding We have a mathematical model for

Chapter 6 Decoding Statistical Machine Translation Decoding We have a mathematical model for

Chapter 6 Decoding Statistical Machine Translation Decoding We have a mathematical model for translation p ( e | f ) Task of decoding: find the translation e best with highest probability e best = argmax e p ( e | f ) Two types of

571 views • 34 slides
Jorge Jimnez Variant calling jjimeneza@cipf.es Index 1. Variant calling pipeline 2.

Jorge Jimnez Variant calling jjimeneza@cipf.es Index 1. Variant calling pipeline 2.

Jorge Jimnez Variant calling jjimeneza@cipf.es Index 1. Variant calling pipeline 2. Alignment processing 3. SNP calling 4. Short indel calling 5. VCF format 6. Structural Variation Jorge Jimnez Variant calling jjimeneza@cipf.es

1.12k views • 40 slides
Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the

Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the

Why decoding? Understanding the neural code. Neural Decoding Given spikes, what was the stimulus? What aspects of the stimulus does the system encode? (capacity is limited) Mark van Rossum What information can be extracted from spike trains:

779 views • 16 slides
By et al Siegfried Engelmann Decoding Strategies: Decoding B1- Teacher's Presentation Book

By et al Siegfried Engelmann Decoding Strategies: Decoding B1- Teacher's Presentation Book

By et al Siegfried Engelmann Decoding Strategies: Decoding B1- Teacher's Presentation Book (Corrective Reading) (3rd Revised edition) From McGraw-Hill Inc.,US By et al Siegfried Engelmann Decoding Strategies: Decoding B1- Teacher's Presentation

233 views • 5 slides
1 Variant II: Command Consensus (BG) Variant III: Interactive Consistency (IC) Variant II:

1 Variant II: Command Consensus (BG) Variant III: Interactive Consistency (IC) Variant II:

Coordination Coordination If the solution to availability and scalability is to decentralize and replicate functions and data, how do we coordinate the nodes? data consistency Failures and Consensus Failures and Consensus update

308 views • 6 slides
Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling

Beyond Sequential decoding toward parallel decoding In the context of neural sequence modelling Kyunghyun Cho New York University & Facebook AI Research Joint work with Jason Lee and Elman Mansimov Neural sequence modeling An arbitrary

461 views • 25 slides
IDN Variant TLD Implementation Status, Recommendations and Next Steps GNSO Council 18 April 2019

IDN Variant TLD Implementation Status, Recommendations and Next Steps GNSO Council 18 April 2019

IDN Variant TLD Implementation Status, Recommendations and Next Steps GNSO Council 18 April 2019 | 1 Objectives of This Session What: Understand 1. 1. IDN variant top-level domains (TLDs) 2. Status of IDN variant TLDs 3. Next

474 views • 24 slides
Decoding challenge Assessing the practical hardness of syndrome decoding for code-based

Decoding challenge Assessing the practical hardness of syndrome decoding for code-based

Decoding challenge Assessing the practical hardness of syndrome decoding for code-based cryptography Matthieu Lequesne Sorbonne Universit Inria Paris, team Cosmiq February 27, 2020 All you ever wanted to know about code-based crypto

1.53k views • 132 slides
Decoding in SMT Nitin Madnani February 8, 2006 The Decoding Problem Search Inputs:

Decoding in SMT Nitin Madnani February 8, 2006 The Decoding Problem Search Inputs:

Decoding in SMT Nitin Madnani February 8, 2006 The Decoding Problem Search Inputs: Input string Bunch of statistical models A function to assign score to any translation Output: Best scoring translation

660 views • 34 slides
CHART | ART FAIR 29. 31. AUGUST 2014 CHART | ART FAIR IS AN INNOVATIVE ART FAIR WITH A HIGH

CHART | ART FAIR 29. 31. AUGUST 2014 CHART | ART FAIR IS AN INNOVATIVE ART FAIR WITH A HIGH

C H A R T CHART | ART FAIR KUNSTHAL CHARLOTTENBORG IN COPENHAGEN third edition www.chartartfair.com CHART | ART FAIR 29. 31. AUGUST 2014 CHART | ART FAIR IS AN INNOVATIVE ART FAIR WITH A HIGH QUALITY ART EXPERIENCE AS ITS MAIN FOCUS.

114 views • 8 slides
Decoding Philipp Koehn 17 September 2020 Philipp Koehn Machine Translation: Decoding 17

Decoding Philipp Koehn 17 September 2020 Philipp Koehn Machine Translation: Decoding 17

Decoding Philipp Koehn 17 September 2020 Philipp Koehn Machine Translation: Decoding 17 September 2020 Decoding 1 We have a mathematical model for translation p ( e | f ) Task of decoding: find the translation e best with highest

1.04k views • 61 slides
Chart 1: Children s Media Use s Media Use Chart 1: Children Chart 1: Childrens Media

Chart 1: Children s Media Use s Media Use Chart 1: Children Chart 1: Childrens Media

Chart 1: Children s Media Use s Media Use Chart 1: Children Chart 1: Childrens Media Use In a typical day, percent of children 6 and under who Use any screen media 83% Watch TV 75% Watch videos/DVDs 32% Use a computer 16%

303 views • 27 slides
Smaller decoding exponents: ball-collision decoding D. J. Bernstein University of Illinois at

Smaller decoding exponents: ball-collision decoding D. J. Bernstein University of Illinois at

Smaller decoding exponents: ball-collision decoding D. J. Bernstein University of Illinois at Chicago Joint work with: Tanja Lange Technische Universiteit Eindhoven Christiane Peters University of Illinois at Chicago Context: speed What is

500 views • 36 slides
Decoding One Out of Many Nicolas Sendrier INRIA Paris-Rocquencourt, equipe-projet SECRET

Decoding One Out of Many Nicolas Sendrier INRIA Paris-Rocquencourt, equipe-projet SECRET

Decoding One Out of Many Nicolas Sendrier INRIA Paris-Rocquencourt, equipe-projet SECRET Code-based Cryptography Workshop 11-12 May 2011, Eindhoven, The Netherlands Computational Syndrome Decoding Problem: Syndrome Decoding H { 0 , 1 }

882 views • 37 slides
GHUMVEE: Efficient, Effective and Flexible Replication Stijn Volckaert Computer Systems Lab

GHUMVEE: Efficient, Effective and Flexible Replication Stijn Volckaert Computer Systems Lab

Vakgroep ELIS GHUMVEE: Efficient, Effective and Flexible Replication Stijn Volckaert Computer Systems Lab Ghent University Belgium N-modular Redundancy variant 1 input variant 2 monitor output variant 3 equivalent components 2

450 views • 16 slides

More recommend