weighted parsing for grammar based language models
play

Weighted parsing for grammar-based language models FSMNLP 2019 - PowerPoint PPT Presentation

Feb 23, 2024 β€’353 likes β€’1.61k views

Weighted parsing for grammar-based language models FSMNLP 2019 Richard Mrbitz Heiko Vogler 2019-09-25 The weighted parsing problem language e.g., English sentences ( ) syntactic object e.g., Fruit fmies like bananas ?

  1. Language algebras 𝑒 ∈ AST (𝐻) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 … Fruit Fruit 𝛽 … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) S β†’ 𝛽( NP , VP ) factors ( Fruit flies like bananas ) = { Fruit , like bananas , … } 7 / 21 βŸ¨π‘¦ 1 flies 𝑦 2 ⟩ 𝜌 𝛡 : T 𝑆 β†’ T 𝛡 interpretation of 𝛡 as operations on the set of syntactic objects β„’ = 𝛦 βˆ— (.) 𝛦 βˆ— : T 𝛡 (terms) β†’ 𝛦 βˆ— (syntactic objects)

  2. Language algebras ⟨ Fruit flies … ⟩ 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 … Fruit Fruit 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) S β†’ 𝛽( NP , VP ) factors ( Fruit flies like bananas ) = { Fruit , like bananas , … } 7 / 21 𝜌 𝛡 : T 𝑆 β†’ T 𝛡 interpretation of 𝛡 as operations on the set of syntactic objects β„’ = 𝛦 βˆ— (.) 𝛦 βˆ— : T 𝛡 (terms) β†’ 𝛦 βˆ— (syntactic objects)

  3. Language algebras ⟨ Fruit flies … ⟩ 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 … Fruit Fruit 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) S β†’ 𝛽( NP , VP ) factors ( Fruit flies like bananas ) = { Fruit , like bananas , … } 7 / 21 𝜌 𝛡 : T 𝑆 β†’ T 𝛡 interpretation of 𝛡 as operations on the set of syntactic objects β„’ = 𝛦 βˆ— (.) 𝛦 βˆ— : T 𝛡 (terms) β†’ 𝛦 βˆ— (syntactic objects)

  4. Language algebras 𝛽 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 … 𝛾 πœ€ 𝛿 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) S β†’ 𝛽( NP , VP ) factors ( Fruit flies like bananas ) = { Fruit , like bananas , … } 7 / 21 𝜌 𝛡 : T 𝑆 β†’ T 𝛡 (.) 𝛦 βˆ— : T 𝛡 β†’ 𝛦 βˆ— interpretation of 𝛡 as operations on the set of syntactic objects β„’ = 𝛦 βˆ— (.) 𝛦 βˆ— : T 𝛡 (terms) β†’ 𝛦 βˆ— (syntactic objects)

  5. Semirings Algebraic structure ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) βŠ— is used to evaluate an AST to a weight βŠ• accumulates the weights of several ASTs Examples ( 𝔺 , ∨, ∧, false , true ) the Boolean semiring with 𝔺 = { false , true } ( β„• ∞ , +, β‹…, 0, 1) the semiring of natural numbers ( β„• ∞ , min , +, ∞, 0) the tropical semiring ( ℝ 1 0 , max , β‹…, 0, 1) the Viterbi semiring Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 8 / 21

  6. Semirings Algebraic structure ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) βŠ— is used to evaluate an AST to a weight βŠ• accumulates the weights of several ASTs Examples ( 𝔺 , ∨, ∧, false , true ) the Boolean semiring with 𝔺 = { false , true } ( β„• ∞ , +, β‹…, 0, 1) the semiring of natural numbers ( β„• ∞ , min , +, ∞, 0) the tropical semiring ( ℝ 1 0 , max , β‹…, 0, 1) the Viterbi semiring Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 8 / 21

  7. 𝛻 βŠ— = { mul 𝕝 ( 𝕝 1 , … , 𝕝 𝑛 ) = 𝕝 βŠ— 𝕝 1 βŠ— β‹― βŠ— 𝕝 𝑛 with 𝛻 med = { del , ins , rep = , rep β‰  , nil } Multioperator monoids (M-monoids) mul 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Minimum edit distance M-monoid ({{π‘œ} ∣ π‘œ ∈ β„• }, min ∘ βˆͺ, βˆ…, 𝛻 med ) 0 , max , 0, 𝛻 mul ) Viterbi M-monoid ( ℝ 1 Examples (𝑛) 𝕝 ∣ 𝕝 ∈ 𝕃 , 𝑛 ∈ β„• } Generalization of semirings (𝑛) M-monoid ( 𝕃 , βŠ• , πŸ™ , 𝛻 βŠ— ) where Semiring ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) ⟢ ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⟢ ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) 9 / 21 binary βŠ— set of 𝑛 -ary operations 𝛻 (here: distributive)

  8. with 𝛻 med = { del , ins , rep = , rep β‰  , nil } Multioperator monoids (M-monoids) Generalization of semirings 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Minimum edit distance M-monoid ({{π‘œ} ∣ π‘œ ∈ β„• }, min ∘ βˆͺ, βˆ…, 𝛻 med ) 0 , max , 0, 𝛻 mul ) Viterbi M-monoid ( ℝ 1 Examples (𝑛) mul ∣ 𝕝 ∈ 𝕃 , 𝑛 ∈ β„• } 𝕝 (𝑛) ⟢ ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⟢ ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) 9 / 21 binary βŠ— set of 𝑛 -ary operations 𝛻 (here: distributive) Semiring ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) ⇝ M-monoid ( 𝕃 , βŠ• , πŸ™ , 𝛻 βŠ— ) where 𝛻 βŠ— = { mul 𝕝 ( 𝕝 1 , … , 𝕝 𝑛 ) = 𝕝 βŠ— 𝕝 1 βŠ— β‹― βŠ— 𝕝 𝑛

  9. Multioperator monoids (M-monoids) Generalization of semirings 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Minimum edit distance M-monoid ({{π‘œ} ∣ π‘œ ∈ β„• }, min ∘ βˆͺ, βˆ…, 𝛻 med ) 0 , max , 0, 𝛻 mul ) Viterbi M-monoid ( ℝ 1 Examples (𝑛) mul ∣ 𝕝 ∈ 𝕃 , 𝑛 ∈ β„• } 𝕝 (𝑛) ⟢ ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⟢ ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) 9 / 21 binary βŠ— set of 𝑛 -ary operations 𝛻 (here: distributive) Semiring ( 𝕃 , βŠ• , βŠ— , πŸ™ , 𝟚 ) ⇝ M-monoid ( 𝕃 , βŠ• , πŸ™ , 𝛻 βŠ— ) where 𝛻 βŠ— = { mul 𝕝 ( 𝕝 1 , … , 𝕝 𝑛 ) = 𝕝 βŠ— 𝕝 1 βŠ— β‹― βŠ— 𝕝 𝑛 with 𝛻 med = { del , ins , rep = , rep β‰  , nil }

  10. 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra) 0.2 β‹… 𝕝 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 Weight algebras S β†’ 𝛽( NP , VP ) (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 10 / 21

  11. 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra) 0.2 β‹… 𝕝 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 Weight algebras S β†’ 𝛽( NP , VP ) (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 10 / 21

  12. 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra) Weight algebras S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 β‹… 𝕝 1 10 / 21 (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2

  13. Weight algebras (.) 𝛦 βˆ— 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 β‹… 𝕝 1 S β†’ 𝛽( NP , VP ) 10 / 21 Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  14. Weight algebras (.) 𝛦 βˆ— 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 S β†’ 𝛽( NP , VP ) 10 / 21 Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  15. Weight algebras S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt 0.6 β‹… … 1.0 0.2 (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 10 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  16. Weight algebras S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) wt (𝑒) ∈ T 𝛻 wt 0.6 β‹… … 1.0 0.2 0.12 β‹… … (.) 𝛦 βˆ— Fruit flies … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 10 / 21 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  17. Weight algebras S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) ℝ 0 0 , max , 0, 𝛻 mul ) ( ℝ 1 wt : 𝑆 (set of rules) β†’ 𝛻 (set of operations) 0 (.) ℝ 1 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 β‹… 𝕝 1 10 / 21 (.) 𝛦 βˆ— NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 Fruit flies … 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 1 : T 𝛻 (terms) β†’ ℝ 1 0 (weight algebra)

  18. β„’ A wRTG-LM is a tuple ( (𝐻 = (𝑂, 𝛡, 𝐡 0 , 𝑆)) Weighted RTG-based language models ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ RTG , ⏟ language algebra ), ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⏟ M-monoid , wt ⏟ wt : 𝑆 β†’ 𝛻 ) . Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 ⏟⏟ Defjnition (weighted RTG-based language model) S β†’ 𝛽( NP , VP ) 𝜌 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 (.) 𝛦 βˆ— 0.2 β‹… 𝕝 1 1.0 … wt wt (𝑒) ∈ T 𝛻 0.12 β‹… … (.) ℝ 1 0 Fruit flies … 11 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2

  19. Weighted RTG-based language models language algebra ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ RTG , ⏟ ), S β†’ 𝛽( NP , VP ) ( 𝕃 , βŠ• , πŸ™ , 𝛻) ⏟ M-monoid , wt ⏟ wt : 𝑆 β†’ 𝛻 ) . Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 ⏟⏟ Defjnition (weighted RTG-based language model) (.) 𝛦 βˆ— 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 11 / 21 0.2 β‹… 𝕝 1 1.0 … wt wt (𝑒) ∈ T 𝛻 0.12 β‹… … (.) ℝ 1 0 Fruit flies … 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 β„’ A wRTG-LM is a tuple ( (𝐻 = (𝑂, 𝛡, 𝐡 0 , 𝑆))

  20. Outline 1 Weighted RTG-based language models 2 The weighted parsing problem 3 The weighted parsing algorithm Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 12 / 21

  21. The weighted parsing problem 0.2 β‹… 𝕝 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) 𝛦 βˆ— Fruit flies … 0 (.) ℝ 1 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt … 1.0 13 / 21 S β†’ 𝛽( NP , VP ) 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 𝜌 𝛡 … 𝛾 πœ€ 𝛿 𝛽 𝑒 ∈ AST (𝐻) … VP β†’ 𝛾( VBZ , PP ) NN β†’ πœ€ NP β†’ 𝛿( NN ) 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2

  22. The weighted parsing problem S β†’ 𝛽( NP , VP ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— Fruit flies … 0 (.) ℝ 1 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt … 1.0 0.2 β‹… 𝕝 1 𝛿 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  23. The weighted parsing problem (.) 𝛦 βˆ— 0.12 β‹… … (.) ℝ 1 0 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 wt (𝑒 β€² ) ∈ T 𝛻 S β†’ 𝛽( NP , VP ) wt 0.0144 (.) ℝ 1 0 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 wt (𝑒) ∈ T 𝛻 wt … πœ€ NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  24. The weighted parsing problem wt (𝑒 β€² ) ∈ T 𝛻 (.) ℝ 1 0 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— wt S β†’ 𝛽( NP , VP ) 0.0144 (.) ℝ 1 0 ( ℝ 1 0 , max , 0, 𝛻 mul ) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt 𝛾 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ … … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  25. The weighted parsing problem wt (𝑒 β€² ) ∈ T 𝛻 (.) ℝ 1 0 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— wt S β†’ 𝛽( NP , VP ) 0.0144 (.) ℝ 1 0 ( ℝ 1 0 , max , 0, 𝛻 mul ) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 0.12 β‹… … wt (𝑒) ∈ T 𝛻 wt … NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 max { 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  26. The weighted parsing problem wt (.) ℝ 1 0 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— wt (𝑒 β€² ) ∈ T 𝛻 0.0144 wt (𝑒) ∈ T 𝛻 (.) ℝ 1 0 ( ℝ 1 0 , max , 0, 𝛻 mul ) parse Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 S β†’ 𝛽( NP , VP ) 0.12 β‹… … wt … NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 max { 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  27. The weighted parsing problem (.) ℝ 1 Fruit flies … (.) 𝛦 βˆ— 𝜌 𝛡 (.) 𝛦 βˆ— wt (𝑒 β€² ) ∈ T 𝛻 wt 0.0144 0 S β†’ 𝛽( NP , VP ) ( ℝ 1 0 , max , 0, 𝛻 mul ) parse parse (𝑏) = βˆ‘ βŠ• π‘’βˆˆ AST (𝐻,𝑏) wt (𝑒) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 0 (.) ℝ 1 0.12 β‹… … wt (𝑒) ∈ T 𝛻 NP β†’ 𝛿( NN ) NN β†’ πœ€ VP β†’ 𝛾( VBZ , PP ) … 𝑒 ∈ AST (𝐻) 𝛽 𝛿 πœ€ 𝛾 … 𝜌 𝛡 𝑒 ∈ 𝑀(𝐻) βŠ† T 𝛡 0.2 β‹… 𝕝 1 1.0 … wt 13 / 21 1.0 β‹… 𝕝 1 β‹… 𝕝 2 0.6 β‹… 𝕝 1 β‹… 𝕝 2 max { 𝑒 β€² ∈ T 𝛡 𝑒 β€² ∈ AST (𝐻)

  28. The weighted parsing problem Algebraic dynamic programming (Giegerich, Meyer, and Stefgen 2004) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) Shamir 1961) Reduct of a grammar and a syntactic object (cf. Bar-Hillel, Perles, and matrix chain multiplication minimum edit distance Parsing with superior grammars (Knuth 1977; Nederhof 2003) Examples π‘œ best derivation(s) best derivation(s) derivation forest probability of best derivation string probability recognition Semiring parsing (Goodman 1999) 14 / 21

  29. Outline 1 Weighted RTG-based language models 2 The weighted parsing problem 3 The weighted parsing algorithm Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 15 / 21

  30. ((𝐻, β„’ ), 𝕃 , wt ) - 𝑏 ∈ β„’ Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) - wRTG-LM parse (𝑏) = βˆ‘ βŠ• π‘’βˆˆ AST (𝐻,𝑏) wt (𝑒) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

  31. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) - wRTG-LM parse (𝑏) = βˆ‘ βŠ• π‘’βˆˆ AST (𝐻,𝑏) wt (𝑒) Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) - 𝑏 ∈ β„’

  32. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) - wRTG-LM parse (𝑏) = βˆ‘ βŠ• π‘’βˆˆ AST (𝐻,𝑏) wt (𝑒) canonical weighted deduction system Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) - 𝑏 ∈ β„’

  33. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  34. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ? = wt β€² (𝑒) π‘’βˆˆ AST (𝐻 β€² ) βˆ‘ βŠ• wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  35. Weighted parsing algorithm Two-phase pipeline (Goodman 1999; Nederhof 2003) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) algorithm computation value ? = wt β€² (𝑒) π‘’βˆˆ AST (𝐻 β€² ) βˆ‘ βŠ• 16 / 21 wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  36. Weighted parsing algorithm π‘’βˆˆ AST (𝐻 β€² ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ? = 0 ) π‘Š(𝐡 β€² algorithm computation value ? = wt β€² (𝑒) βˆ‘ βŠ• Two-phase pipeline (Goodman 1999; Nederhof 2003) wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  37. Weighted parsing algorithm π‘’βˆˆ AST (𝐻 β€² ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) weighted parsing algorithm ? = 0 ) π‘Š(𝐡 β€² algorithm computation value ? = wt β€² (𝑒) βˆ‘ βŠ• Two-phase pipeline (Goodman 1999; Nederhof 2003) wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 16 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  38. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  39. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  40. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  41. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 17 / 21 { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  42. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  43. Canonical weighted deduction system Weight preserving 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) wt (𝑒) = wt β€² (πœ”(𝑒)) for every 𝑒 ∈ AST (𝐻, 𝑏) 2 Bijection πœ”: AST (𝐻, 𝑏) β†’ AST (𝐻 β€² ) 1 𝐡 β†’ 𝜏(𝐡 1 , … , 𝐡 𝑛 ) is a rule { [𝐡 , 𝑏 0 ] Parsing as deduction (Shieber, Schabes, and Pereira 1995) cwds 17 / 21 - wRTG-LM ((𝐻, β„’ ), 𝕃 , wt ) wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’ [𝐡 1 , 𝑏 1 ] … [𝐡 𝑛 , 𝑏 𝑛 ] 𝑏 0 , 𝑏 1 , … , 𝑏 𝑛 ∈ factors (𝑏) 𝑏 0 = 𝜏(𝑏 1 , … , 𝑏 𝑛 )

  44. Weighted parsing algorithm π‘’βˆˆ AST (𝐻 β€² ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) weighted parsing algorithm ? = 0 ) π‘Š(𝐡 β€² algorithm computation value ? = wt β€² (𝑒) βˆ‘ βŠ• Two-phase pipeline (Goodman 1999; Nederhof 2003) wRTG-LM system deduction weighted canonical wt (𝑒) π‘’βˆˆ AST (𝐻,𝑏) parse (𝑏) = βˆ‘ βŠ• - wRTG-LM 18 / 21 ((𝐻, β„’ ), 𝕃 , wt ) ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), 𝕃 , wt β€² ) - 𝑏 ∈ β„’

  45. Value computation algorithm 5: 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 12: until changed = false π‘Š(𝐡) ← π‘Š new 11: changed ← true 10: 9: 8: 7: 6: changed ← false π‘Š(𝐡) ← πŸ™ 0 , 𝑆 β€² ) Output: π‘Š(𝐡 β€² 0 ) 2: 19 / 21 3: repeat 4: Input: a wRTG-LM ((𝐻 β€² , π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , βŠ• , πŸ™ , 𝛻), wt β€² ) with 𝐻 β€² = (𝑂 β€² , 𝛡 β€² , 𝐡 β€² Variables: π‘Š: 𝑂 β€² β†’ 𝕃 , π‘Š new ∈ 𝕃 , changed ∈ 𝔺 1: for each 𝐡 ∈ 𝑂 β€² do for each 𝐡 ∈ 𝑂 β€² do π‘Š new ← πŸ™ for each 𝑠 = (𝐡 β†’ βŸ¨π‘¦ 1 … 𝑦 𝑛 ⟩(𝐡 1 , … , 𝐡 𝑛 )) in 𝑆 β€² do π‘Š new ← π‘Š new βŠ• wt β€² (𝑠)(π‘Š(𝐡 1 ), … , π‘Š(𝐡 𝑛 )) if π‘Š(𝐡) β‰  π‘Š new then

  46. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  47. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  48. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  49. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  50. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  51. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ A πœ• 3 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  52. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ … 𝜏 πŸ™ πŸ™ πŸ™ ( B A S S πœ€ 𝛿 𝛾 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ ) Value computation algorithm (example) πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) A βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) πœ• 3 βˆ’ βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt )

  53. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) = ( ) ↦ ( ) = ( ) ↦ … πœ€ πŸ™ πŸ™ ( B A S 𝜏 𝛾 𝛿 S πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ Value computation algorithm (example) πœ• 3 () 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ ) βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) A βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) πœ• 3 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ (

  54. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  55. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  56. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  57. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  58. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ↦ ( ) = ( ) ↦ … Value computation algorithm (example) 𝛿 πŸ™ ( B A S 𝜏 πœ€ 𝛾 S πŸ™ 𝛽 B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ πŸ™ 𝕝 1 πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 3 () πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) ) 𝕝 3 𝕝 2 βˆ’ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () βˆ’ βˆ’ βˆ’ S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S β†’ πœ€( B ) A 20 / 21 βˆ’ πœ• 5 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = (

  59. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) = ( ) ↦ … Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) ) 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 1 ) βŠ• πœ• 5 () πœ• 3 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 β†’ 𝛿( A ) A βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S πœ• 5 β†’ πœ€( B ) B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 πœ• 3 βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ B βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ (

  60. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  61. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  62. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  63. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  64. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) S πŸ™ ( B A S 𝜏 πœ€ 𝛿 𝛽 𝛾 πŸ™ B A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 1 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) ↦ … ) 3 𝕝 β€² 2 𝕝 β€² 𝕝 β€² πœ• 3 () 1 ) βŠ• πœ• 5 () 2 , 𝕝 β€² πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 βˆ’ βˆ’ βˆ’ 20 / 21 β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ πœ• 5 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ β†’ πœ€( B ) A πœ• 4 B β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = (

  65. ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) Value computation algorithm (example) 𝛾 ( B A S 𝜏 πœ€ 𝛿 S B 𝛽 πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πŸ™ πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πŸ™ 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πœ• 5 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () βˆ’ βˆ’ B β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ 20 / 21 A πœ• 3 β†’ 𝜏( A , S ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 4 B β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  66. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πœ• 5 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B πœ• 4 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  67. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πœ• 5 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 2 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B πœ• 4 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  68. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πœ• 5 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B πœ• 4 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ πœ• 3 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  69. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) πœ• 5 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B πœ• 4 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  70. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B πŸ™ A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( πŸ™ B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² πŸ™ πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  71. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () πœ• 1 ( πŸ™ ) βŠ• πœ• 2 ( πŸ™ ) 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  72. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () 0.8 β‹… 0 max 0.1 β‹… 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  73. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () πœ• 3 () 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  74. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 πœ• 4 ( 𝕝 2 , 𝕝 1 ) βŠ• πœ• 5 () 0.5 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  75. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 0.7 β‹… 0.5 β‹… 0 max 0.1 0.5 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  76. Value computation algorithm (example) 𝛽 B A S 𝜏 πœ€ 𝛿 S 𝛾 B 0 A β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ ( 0 B 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 𝕝 3 𝕝 2 𝕝 1 0.1 0.5 0 0.1 βˆ’ β†’ 𝜏( A , S ) β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ 20 / 21 β†’ πœ€( B ) β†’ 𝛽 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A B 0.7β‹… 𝕝 1 β‹… 𝕝 2 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) = ( ) ↦ ( ) = ( ) ↦ …

  77. Value computation algorithm (example) B S 𝜏 πœ€ 𝛿 S 𝛾 𝛽 A B β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A ( β†’ 𝜏( A , S ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () πœ• 1 ( 𝕝 2 ) βŠ• πœ• 2 ( 𝕝 3 ) 0.1 0.5 0 0 0 0.1 B βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.7β‹… 𝕝 1 β‹… 𝕝 2 A β†’ 𝛽 B βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) ↦ ( ) = ( ) ↦ …

  78. Value computation algorithm (example) B S 𝜏 πœ€ 𝛿 S 𝛾 𝛽 A B β†’ 𝛾 βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ A ( β†’ 𝜏( A , S ) 1 ) βŠ• πœ• 5 () 2019-09-25 Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 3 𝕝 β€² 2 𝕝 β€² 1 𝕝 β€² 2 , 𝕝 β€² 0 πœ• 4 ( 𝕝 β€² πœ• 3 () 0.8 β‹… 0.5 max 0.1 β‹… 0.1 0.1 0.5 0 0 0 0.1 B βˆ’ β†’ 𝛿( A ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.1β‹… 𝕝 1 S βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.8β‹… 𝕝 1 S βˆ’ βˆ’ β†’ πœ€( B ) βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.7β‹… 𝕝 1 β‹… 𝕝 2 A β†’ 𝛽 B βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ 0.5 20 / 21 ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( 𝕃 , πŸ™ , βŠ• , 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± 𝒣 βˆ… ), ( ℝ 1 0 , 0, max , 𝛻 mul ), wt ) ) ↦ ( ) ↦ ( ) = ( ) ↦ …

Recommend

TDT4205 Lecture 07 2 Parsing by recursive descent Take this grammar which models

TDT4205 Lecture 07 2 Parsing by recursive descent Take this grammar which models

1 Top-down parsing and LL(1) parser construction TDT4205 Lecture 07 2 Parsing by recursive descent Take this grammar which models ifs and whiles: P iCtSz | iCtSeSz | wCdSz C c S s Lets parse the

606 views β€’ 29 slides
Identifying Grammar Rules for Language Education with Dependency Parsing in German Eleni

Identifying Grammar Rules for Language Education with Dependency Parsing in German Eleni

Identifying Grammar Rules for Language Education with Dependency Parsing in German Eleni Metheniti, Pomi Park, Kristina Kolesova, Gnter Neumann August 27, 2019 Depling SyntaxFest Why identify grammar rules? Our larger mission: find

523 views β€’ 29 slides
CSE 447/547 Natural Language Processing Winter 2018 Dependency Parsing And Other Grammar

CSE 447/547 Natural Language Processing Winter 2018 Dependency Parsing And Other Grammar

CSE 447/547 Natural Language Processing Winter 2018 Dependency Parsing And Other Grammar Formalisms Yejin Choi - University of Washington Dependency Grammar For each word, find one parent. Child Parent A child is dependent on the

733 views β€’ 56 slides
CSE 490 U Natural Language Processing Spring 2016 Dependency Parsing And Other Grammar

CSE 490 U Natural Language Processing Spring 2016 Dependency Parsing And Other Grammar

CSE 490 U Natural Language Processing Spring 2016 Dependency Parsing And Other Grammar Formalisms Yejin Choi - University of Washington Dependency Grammar For each word, find one parent. Child Parent A child is dependent on the

814 views β€’ 55 slides
Tree-Adjoining Grammar Parsing and Vector Representations of Supertags Jungo Kasai Yale

Tree-Adjoining Grammar Parsing and Vector Representations of Supertags Jungo Kasai Yale

Background and Motivations Supertagging Models Parsing Models Vector Representations of Supertags Ongoing TAG Parsing W Tree-Adjoining Grammar Parsing and Vector Representations of Supertags Jungo Kasai Yale University December 14, 2017

799 views β€’ 62 slides
CSE 517 Natural Language Processing Winter 2017 Dependency Parsing And Other Grammar Formalisms

CSE 517 Natural Language Processing Winter 2017 Dependency Parsing And Other Grammar Formalisms

CSE 517 Natural Language Processing Winter 2017 Dependency Parsing And Other Grammar Formalisms Yejin Choi - University of Washington Dependency Grammar For each word, find one parent. Child Parent A child is dependent on the

1.83k views β€’ 55 slides
Natural Language Processing Lecture 13: More on CFG Parsing Probabilistjc/Weighted Parsing

Natural Language Processing Lecture 13: More on CFG Parsing Probabilistjc/Weighted Parsing

Natural Language Processing Lecture 13: More on CFG Parsing Probabilistjc/Weighted Parsing Example: ambiguous parse Probabilistjc CFG Ambiguous parse w/probabilitjes 0.05 0.05 0.20 0.10 0.30 0.20 0.20 0.30 0.20 0.20 0.60 0.60 0.75

1.05k views β€’ 68 slides
1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G ,

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G ,

1 Determinism and Parsing The parsing problem is, given a string w and a context-free grammar G , to decide if w L ( G ), and if so, to construct a derivation or a parse tree for w . Parsing is studied in courses in compilers. To be efficient

519 views β€’ 12 slides
Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency Parsing Dependency Parsing Pure dependency parsing is only cubic [Eisner 99] Lexicalized parsers can be seen as producing dependency trees

314 views β€’ 5 slides
Parsing beyond context-free grammar: necessarily adjacent. Range Concatenation Grammar Parsing

Parsing beyond context-free grammar: necessarily adjacent. Range Concatenation Grammar Parsing

Kallmeyer/Maier ESSLLI 2008 Kallmeyer/Maier ESSLLI 2008 Range Concatenation Grammar The idea behind range concatenation grammar (RCG) is comparable to the idea behind MCFG. Predicate-rewriting clauses describe ranges which are not Parsing

396 views β€’ 10 slides
Lagrangian Based Approaches for Lexicalized Tree Adjoining Grammar Parsing Caio Corro

Lagrangian Based Approaches for Lexicalized Tree Adjoining Grammar Parsing Caio Corro

Lagrangian Based Approaches for Lexicalized Tree Adjoining Grammar Parsing Caio Corro Supervision: Adeline Nazarenko & Joseph Le Roux 9 march 2018 1 / 51 Syntax: description of structures in natural languages She walks the woman dog

1.77k views β€’ 162 slides
Advanced Natural Language Processing Guest Lecture: Modeling Human Parsing Frank Keller

Advanced Natural Language Processing Guest Lecture: Modeling Human Parsing Frank Keller

Introduction Prediction and Grammar Predictive Parsing Evaluation Advanced Natural Language Processing Guest Lecture: Modeling Human Parsing Frank Keller Institute for Language, Cognition and Computation School of Informatics, University of

683 views β€’ 47 slides
CSE 3341: Principles of Programming Languages Recursive Descent Parsing Jeremy Morris 1

CSE 3341: Principles of Programming Languages Recursive Descent Parsing Jeremy Morris 1

CSE 3341: Principles of Programming Languages Recursive Descent Parsing Jeremy Morris 1 Parsing A grammar is a generator for a language The rules tell us how to create strings in the language A parser is a recognizer for a language

798 views β€’ 32 slides
Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E

Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E

Top-Down Parsing Top-Down Parsing #1 Extra Credit Question Given this grammar G: E E + T E T T T * int T int T ( E ) Is the string int * (int + int) in L(G)? Give a derivation or prove that it

1.08k views β€’ 61 slides
Statistical Natural Language Parsing Gerald Penn [based on slides by Christopher Manning]

Statistical Natural Language Parsing Gerald Penn [based on slides by Christopher Manning]

Statistical Natural Language Parsing Gerald Penn [based on slides by Christopher Manning] Parsing examples Fed raises interest rates Parsing in the early 1990s The parsers produced detailed, linguistically rich representations

849 views β€’ 46 slides
Efficient parsing with a large-scale unification-based grammar Lessons from a multi-year,

Efficient parsing with a large-scale unification-based grammar Lessons from a multi-year,

0. Efficient parsing with a large-scale unification-based grammar Lessons from a multi-year, multi-team endeavour Liviu Ciortuz Department of Computer Science University of Iasi, Romania ALEAR Workshop, FP7 E.U. Project Humbold

947 views β€’ 76 slides
SI425 : NLP Set 7 Syntax and Parsing Syntax Grammar, or syntax: The kind of implicit

SI425 : NLP Set 7 Syntax and Parsing Syntax Grammar, or syntax: The kind of implicit

SI425 : NLP Set 7 Syntax and Parsing Syntax Grammar, or syntax: The kind of implicit knowledge of your native language that you had mastered by the time you were 3 years old Not the kind of stuff you were later taught in

677 views β€’ 57 slides
SI485i : NLP Set 7 Syntax and Parsing Syntax Grammar, or syntax: The kind of implicit

SI485i : NLP Set 7 Syntax and Parsing Syntax Grammar, or syntax: The kind of implicit

SI485i : NLP Set 7 Syntax and Parsing Syntax Grammar, or syntax: The kind of implicit knowledge of your native language that you had mastered by the time you were 3 years old Not the kind of stuff you were later taught in

963 views β€’ 56 slides
Natural Language Processing Learning PCFGs Parsing II Dan Klein UC Berkeley Treebank PCFGs

Natural Language Processing Learning PCFGs Parsing II Dan Klein UC Berkeley Treebank PCFGs

Natural Language Processing Learning PCFGs Parsing II Dan Klein UC Berkeley Treebank PCFGs Conditional Independence? [Charniak 96] Use PCFGs for broad coverage parsing Can take a grammar right off the trees (doesnt work well): ROOT

224 views β€’ 10 slides
Natural Language Processing Parsing II Dan Klein UC Berkeley 1 Learning PCFGs 2 Treebank

Natural Language Processing Parsing II Dan Klein UC Berkeley 1 Learning PCFGs 2 Treebank

Natural Language Processing Parsing II Dan Klein UC Berkeley 1 Learning PCFGs 2 Treebank PCFGs [Charniak 96] Use PCFGs for broad coverage parsing Can take a grammar right off the trees (doesnt work well): ROOT S 1 S NP

586 views β€’ 56 slides
Evaluating grammar formalisms for applications to natural language processing and biological

Evaluating grammar formalisms for applications to natural language processing and biological

Evaluating grammar formalisms for applications to natural language processing and biological sequence analysis David Chiang 28 June 2004 Applications of grammars Statistical parsing (Charniak, 1997; Collins, 1997) Language modeling

766 views β€’ 61 slides
Grammar-based Specification and Parsing for Binary File Formats William Underwood Georgia Tech

Grammar-based Specification and Parsing for Binary File Formats William Underwood Georgia Tech

Grammar-based Specification and Parsing for Binary File Formats William Underwood Georgia Tech Research Institute Atlanta, Georgia, USA 7 th International Digital Curation Conference 2011 Bristol, UK December 5-7, 2011 GTRI_B-#

400 views β€’ 26 slides
SI425 : NLP Set 10 Syntax and Parsing Fall 2020 : Chambers Syntax Grammar, or syntax:

SI425 : NLP Set 10 Syntax and Parsing Fall 2020 : Chambers Syntax Grammar, or syntax:

SI425 : NLP Set 10 Syntax and Parsing Fall 2020 : Chambers Syntax Grammar, or syntax: The kind of implicit knowledge of your native language that you had mastered by the time you were 3 years old Not the kind of stuff you were later

854 views β€’ 57 slides
Grammar formalisms Tree Adjoining Grammar: Formal Properties, Part I Parsing Formal Properties

Grammar formalisms Tree Adjoining Grammar: Formal Properties, Part I Parsing Formal Properties

Some sample languages Pumping Lemma for TAL Closure Properties Grammar formalisms Tree Adjoining Grammar: Formal Properties, Part I Parsing Formal Properties of TAG Laura Kallmeyer, Timm Lichte, Wolfgang Maier Universit at T ubingen

260 views β€’ 14 slides

More recommend