Weighted parsing for grammar-based language models FSMNLP 2019 - - PowerPoint PPT Presentation

β–Ά
weighted parsing for grammar based language models
SMART_READER_LITE
LIVE PREVIEW

Weighted parsing for grammar-based language models FSMNLP 2019 - - PowerPoint PPT Presentation

Feb 23, 2024 353 likes β€’1.61k views

Weighted parsing for grammar-based language models FSMNLP 2019 Richard Mrbitz Heiko Vogler 2019-09-25 The weighted parsing problem language e.g., English sentences ( ) syntactic object e.g., Fruit fmies like bananas ?

slide-1
SLIDE 1

Weighted parsing for grammar-based language models

FSMNLP 2019 Richard MΓΆrbitz Heiko Vogler 2019-09-25

slide-2
SLIDE 2

The weighted parsing problem

language

e.g., English sentences (βŠ† π›¦βˆ—)

syntactic object

e.g., Fruit fmies like bananas

∈ ?

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 2 / 21

slide-3
SLIDE 3

The weighted parsing problem

language

e.g., English sentences (βŠ† π›¦βˆ—)

syntactic object

e.g., Fruit fmies like bananas

∈ ? language model

e.g., context-free grammar (CFG)

structural representations

e.g., abstract syntax trees (ASTs)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 2 / 21

slide-4
SLIDE 4

The weighted parsing problem

language

e.g., English sentences (βŠ† π›¦βˆ—)

syntactic object

e.g., Fruit fmies like bananas

∈ ? language model

e.g., context-free grammar (CFG)

structural representations

e.g., abstract syntax trees (ASTs)

value in a weight algebra

(Goodman 1999; Nederhof 2003)

parse

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 2 / 21

slide-5
SLIDE 5

Overview

Semiring parsing

(Goodman 1999)

Weighted deductive parsing

(Nederhof 2003)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 3 / 21

slide-6
SLIDE 6

Overview

Semiring parsing

(Goodman 1999)

Weighted deductive parsing

(Nederhof 2003)

Mohri (2002)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 3 / 21

slide-7
SLIDE 7

Overview

Semiring parsing

(Goodman 1999)

Weighted deductive parsing

(Nederhof 2003)

Mohri (2002) Algebraic dynamic programming

(Giegerich, Meyer, and Stefgen 2004)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 3 / 21

slide-8
SLIDE 8

Overview

Semiring parsing

(Goodman 1999)

Weighted deductive parsing

(Nederhof 2003)

Mohri (2002) Algebraic dynamic programming

(Giegerich, Meyer, and Stefgen 2004)

Our approach

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 3 / 21

slide-9
SLIDE 9

Outline

1

Weighted RTG-based language models

2

The weighted parsing problem

3

The weighted parsing algorithm

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 4 / 21

slide-10
SLIDE 10

Outline

1

Weighted RTG-based language models

2

The weighted parsing problem

3

The weighted parsing algorithm

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 5 / 21

slide-11
SLIDE 11

Regular tree grammars (RTG)

Tuple 𝐻 = (𝑂, 𝛡, 𝐡0, 𝑆) Example rules: S β†’ 𝛽(NP, VP) VP β†’ 𝛾(VBZ, PP) NP β†’ 𝛿(NN) NN β†’ πœ€ … S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) … abstract syntax tree 𝑒 ∈ AST(𝐻)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 6 / 21

slide-12
SLIDE 12

Regular tree grammars (RTG)

Tuple 𝐻 = (𝑂, 𝛡, 𝐡0, 𝑆) Example rules: S β†’ 𝛽(NP, VP) VP β†’ 𝛾(VBZ, PP) NP β†’ 𝛿(NN) NN β†’ πœ€ … S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) … abstract syntax tree 𝑒 ∈ AST(𝐻)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 6 / 21

slide-13
SLIDE 13

Regular tree grammars (RTG)

Tuple 𝐻 = (𝑂, 𝛡, 𝐡0, 𝑆) Example rules: S β†’ 𝛽(NP, VP) VP β†’ 𝛾(VBZ, PP) NP β†’ 𝛿(NN) NN β†’ πœ€ … S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) … abstract syntax tree 𝑒 ∈ AST(𝐻) 𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 6 / 21

slide-14
SLIDE 14

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-15
SLIDE 15

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-16
SLIDE 16

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

⟨Fruit⟩

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-17
SLIDE 17

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

⟨Fruit⟩ βŸ¨π‘¦1⟩

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-18
SLIDE 18

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

⟨Fruit⟩ βŸ¨π‘¦1⟩ βŸ¨π‘¦1𝑦2⟩

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-19
SLIDE 19

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

⟨Fruit⟩ βŸ¨π‘¦1⟩ βŸ¨π‘¦1𝑦2⟩ βŸ¨π‘¦1 flies 𝑦2⟩

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-20
SLIDE 20

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

⟨Fruit⟩ βŸ¨π‘¦1⟩ βŸ¨π‘¦1𝑦2⟩ βŸ¨π‘¦1 flies 𝑦2⟩

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-21
SLIDE 21

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 Fruit 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

βŸ¨π‘¦1⟩ βŸ¨π‘¦1𝑦2⟩ βŸ¨π‘¦1 flies 𝑦2⟩

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-22
SLIDE 22

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 Fruit Fruit 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

βŸ¨π‘¦1𝑦2⟩ βŸ¨π‘¦1 flies 𝑦2⟩

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-23
SLIDE 23

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 Fruit Fruit …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

βŸ¨π‘¦1 flies 𝑦2⟩

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-24
SLIDE 24

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

⟨Fruit flies … ⟩ Fruit Fruit …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-25
SLIDE 25

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

⟨Fruit flies … ⟩ Fruit Fruit …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-26
SLIDE 26

Language algebras

interpretation of 𝛡 as operations on the set of syntactic objects β„’ = π›¦βˆ— (.)π›¦βˆ—: T𝛡 (terms) β†’ π›¦βˆ— (syntactic objects) factors(Fruit flies like bananas) = {Fruit, like bananas, … } S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅: T𝑆 β†’ T𝛡

𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Fruit flies …

(.)π›¦βˆ—: T𝛡 β†’ π›¦βˆ—

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 7 / 21

slide-27
SLIDE 27

Semirings

Algebraic structure (𝕃, βŠ•, βŠ—, πŸ™, 𝟚) βŠ— is used to evaluate an AST to a weight βŠ• accumulates the weights of several ASTs Examples (𝔺, ∨, ∧, false, true) the Boolean semiring with 𝔺 = {false, true} (β„•βˆž, +, β‹…, 0, 1) the semiring of natural numbers (β„•βˆž, min, +, ∞, 0) the tropical semiring (ℝ1

0, max, β‹…, 0, 1) the Viterbi semiring

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 8 / 21

slide-28
SLIDE 28

Semirings

Algebraic structure (𝕃, βŠ•, βŠ—, πŸ™, 𝟚) βŠ— is used to evaluate an AST to a weight βŠ• accumulates the weights of several ASTs Examples (𝔺, ∨, ∧, false, true) the Boolean semiring with 𝔺 = {false, true} (β„•βˆž, +, β‹…, 0, 1) the semiring of natural numbers (β„•βˆž, min, +, ∞, 0) the tropical semiring (ℝ1

0, max, β‹…, 0, 1) the Viterbi semiring

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 8 / 21

slide-29
SLIDE 29

Multioperator monoids (M-monoids)

Generalization of semirings (𝕃, βŠ•, βŠ—, πŸ™, 𝟚) ⟢ (𝕃, βŠ•, πŸ™, 𝛻) binary βŠ— ⟢ set of 𝑛-ary operations 𝛻 (here: distributive) Semiring (𝕃, βŠ•, βŠ—, πŸ™, 𝟚) M-monoid (𝕃, βŠ•, πŸ™, π›»βŠ—) where π›»βŠ— = {mul

(𝑛) 𝕝

∣ 𝕝 ∈ 𝕃, 𝑛 ∈ β„•} mul

(𝑛) 𝕝 (𝕝1, … , 𝕝𝑛) = 𝕝 βŠ— 𝕝1 βŠ— β‹― βŠ— 𝕝𝑛

Examples Viterbi M-monoid (ℝ1

0, max, 0, 𝛻mul)

Minimum edit distance M-monoid ({{π‘œ} ∣ π‘œ ∈ β„•}, min ∘ βˆͺ, βˆ…, 𝛻med) with 𝛻med = {del, ins, rep=, repβ‰ , nil}

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 9 / 21

slide-30
SLIDE 30

Multioperator monoids (M-monoids)

Generalization of semirings (𝕃, βŠ•, βŠ—, πŸ™, 𝟚) ⟢ (𝕃, βŠ•, πŸ™, 𝛻) binary βŠ— ⟢ set of 𝑛-ary operations 𝛻 (here: distributive) Semiring (𝕃, βŠ•, βŠ—, πŸ™, 𝟚) ⇝ M-monoid (𝕃, βŠ•, πŸ™, π›»βŠ—) where π›»βŠ— = {mul

(𝑛) 𝕝

∣ 𝕝 ∈ 𝕃, 𝑛 ∈ β„•} mul

(𝑛) 𝕝 (𝕝1, … , 𝕝𝑛) = 𝕝 βŠ— 𝕝1 βŠ— β‹― βŠ— 𝕝𝑛

Examples Viterbi M-monoid (ℝ1

0, max, 0, 𝛻mul)

Minimum edit distance M-monoid ({{π‘œ} ∣ π‘œ ∈ β„•}, min ∘ βˆͺ, βˆ…, 𝛻med) with 𝛻med = {del, ins, rep=, repβ‰ , nil}

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 9 / 21

slide-31
SLIDE 31

Multioperator monoids (M-monoids)

Generalization of semirings (𝕃, βŠ•, βŠ—, πŸ™, 𝟚) ⟢ (𝕃, βŠ•, πŸ™, 𝛻) binary βŠ— ⟢ set of 𝑛-ary operations 𝛻 (here: distributive) Semiring (𝕃, βŠ•, βŠ—, πŸ™, 𝟚) ⇝ M-monoid (𝕃, βŠ•, πŸ™, π›»βŠ—) where π›»βŠ— = {mul

(𝑛) 𝕝

∣ 𝕝 ∈ 𝕃, 𝑛 ∈ β„•} mul

(𝑛) 𝕝 (𝕝1, … , 𝕝𝑛) = 𝕝 βŠ— 𝕝1 βŠ— β‹― βŠ— 𝕝𝑛

Examples Viterbi M-monoid (ℝ1

0, max, 0, 𝛻mul)

Minimum edit distance M-monoid ({{π‘œ} ∣ π‘œ ∈ β„•}, min ∘ βˆͺ, βˆ…, 𝛻med) with 𝛻med = {del, ins, rep=, repβ‰ , nil}

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 9 / 21

slide-32
SLIDE 32

Weight algebras

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Fruit flies …

(.)π›¦βˆ—

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

wt: 𝑆 (set of rules) β†’ 𝛻 (set of operations)

(ℝ1

0, max, 0, 𝛻mul)

(.)ℝ0

1: T𝛻 (terms) β†’ ℝ1

0 (weight algebra)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 10 / 21

slide-33
SLIDE 33

Weight algebras

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Fruit flies …

(.)π›¦βˆ—

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

wt: 𝑆 (set of rules) β†’ 𝛻 (set of operations)

(ℝ1

0, max, 0, 𝛻mul)

(.)ℝ0

1: T𝛻 (terms) β†’ ℝ1

0 (weight algebra)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 10 / 21

slide-34
SLIDE 34

Weight algebras

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Fruit flies …

(.)π›¦βˆ—

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

wt: 𝑆 (set of rules) β†’ 𝛻 (set of operations)

(ℝ1

0, max, 0, 𝛻mul)

(.)ℝ0

1: T𝛻 (terms) β†’ ℝ1

0 (weight algebra)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 10 / 21

slide-35
SLIDE 35

Weight algebras

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Fruit flies …

(.)π›¦βˆ—

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

wt: 𝑆 (set of rules) β†’ 𝛻 (set of operations)

(ℝ1

0, max, 0, 𝛻mul)

(.)ℝ0

1: T𝛻 (terms) β†’ ℝ1

0 (weight algebra)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 10 / 21

slide-36
SLIDE 36

Weight algebras

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Fruit flies …

(.)π›¦βˆ—

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

wt: 𝑆 (set of rules) β†’ 𝛻 (set of operations)

(ℝ1

0, max, 0, 𝛻mul)

(.)ℝ0

1: T𝛻 (terms) β†’ ℝ1

0 (weight algebra)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 10 / 21

slide-37
SLIDE 37

Weight algebras

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Fruit flies …

(.)π›¦βˆ—

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 1.0 0.6 β‹… …

wt wt(𝑒) ∈ T𝛻

wt: 𝑆 (set of rules) β†’ 𝛻 (set of operations)

(ℝ1

0, max, 0, 𝛻mul)

(.)ℝ0

1: T𝛻 (terms) β†’ ℝ1

0 (weight algebra)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 10 / 21

slide-38
SLIDE 38

Weight algebras

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Fruit flies …

(.)π›¦βˆ—

0.12 β‹… … 0.2 1.0 0.6 β‹… …

wt wt(𝑒) ∈ T𝛻

wt: 𝑆 (set of rules) β†’ 𝛻 (set of operations)

(ℝ1

0, max, 0, 𝛻mul)

(.)ℝ0

1: T𝛻 (terms) β†’ ℝ1

0 (weight algebra)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 10 / 21

slide-39
SLIDE 39

Weight algebras

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

Fruit flies …

(.)π›¦βˆ—

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

wt: 𝑆 (set of rules) β†’ 𝛻 (set of operations)

(ℝ1

0, max, 0, 𝛻mul)

(.)ℝ0

1: T𝛻 (terms) β†’ ℝ1

0 (weight algebra)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 10 / 21

slide-40
SLIDE 40

Weighted RTG-based language models

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

Fruit flies …

(.)π›¦βˆ—

Defjnition (weighted RTG-based language model)

A wRTG-LM is a tuple ( (𝐻 = (𝑂, 𝛡, 𝐡0, 𝑆)) ⏟⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟

RTG

, β„’ ⏟

language algebra

), (𝕃, βŠ•, πŸ™, 𝛻) ⏟

M-monoid

, wt ⏟

wt: 𝑆 β†’ 𝛻

).

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 11 / 21

slide-41
SLIDE 41

Weighted RTG-based language models

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

Fruit flies …

(.)π›¦βˆ—

Defjnition (weighted RTG-based language model)

A wRTG-LM is a tuple ( (𝐻 = (𝑂, 𝛡, 𝐡0, 𝑆)) ⏟⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟

RTG

, β„’ ⏟

language algebra

), (𝕃, βŠ•, πŸ™, 𝛻) ⏟

M-monoid

, wt ⏟

wt: 𝑆 β†’ 𝛻

).

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 11 / 21

slide-42
SLIDE 42

Outline

1

Weighted RTG-based language models

2

The weighted parsing problem

3

The weighted parsing algorithm

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 12 / 21

slide-43
SLIDE 43

The weighted parsing problem

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

Fruit flies …

(.)π›¦βˆ—

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 13 / 21

slide-44
SLIDE 44

The weighted parsing problem

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

Fruit flies …

(.)π›¦βˆ—

𝑒′ ∈ AST(𝐻) 𝑒′ ∈ T𝛡

πœŒπ›΅ (.)π›¦βˆ—

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 13 / 21

slide-45
SLIDE 45

The weighted parsing problem

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

Fruit flies …

(.)π›¦βˆ—

𝑒′ ∈ AST(𝐻) 𝑒′ ∈ T𝛡

πœŒπ›΅ (.)π›¦βˆ—

wt(𝑒′) ∈ T𝛻

wt

0.0144

(.)ℝ1

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 13 / 21

slide-46
SLIDE 46

The weighted parsing problem

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

Fruit flies …

(.)π›¦βˆ—

𝑒′ ∈ AST(𝐻) 𝑒′ ∈ T𝛡

πœŒπ›΅ (.)π›¦βˆ—

wt(𝑒′) ∈ T𝛻

wt

0.0144

(.)ℝ1

(ℝ1

0, max, 0, 𝛻mul)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 13 / 21

slide-47
SLIDE 47

The weighted parsing problem

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

Fruit flies …

(.)π›¦βˆ—

𝑒′ ∈ AST(𝐻) 𝑒′ ∈ T𝛡

πœŒπ›΅ (.)π›¦βˆ—

wt(𝑒′) ∈ T𝛻

wt

0.0144

(.)ℝ1

(ℝ1

0, max, 0, 𝛻mul)

max {

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 13 / 21

slide-48
SLIDE 48

The weighted parsing problem

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

Fruit flies …

(.)π›¦βˆ—

𝑒′ ∈ AST(𝐻) 𝑒′ ∈ T𝛡

πœŒπ›΅ (.)π›¦βˆ—

wt(𝑒′) ∈ T𝛻

wt

0.0144

(.)ℝ1

(ℝ1

0, max, 0, 𝛻mul)

max {

parse

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 13 / 21

slide-49
SLIDE 49

The weighted parsing problem

S β†’ 𝛽(NP, VP) NP β†’ 𝛿(NN) NN β†’ πœ€ VP β†’ 𝛾(VBZ, PP) …

𝑒 ∈ AST(𝐻)

𝛽 𝛿 πœ€ 𝛾 …

πœŒπ›΅ 𝑒 ∈ 𝑀(𝐻) βŠ† T𝛡

1.0 β‹… 𝕝1 β‹… 𝕝2 0.2 β‹… 𝕝1 1.0 0.6 β‹… 𝕝1 β‹… 𝕝2 …

wt wt(𝑒) ∈ T𝛻

0.12 β‹… …

(.)ℝ1

Fruit flies …

(.)π›¦βˆ—

𝑒′ ∈ AST(𝐻) 𝑒′ ∈ T𝛡

πœŒπ›΅ (.)π›¦βˆ—

wt(𝑒′) ∈ T𝛻

wt

0.0144

(.)ℝ1

(ℝ1

0, max, 0, 𝛻mul)

max {

parse

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 13 / 21

slide-50
SLIDE 50

The weighted parsing problem

Examples Semiring parsing (Goodman 1999)

recognition string probability probability of best derivation derivation forest best derivation(s) π‘œ best derivation(s)

Parsing with superior grammars (Knuth 1977; Nederhof 2003) Algebraic dynamic programming (Giegerich, Meyer, and Stefgen 2004)

minimum edit distance matrix chain multiplication

Reduct of a grammar and a syntactic object (cf. Bar-Hillel, Perles, and Shamir 1961)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 14 / 21

slide-51
SLIDE 51

Outline

1

Weighted RTG-based language models

2

The weighted parsing problem

3

The weighted parsing algorithm

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 15 / 21

slide-52
SLIDE 52

Weighted parsing algorithm

Two-phase pipeline (Goodman 1999; Nederhof 2003)

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

slide-53
SLIDE 53

Weighted parsing algorithm

Two-phase pipeline (Goodman 1999; Nederhof 2003)

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

slide-54
SLIDE 54

Weighted parsing algorithm

Two-phase pipeline (Goodman 1999; Nederhof 2003)

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒) canonical weighted deduction system

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

slide-55
SLIDE 55

Weighted parsing algorithm

Two-phase pipeline (Goodman 1999; Nederhof 2003)

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

slide-56
SLIDE 56

Weighted parsing algorithm

Two-phase pipeline (Goodman 1999; Nederhof 2003)

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

?

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

slide-57
SLIDE 57

Weighted parsing algorithm

Two-phase pipeline (Goodman 1999; Nederhof 2003)

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

? value computation algorithm

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

slide-58
SLIDE 58

Weighted parsing algorithm

Two-phase pipeline (Goodman 1999; Nederhof 2003)

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

? value computation algorithm π‘Š(𝐡′

0)

=

?

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

slide-59
SLIDE 59

Weighted parsing algorithm

Two-phase pipeline (Goodman 1999; Nederhof 2003)

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

? value computation algorithm π‘Š(𝐡′

0)

=

? weighted parsing algorithm

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 16 / 21

slide-60
SLIDE 60

Canonical weighted deduction system

  • wRTG-LM ((𝐻, β„’), 𝕃, wt )
  • 𝑏 ∈ β„’

wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

cwds

Parsing as deduction (Shieber, Schabes, and Pereira 1995) [𝐡1 , 𝑏1] … [𝐡𝑛 , 𝑏𝑛] [𝐡 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 17 / 21

slide-61
SLIDE 61

Canonical weighted deduction system

  • wRTG-LM ((𝐻, β„’), 𝕃, wt )
  • 𝑏 ∈ β„’

wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

cwds

Parsing as deduction (Shieber, Schabes, and Pereira 1995) [𝐡1 , 𝑏1] … [𝐡𝑛 , 𝑏𝑛] [𝐡 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 17 / 21

slide-62
SLIDE 62

Canonical weighted deduction system

  • wRTG-LM ((𝐻, β„’), 𝕃, wt )
  • 𝑏 ∈ β„’

wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

cwds

Parsing as deduction (Shieber, Schabes, and Pereira 1995) [𝐡1 , 𝑏1] … [𝐡𝑛 , 𝑏𝑛] [𝐡 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 17 / 21

slide-63
SLIDE 63

Canonical weighted deduction system

  • wRTG-LM ((𝐻, β„’), 𝕃, wt )
  • 𝑏 ∈ β„’

wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

cwds

Parsing as deduction (Shieber, Schabes, and Pereira 1995) [𝐡1 , 𝑏1] … [𝐡𝑛 , 𝑏𝑛] [𝐡 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 17 / 21

slide-64
SLIDE 64

Canonical weighted deduction system

  • wRTG-LM ((𝐻, β„’), 𝕃, wt )
  • 𝑏 ∈ β„’

wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

cwds

Parsing as deduction (Shieber, Schabes, and Pereira 1995) [𝐡1 , 𝑏1] … [𝐡𝑛 , 𝑏𝑛] [𝐡 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 17 / 21

slide-65
SLIDE 65

Canonical weighted deduction system

  • wRTG-LM ((𝐻, β„’), 𝕃, wt )
  • 𝑏 ∈ β„’

wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

cwds

Parsing as deduction (Shieber, Schabes, and Pereira 1995) [𝐡1 , 𝑏1] … [𝐡𝑛 , 𝑏𝑛] [𝐡 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 17 / 21

slide-66
SLIDE 66

Weighted parsing algorithm

Two-phase pipeline (Goodman 1999; Nederhof 2003)

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) = βˆ‘βŠ•

π‘’βˆˆAST(𝐻,𝑏)

wt(𝑒) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

? value computation algorithm π‘Š(𝐡′

0)

=

? weighted parsing algorithm

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 18 / 21

slide-67
SLIDE 67

Value computation algorithm

Input: a wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), (𝕃, βŠ•, πŸ™, 𝛻), wtβ€² ) with 𝐻′ = (𝑂′, 𝛡′, 𝐡′

0, 𝑆′)

Variables: π‘Š: 𝑂′ β†’ 𝕃, π‘Šnew ∈ 𝕃, changed ∈ 𝔺 Output: π‘Š(𝐡′

0)

1: for each 𝐡 ∈ 𝑂′ do 2:

π‘Š(𝐡) ← πŸ™

3: repeat 4:

changed ← false

5:

for each 𝐡 ∈ 𝑂′ do

6:

π‘Šnew ← πŸ™

7:

for each 𝑠 = (𝐡 β†’ βŸ¨π‘¦1 … π‘¦π‘›βŸ©(𝐡1, … , 𝐡𝑛)) in 𝑆′ do

8:

π‘Šnew ← π‘Šnew βŠ• wtβ€²(𝑠)(π‘Š(𝐡1), … , π‘Š(𝐡𝑛))

9:

if π‘Š(𝐡) β‰  π‘Šnew then

10:

changed ← true

11:

π‘Š(𝐡) ← π‘Šnew

12: until changed = false

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 19 / 21

slide-68
SLIDE 68

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-69
SLIDE 69

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-70
SLIDE 70

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-71
SLIDE 71

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-72
SLIDE 72

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-73
SLIDE 73

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-74
SLIDE 74

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-75
SLIDE 75

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-76
SLIDE 76

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-77
SLIDE 77

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-78
SLIDE 78

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-79
SLIDE 79

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-80
SLIDE 80

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-81
SLIDE 81

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-82
SLIDE 82

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-83
SLIDE 83

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-84
SLIDE 84

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-85
SLIDE 85

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-86
SLIDE 86

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-87
SLIDE 87

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-88
SLIDE 88

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

πœ•1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-89
SLIDE 89

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

πœ•2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-90
SLIDE 90

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

πœ•3

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-91
SLIDE 91

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

πœ•4

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

πœ•5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-92
SLIDE 92

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( πŸ™ πŸ™ πŸ™ ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-93
SLIDE 93

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( πœ•1(πŸ™) βŠ• πœ•2(πŸ™) πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-94
SLIDE 94

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.8 β‹… 0 max 0.1 β‹… 0 πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-95
SLIDE 95

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( πœ•3() πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-96
SLIDE 96

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 πœ•4(𝕝2, 𝕝1) βŠ• πœ•5() ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-97
SLIDE 97

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.7 β‹… 0.5 β‹… 0 max 0.1 ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-98
SLIDE 98

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.1 ) = ( 𝕝1 𝕝2 𝕝3 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-99
SLIDE 99

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.1 ) ↦ ( πœ•1(𝕝2) βŠ• πœ•2(𝕝3) πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-100
SLIDE 100

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.1 ) ↦ ( 0.8 β‹… 0.5 max 0.1 β‹… 0.1 πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-101
SLIDE 101

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.1 ) ↦ ( 0.4 πœ•3() πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-102
SLIDE 102

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.1 ) ↦ ( 0.4 0.5 πœ•4(𝕝′

2, 𝕝′ 1) βŠ• πœ•5()

) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-103
SLIDE 103

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.1 ) ↦ ( 0.4 0.5 0.7 β‹… 0.5 β‹… 0.4 max 0.1 ) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-104
SLIDE 104

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.1 ) ↦ ( 0.4 0.5 0.14 ) = ( 𝕝′

1

𝕝′

2

𝕝′

3

) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-105
SLIDE 105

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.1 ) ↦ ( 0.4 0.5 0.14 ) ↦ …

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-106
SLIDE 106

Value computation algorithm (example)

((𝐻, π’Ÿβ„± π’£βˆ…), (𝕃, πŸ™, βŠ•, 𝛻), wt ) ⇝ ((𝐻, π’Ÿβ„± π’£βˆ…), (ℝ1

0, 0, max, 𝛻mul), wt )

S

0.8⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛿(A) S

0.1⋅𝕝1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ πœ€(B) A

0.5

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛽 B

0.7⋅𝕝1⋅𝕝2

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝜏(A, S) B

0.1

βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ βˆ’ β†’ 𝛾 A B

𝛽 𝛾

S

𝛿 πœ€ 𝜏

S A B ( ) ↦ ( 0.5 0.1 ) ↦ ( 0.4 0.5 0.14 ) ↦ ( 0.4 0.5 0.14 )

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 20 / 21

slide-107
SLIDE 107

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

? value computation algorithm π‘Š(𝐡0

β€²)

=

?

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-108
SLIDE 108

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

? value computation algorithm π‘Š(𝐡0

β€²)

=

closed

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-109
SLIDE 109

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

value computation algorithm π‘Š(𝐡0

β€²)

=

closed weight preserving

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-110
SLIDE 110

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

value computation algorithm π‘Š(𝐡0

β€²)

=

closed weight preserving

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-111
SLIDE 111

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

value computation algorithm π‘Š(𝐡0

β€²)

=

closed weight preserving

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-112
SLIDE 112

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

value computation algorithm π‘Š(𝐡0

β€²)

=

closed weight preserving

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-113
SLIDE 113

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

value computation algorithm π‘Š(𝐡0

β€²)

=

closed weight preserving

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-114
SLIDE 114

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

value computation algorithm π‘Š(𝐡0

β€²)

=

closed weight preserving

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-115
SLIDE 115

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

value computation algorithm π‘Š(𝐡0

β€²)

=

closed weight preserving

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Goodman (1999)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-116
SLIDE 116

Termination and correctness

  • wRTG-LM

((𝐻, β„’), 𝕃, wt )

  • 𝑏 ∈ β„’

parse(𝑏) canonical weighted deduction system wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² )

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wtβ€²(𝑒)

=

value computation algorithm π‘Š(𝐡0

β€²)

=

closed weight preserving

Conditions

Suffjcient: ((𝐻, β„’), 𝕃, wt ) is closed or nonlooping e.g., acyclic RTGs, superior M-monoids, algebraic dynamic programming β„’ is fjnitely decomposable e.g., CFG, LCFRS, TAG

Goodman (1999) Nederhof (2003)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 21 / 21

slide-117
SLIDE 117

References I

  • Y. Bar-Hillel, M. Perles, and E. Shamir (1961). β€œOn Formal Properties of Simple

Phrase Structure Grammars”. Zeitschrift fΓΌr Phonetik, Sprachwissenschaft und

  • Kommunikationsforschung. Reprinted in Y. Bar-Hillel. (1964). Language and

Information: Selected Essays on their Theory and Application, Addison-Wesley 1964, 116–150.

  • R. Giegerich, C. Meyer, and P. Stefgen (2004). β€œA discipline of dynamic

programming over sequence data”. Science of Computer Programming.

  • J. Goodman (1999). β€œSemiring Parsing”. Computational Linguistics, 4.
  • D. E. Knuth (1977). β€œA generalization of Dijkstra’s algorithm”. Information

Processing Letters.

  • M. Mohri (2002). β€œSemiring frameworks and algorithms for shortest-distance

problems”. Journal of Automata, Languages and Combinatorics. M.-J. Nederhof (2003). β€œSquibs and Discussions: Weighted deductive parsing and Knuth’s algorithm”. Computational Linguistics.

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 1 / 5

slide-118
SLIDE 118

References II

  • S. Shieber, Y. Schabes, and F. Pereira (1995). β€œPrinciples and implementation of

deductive parsing”. The Journal of Logic Programming.

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 2 / 5

slide-119
SLIDE 119

Canonical weighted deduction system

wRTG-LM ((𝐻, β„’), 𝕃, wt ) and 𝑏 ∈ β„’ ⇝ wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² ) [𝐡1 , 𝜏1 , 𝑏1] … [𝐡𝑛 , πœπ‘› , 𝑏𝑛] [𝐡 , 𝜏 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) 𝐡1 β†’ 𝜏1(… ), … , 𝐡𝑛 β†’ πœπ‘›(… ) are rules [𝐡0, 𝜏, 𝑏] [𝐡0, 𝑏] {𝐡0 β†’ 𝜏(… ) is a rule Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 3 / 5

slide-120
SLIDE 120

Canonical weighted deduction system

wRTG-LM ((𝐻, β„’), 𝕃, wt ) and 𝑏 ∈ β„’ ⇝ wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² ) [𝐡1 , 𝜏1 , 𝑏1] … [𝐡𝑛 , πœπ‘› , 𝑏𝑛] [𝐡 , 𝜏 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) 𝐡1 β†’ 𝜏1(… ), … , 𝐡𝑛 β†’ πœπ‘›(… ) are rules [𝐡0, 𝜏, 𝑏] [𝐡0, 𝑏] {𝐡0 β†’ 𝜏(… ) is a rule Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 3 / 5

slide-121
SLIDE 121

Canonical weighted deduction system

wRTG-LM ((𝐻, β„’), 𝕃, wt ) and 𝑏 ∈ β„’ ⇝ wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² ) [𝐡1 , 𝜏1 , 𝑏1] … [𝐡𝑛 , πœπ‘› , 𝑏𝑛] [𝐡 , 𝜏 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) 𝐡1 β†’ 𝜏1(… ), … , 𝐡𝑛 β†’ πœπ‘›(… ) are rules [𝐡0, 𝜏, 𝑏] [𝐡0, 𝑏] {𝐡0 β†’ 𝜏(… ) is a rule Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 3 / 5

slide-122
SLIDE 122

Canonical weighted deduction system

wRTG-LM ((𝐻, β„’), 𝕃, wt ) and 𝑏 ∈ β„’ ⇝ wRTG-LM ((𝐻′, π’Ÿβ„± π’£βˆ…), 𝕃, wtβ€² ) [𝐡1 , 𝜏1 , 𝑏1] … [𝐡𝑛 , πœπ‘› , 𝑏𝑛] [𝐡 , 𝜏 , 𝑏0] { 𝐡 β†’ 𝜏(𝐡1, … , 𝐡𝑛) is a rule 𝑏0, 𝑏1, … , 𝑏𝑛 ∈ factors(𝑏) 𝑏0 = 𝜏(𝑏1, … , 𝑏𝑛) 𝐡1 β†’ 𝜏1(… ), … , 𝐡𝑛 β†’ πœπ‘›(… ) are rules [𝐡0, 𝜏, 𝑏] [𝐡0, 𝑏] {𝐡0 β†’ 𝜏(… ) is a rule Weight preserving

1

Bijection πœ”: AST(𝐻, 𝑏) β†’ AST(𝐻′)

2

wt(𝑒) = wtβ€²(πœ”(𝑒)) for every 𝑒 ∈ AST(𝐻, 𝑏)

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 3 / 5

slide-123
SLIDE 123

Closed wRTG-LMs

cutout(𝑒, 𝜍)

𝑠3 𝑠1 𝑠4 𝑠2 𝑠1 𝑠2 𝑠4 𝑠3 𝑠1 𝑠2 𝑠4 𝑠2 𝑠1 𝑠4 𝑠4 𝑠3 𝑠1 𝑠2 𝑠4 𝑠3 𝑠1 𝑠2 𝑠4 𝑠2 𝑠1 𝑠4 𝑠4 𝑠3 𝑠1 𝑠4 𝑠2 𝑠1 𝑠2 𝑠4 𝑠3 𝑠1 𝑠4 𝑠4 𝑠3 𝑠1 𝑠2 𝑠4 𝑠3 𝑠1 𝑠4 𝑠4

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 4 / 5

slide-124
SLIDE 124

Closed wRTG-LMs

Defjnition

Let 𝑑 ∈ β„•. A wRTG-LM 𝒣 = ((𝐻, β„’), 𝕃, wt ) is 𝑑-closed if 𝕃 is distributive and d-complete, and for each 𝑒 ∈ T𝑆 and cyclic string 𝜍 ∈ π‘†βˆ— the following holds: if there is a (𝑑, 𝜍)-cyclic path in 𝑒, then wt(𝑒)𝕃 βŠ• ⨁

π‘’βˆˆcutout(𝑒,𝜍)

wt(𝑒)𝕃 = ⨁

π‘’βˆˆcutout(𝑒,𝜍)

wt(𝑒)𝕃 . AST(𝐻)(𝑑): each cycle at most 𝑑 times closed, distributive, d-complete

Theorem

For every 𝑑 ∈ β„• and 𝑑-closed wRTG-LM ((𝐻, β„’), 𝕃, wt ) the following holds:

βˆ‘βŠ•

π‘’βˆˆAST(𝐻′)

wt(𝑒)𝕃 = ⨁

π‘’βˆˆAST(𝐻)(𝑑)

wt(𝑒)𝕃 .

Richard MΓΆrbitz, Heiko Vogler: Weighted parsing for grammar-based language models (FSMNLP 2019) 2019-09-25 5 / 5