Practical problems with Chomsky-Schtzenberger parsing for weighted - PowerPoint PPT Presentation

Practical problems with Chomsky-Schützenberger parsing for weighted multiple context-free grammars 1 Tobias Denkinger tobias.denkinger@tu-dresden.de Institute of Theoretical Computer Science Faculty of Computer Science Technische Universität Dresden WATA, Leipzig, 2018-05-23 1 based on T. Denkinger (2017). “Chomsky-Schützenberger parsing for weighted multiple context-free languages”.

a suitable partial order ⊴ on (𝒝, ⊙, 𝟚, 𝟙) The problem: 𝑙 -best parsing a word 𝑥 WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs (not unique) of 𝑥 in 𝐻 s 2 derivation sequence of 𝑙 best a Output: a number 𝑙 ∈ ℕ 𝑙 -best , wt ) 𝐻 ( grammar (𝒝, ⊙, 𝟚, 𝟙) -weighted a Input: [Huang and Chiang 2005] parsing problem 2 / 13

The problem: 𝑙 -best parsing 𝑙 -best parsing problem [Huang and Chiang 2005] Input: a (𝒝, ⊙, 𝟚, 𝟙) -weighted grammar (𝐻, wt ) a suitable partial order ⊴ on (𝒝, ⊙, 𝟚, 𝟙) a number 𝑙 ∈ ℕ a word 𝑥 Output: (not unique) T. Denkinger: Practical problems with CS-parsing for wMCFGs WATA, Leipzig, 2018-05-23 2 / 13 a sequence of 𝑙 best derivations 2 of 𝑥 in 𝐻 2 w.r.t. wt and ⊴ (greater is betuer)

composes tuples of strings Multiple context-free grammars ⏟ ⏟ ⏟ ⏟⏟⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟⏟ (𝛵 ∗ ×𝛵 ∗ )×(𝛵 ∗ ×𝛵 ∗ )→(𝛵 ∗ ×𝛵 ∗ ) ](𝐵, 𝐶) ⟹ extra expressive power useful for natural language processing T. Denkinger: Practical problems with CS-parsing for wMCFGs WATA, Leipzig, 2018-05-23 ⏟ ⏟ context-free grammars ⏟ 𝐵 → a 𝐵 b 𝐶 composes strings 𝐵 → [(𝑦, 𝑧) ↦ a 𝑦 b 𝑧 ⏟ ⏟ ⏟ ⏟ ⏟⏟ ⏟ ⏟ 𝛵 ∗ ×𝛵 ∗ →𝛵 ∗ ](𝐵, 𝐶) multiple context-free grammars [Seki, Matsumura, Fujii, and Kasami 1991] 𝐵 → [((𝑦 1 , 𝑦 2 ), (𝑧 1 , 𝑧 2 )) ↦ ( a 𝑦 1 𝑧 2 b , 𝑧 1 c 𝑦 2 ) 3 / 13

composes tuples of strings Multiple context-free grammars ⏟ ⏟ ⏟ ⏟⏟⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟⏟ (𝛵 ∗ ×𝛵 ∗ )×(𝛵 ∗ ×𝛵 ∗ )→(𝛵 ∗ ×𝛵 ∗ ) ](𝐵, 𝐶) ⟹ extra expressive power useful for natural language processing T. Denkinger: Practical problems with CS-parsing for wMCFGs WATA, Leipzig, 2018-05-23 ⏟ ⏟ context-free grammars ⏟ 𝐵 → a 𝐵 b 𝐶 composes strings 𝐵 → [(𝑦, 𝑧) ↦ a 𝑦 b 𝑧 ⏟ ⏟ ⏟ ⏟ ⏟⏟ ⏟ ⏟ 𝛵 ∗ ×𝛵 ∗ →𝛵 ∗ ](𝐵, 𝐶) multiple context-free grammars [Seki, Matsumura, Fujii, and Kasami 1991] 𝐵 → [((𝑦 1 , 𝑦 2 ), (𝑧 1 , 𝑧 2 )) ↦ ( a 𝑦 1 𝑧 2 b , 𝑧 1 c 𝑦 2 ) 3 / 13

composes tuples of strings Multiple context-free grammars ⏟⏟⏟ WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs ⟹ extra expressive power useful for natural language processing ](𝐵, 𝐶) (𝛵 ∗ ×𝛵 ∗ )×(𝛵 ∗ ×𝛵 ∗ )→(𝛵 ∗ ×𝛵 ∗ ) ⏟⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ context-free grammars ⏟ ⏟ ⏟ ⏟ ⏟⏟ 𝐵 → [((𝑦 1 , 𝑦 2 ), (𝑧 1 , 𝑧 2 )) ↦ ( a 𝑦 1 𝑧 2 b , 𝑧 1 c 𝑦 2 ) [Seki, Matsumura, Fujii, and Kasami 1991] multiple context-free grammars ](𝐵, 𝐶) 𝐵 → [ composes strings 𝐵 → a 𝐵 b 𝐶 3 / 13 a 𝑦 b 𝑧

Multiple context-free grammars ⏟⏟⏟ WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs ⟹ extra expressive power useful for natural language processing composes tuples of strings ](𝐵, 𝐶) (𝛵 ∗ ×𝛵 ∗ )×(𝛵 ∗ ×𝛵 ∗ )→(𝛵 ∗ ×𝛵 ∗ ) ⏟⏟ ⏟ ⏟ ⏟ ⏟ ⏟ ⏟ context-free grammars ⏟ ⏟ ⏟ ⏟ ⏟⏟ 𝐵 → [((𝑦 1 , 𝑦 2 ), (𝑧 1 , 𝑧 2 )) ↦ ( a 𝑦 1 𝑧 2 b , 𝑧 1 c 𝑦 2 ) [Seki, Matsumura, Fujii, and Kasami 1991] multiple context-free grammars ](𝐵, 𝐶) 𝐵 → [ composes strings 𝐵 → a 𝐵 b 𝐶 3 / 13 a 𝑦 b 𝑧

Multiple context-free grammars 𝐵 → [ WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs ⟹ extra expressive power useful for natural language processing composes tuples of strings ](𝐵, 𝐶) a 𝑦 1 𝑧 2 b , 𝑧 1 c 𝑦 2 [Seki, Matsumura, Fujii, and Kasami 1991] context-free grammars multiple context-free grammars ](𝐵, 𝐶) 𝐵 → [ composes strings 𝐵 → a 𝐵 b 𝐶 3 / 13 a 𝑦 b 𝑧

Multiple context-free grammars 𝐵 → [ WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs ⟹ extra expressive power useful for natural language processing composes tuples of strings ](𝐵, 𝐶) a 𝑦 1 𝑧 2 b , 𝑧 1 c 𝑦 2 [Seki, Matsumura, Fujii, and Kasami 1991] context-free grammars multiple context-free grammars ](𝐵, 𝐶) 𝐵 → [ composes strings 𝐵 → a 𝐵 b 𝐶 3 / 13 a 𝑦 b 𝑧

The Chomsky-Schützenberger theorem Dyck language 𝐸 , WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs Use the decomposition provided by (1. → 2.) for parsing. [Hulden 2011, for CFGs] Idea s.t. 𝑀 = ℎ(𝑆 ∩ 𝐸) ∃ homomorphism ℎ multiple CS-theorem s ∃ 2. ∃ regular language 𝑆 , CFG 𝐻 s.t. 𝑀 = L (𝐻) M 1. ∃ Let 𝑀 be a language. T.f.a.e. [Yoshinaka, Kaji, and Seki 2010] [Chomsky and Schützenberger 1963] 4 / 13

The Chomsky-Schützenberger theorem CS-theorem s [Chomsky and Schützenberger 1963] [Yoshinaka, Kaji, and Seki 2010] Let 𝑀 be a language. T.f.a.e. 2. ∃ regular language 𝑆 , ∃ homomorphism ℎ s.t. 𝑀 = ℎ(𝑆 ∩ 𝐸) Idea [Hulden 2011, for CFGs] Use the decomposition provided by (1. → 2.) for parsing. T. Denkinger: Practical problems with CS-parsing for wMCFGs WATA, Leipzig, 2018-05-23 4 / 13 1. ∃ MCFG 𝐻 s.t. 𝑀 = L (𝐻) ∃ multiple Dyck language 𝐸 ,

The Chomsky-Schützenberger theorem CS-theorem s [Chomsky and Schützenberger 1963] [Yoshinaka, Kaji, and Seki 2010] Let 𝑀 be a language. T.f.a.e. 2. ∃ regular language 𝑆 , ∃ homomorphism ℎ s.t. 𝑀 = ℎ(𝑆 ∩ 𝐸) Idea [Hulden 2011, for CFGs] Use the decomposition provided by (1. → 2.) for parsing. T. Denkinger: Practical problems with CS-parsing for wMCFGs WATA, Leipzig, 2018-05-23 4 / 13 1. ∃ MCFG 𝐻 s.t. 𝑀 = L (𝐻) ∃ multiple Dyck language 𝐸 ,

= ( take 𝑙 ∘ sort wt ⊴ ∘ toDeriv ∘ filter ∩𝐸 )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ sort wt ⊴ ∘ filter ∩𝐸 )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ filter ∩𝐸 ∘ sort wt ⊴ )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ filter ∩𝐸 ∘ sort ⊴ )(𝑆 wt WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs ℎ −1 (𝑥)) From the CS-theorem to CS-parsing 𝑥 ∈ L (𝐻) parse 𝐻, wt ,𝑙 (𝑥) 𝑙 -best CS-parsing Each 𝑣 ∈ 𝑆 ∩ 𝐸 encodes a derivation of 𝐻 . Observation ⟺ ∃𝑣 ∈ 𝑆 ∩ ℎ −1 (𝑥): 𝑣 ∈ 𝐸 ⟺ ∃𝑣 ∈ 𝑆 ∩ 𝐸: ℎ(𝑣) = 𝑥 (CS-theorem) ⟺ 𝑥 ∈ ℎ(𝑆 ∩ 𝐸) 5 / 13

= ( take 𝑙 ∘ sort wt ⊴ ∘ toDeriv ∘ filter ∩𝐸 )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ sort wt ⊴ ∘ filter ∩𝐸 )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ filter ∩𝐸 ∘ sort wt ⊴ )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ filter ∩𝐸 ∘ sort ⊴ )(𝑆 wt WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs ℎ −1 (𝑥)) From the CS-theorem to CS-parsing 𝑥 ∈ L (𝐻) ⟺ 𝑥 ∈ ℎ(𝑆 ∩ 𝐸) parse 𝐻, wt ,𝑙 (𝑥) 𝑙 -best CS-parsing Each 𝑣 ∈ 𝑆 ∩ 𝐸 encodes a derivation of 𝐻 . Observation ⟺ ∃𝑣 ∈ 𝑆 ∩ ℎ −1 (𝑥): 𝑣 ∈ 𝐸 ⟺ ∃𝑣 ∈ 𝑆 ∩ 𝐸: ℎ(𝑣) = 𝑥 (CS-theorem) 5 / 13

= ( take 𝑙 ∘ sort wt ⊴ ∘ toDeriv ∘ filter ∩𝐸 )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ sort wt ⊴ ∘ filter ∩𝐸 )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ filter ∩𝐸 ∘ sort wt ⊴ )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ filter ∩𝐸 ∘ sort ⊴ )(𝑆 wt WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs ℎ −1 (𝑥)) From the CS-theorem to CS-parsing 𝑥 ∈ L (𝐻) ⟺ 𝑥 ∈ ℎ(𝑆 ∩ 𝐸) parse 𝐻, wt ,𝑙 (𝑥) 𝑙 -best CS-parsing Each 𝑣 ∈ 𝑆 ∩ 𝐸 encodes a derivation of 𝐻 . Observation ⟺ ∃𝑣 ∈ 𝑆 ∩ ℎ −1 (𝑥): 𝑣 ∈ 𝐸 ⟺ ∃𝑣 ∈ 𝑆 ∩ 𝐸: ℎ(𝑣) = 𝑥 (CS-theorem) 5 / 13

= ( take 𝑙 ∘ sort wt ⊴ ∘ toDeriv ∘ filter ∩𝐸 )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ sort wt ⊴ ∘ filter ∩𝐸 )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ filter ∩𝐸 ∘ sort wt ⊴ )(𝑆 ∩ ℎ −1 (𝑥)) = ( toDeriv ∘ take 𝑙 ∘ filter ∩𝐸 ∘ sort ⊴ )(𝑆 wt WATA, Leipzig, 2018-05-23 T. Denkinger: Practical problems with CS-parsing for wMCFGs ℎ −1 (𝑥)) From the CS-theorem to CS-parsing 𝑥 ∈ L (𝐻) ⟺ 𝑥 ∈ ℎ(𝑆 ∩ 𝐸) parse 𝐻, wt ,𝑙 (𝑥) 𝑙 -best CS-parsing Each 𝑣 ∈ 𝑆 ∩ 𝐸 encodes a derivation of 𝐻 . Observation ⟺ ∃𝑣 ∈ 𝑆 ∩ ℎ −1 (𝑥): 𝑣 ∈ 𝐸 ⟺ ∃𝑣 ∈ 𝑆 ∩ 𝐸: ℎ(𝑣) = 𝑥 (CS-theorem) 5 / 13