Tree Transducers in Machine Translation Andreas Maletti Universitt - PowerPoint PPT Presentation

Word-based system (FST) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: the matter was decided , and everything was put in place Output: f kAn Tree Transducers in MT A. Maletti 6 ·

Word-based system (FST) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: the matter , and everything was put in place Output: f kAn An tm AlHsm Tree Transducers in MT A. Maletti 6 ·

Word-based system (FST) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: the matter and everything was put in place Output: f kAn An tm AlHsm Tree Transducers in MT A. Maletti 6 ·

Word-based system (FST) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: the matter everything was put in place Output: f kAn An tm AlHsm w Tree Transducers in MT A. Maletti 6 ·

Word-based system (FST) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: the matter was put in place Output: f kAn An tm AlHsm w Tree Transducers in MT A. Maletti 6 ·

Word-based system (FST) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: the matter in place Output: f kAn An tm AlHsm w wDEt Tree Transducers in MT A. Maletti 6 ·

Word-based system (FST) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: in place Output: f kAn An tm AlHsm w wDEt Al > mwr Tree Transducers in MT A. Maletti 6 ·

Word-based system (FST) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: place Output: f kAn An tm AlHsm w wDEt Al > mwr fy Tree Transducers in MT A. Maletti 6 ·

Word-based system (FST) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: Output: f kAn An tm AlHsm w wDEt Al > mwr fy nSAb hA Tree Transducers in MT A. Maletti 6 ·

Phrase-based machine translation Schema Machine Language model Input − → translation − → − → Output system Phrase-based systems Language Machine model Input − → Segmenter − → translation − → − → Output system Tree Transducers in MT A. Maletti 7 ·

Phrase-based system (FST+Perm) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: And then the matter was decided , and everything was put in place Output: Tree Transducers in MT A. Maletti 8 ·

Phrase-based system (FST+Perm) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: And then 1 the matter 5 was decided 2 , and everything 3 was put 4 in place 6 Output: Tree Transducers in MT A. Maletti 8 ·

Phrase-based system (FST+Perm) And then the matter was decided , and everything was put in place � �� w f tm wDEt fy kAn An AlHsm nSAb hA Al > mwr Derivation Input: And then 1 the matter 5 was decided 2 , and everything 3 was put 4 in place 6 Output: f kAn 1 An tm AlHsm 2 w 3 wDEt 4 Almwr 5 fy nSAb hA 6 Tree Transducers in MT A. Maletti 8 ·

Machine translation (cont’d) Phrase-based systems Language Machine model Input − → Segmenter − → translation − → − → Output system Syntax-based systems Language Machine model Input − → Parser − → translation − → − → Output system Tree Transducers in MT A. Maletti 9 ·

Parser S S CC S CC ADVP NP-SBJ-9 VP and NP-SBJ-1 VP And RB DT NN VBD VP NN VBD VP was everything was then the matter VBN NP-9 VBN NP-1 PP , put decided * IN NP in NN place And then the matter was decided , and everything was put in place (thanks to K EVIN K NIGHT for the data) Tree Transducers in MT A. Maletti 10 ·

S S CC S CC ADVP NP-SBJ-9 VP and NP-SBJ-1 VP And RB DT NN VBD VP NN VBD VP was everything was then the matter VBN NP-9 VBN NP-1 PP , decided put * IN NP in NN place hA nSAb POSS AlHsm Almwr fy NN NP tm DET-NN wDEt DET-NN * PREP NP PV NP-SBJ PV NP-SBJ1 NP-OBJ1 PP w VP VP An S CONJ S kAn * SUB S f PV NP-SBJ SBAR CONJ VP S Tree Transducers in MT A. Maletti 11 ·

S NP-SBJ VP NML NNP VBD PP signed JJ NNP Voislav IN NP Yugoslav President for NNP Serbia SrbyA Alr } ys AltwqyE En NN-PROP AlywgwslAfy fwyslAf DET-NN PREP NP DET-NN DET-ADJ NN-PROP twlY NP PP NP NP w PV NP-OBJ NP-SBJ CONJ VP S Tree Transducers in MT A. Maletti 12 ·

Contents Machine Translation 1 Extended Top-down Tree Transducers 2 Multi Bottom-up Tree Transducers 3 Synchronous Tree-Adjoining Grammars 4 Tree Transducers in MT A. Maletti 13 ·

Weight structure Definition Commutative semiring ( C , + , · , 0 , 1 ) if ( C , + , 0 ) and ( C , · , 1 ) commutative monoids · distributes over finite (incl. empty) sums Example B OOLEAN semiring ( { 0 , 1 } , max , min , 0 , 1 ) Semiring ( R ≥ 0 , + , · , 0 , 1 ) of probabilities Tropical semiring ( N ∪ {∞} , min , + , ∞ , 0 ) Any field, ring, etc. Most of the talk: B OOLEAN semiring Tree Transducers in MT A. Maletti 14 ·

Syntax Definition (A RNOLD , D AUCHET 1976, G RAEHL , K NIGHT 2004) Extended top-down tree transducer (XTOP) M = ( Q , Σ , ∆ , I , R ) with finitely many rules q ∆ → Σ q ′ ( x i ) . . . p ( x j ) . . . x 1 x k q , q ′ , p ∈ Q are states i , j ∈ { 1 , . . . , k } Tree Transducers in MT A. Maletti 15 ·

Syntax (cont’d) Definition (R OUNDS 1970, T HATCHER 1970) Top-down tree transducer (TOP) if all rules q ∆ → σ . . . x 1 x k q ′ ( x i ) . . . p ( x j ) linear if no variable occurs twice in r for all rules l → r nondeleting if var ( l ) = var ( r ) for all rules l → r Tree Transducers in MT A. Maletti 16 ·

Semantics Example States { q S , q V , q NP } of which only q S is initial q S S ′ q V q NP q V q NP → → → q V q NP q NP S VP VP x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 2 x 1 x 2 Derivation q S S ′ S ′ S ′ q V q NP q NP q V q NP q NP S ⇒ ⇒ ⇒ q V q NP q NP t 1 VP t 1 t 2 t 1 VP VP VP t 2 t 1 t 3 t 2 t 3 t 3 t 3 t 3 t 2 t 2 t 2 Tree Transducers in MT A. Maletti 17 ·

Semantics (cont’d) Definition Computed transformation: τ M = { ( t , u ) ∈ T Σ × T ∆ | ∃ q ∈ I : q ( t ) ⇒ ∗ u } Tree Transducers in MT A. Maletti 18 ·

Rule extraction S q CONJ VP S q S w PV NP-OBJ x 1 VP → NP-SBJ VP q x 1 twlY NP NML NNP VBD PP x 2 VBD x 2 DET-NN signed JJ NNP Voislav IN NP signed Yugoslav President for NNP AltwqyE Serbia SrbyA AltwqyE Alr } ys AlywgwslAfy fwyslAf En NN-PROP DET-NN PREP NP DET-NN DET-ADJ NN-PROP twlY NP PP NP NP w PV NP-OBJ NP-SBJ CONJ VP S Tree Transducers in MT A. Maletti 20 ·

Rule extraction S q CONJ VP S q S w PV NP-OBJ x 1 VP → NP-SBJ VP q x 1 twlY NP NML NNP VBD PP x 2 VBD x 2 DET-NN signed JJ NNP Voislav IN NP signed Yugoslav President for NNP AltwqyE Serbia q NP-SBJ SrbyA q NP NP-SBJ AltwqyE Alr } ys AlywgwslAfy fwyslAf → En NN-PROP x 1 x 1 NN-PROP NNP DET-NN PREP NP DET-NN DET-ADJ NN-PROP twlY NP PP NP NP fwyslAf Voislav w PV NP-OBJ NP-SBJ CONJ VP S Tree Transducers in MT A. Maletti 20 ·

Rule extraction S q CONJ VP S q S w PV NP-OBJ x 1 VP → NP-SBJ VP q x 1 twlY NP NML NNP VBD PP x 2 VBD x 2 DET-NN signed JJ NNP Voislav IN NP signed Yugoslav President for NNP AltwqyE Serbia q NP-SBJ SrbyA q NP NP-SBJ AltwqyE Alr } ys AlywgwslAfy fwyslAf → En NN-PROP x 1 x 1 NN-PROP NNP DET-NN PREP NP DET-NN DET-ADJ NN-PROP twlY NP PP NP NP fwyslAf Voislav w PV NP-OBJ NP-SBJ CONJ VP q S NP NML → DET-NN DET-ADJ JJ NNP Alr } ys AlywgwslAfy Yugoslav President Tree Transducers in MT A. Maletti 20 ·

Symmetry Original rule S q CONJ VP S w q PV NP-OBJ x 1 → VP q x 1 twlY NP x 2 VBD x 2 DET-NN signed AltwqyE Tree Transducers in MT A. Maletti 21 ·

Symmetry Original rule Inverted rule q S q S CONJ VP S S w q CONJ VP PV NP-OBJ q VP x 1 → VP w q → q x 1 PV NP-OBJ twlY NP x 1 q x 2 VBD VBD q x 1 x 2 twlY NP DET-NN signed x 2 signed x 2 DET-NN AltwqyE AltwqyE Tree Transducers in MT A. Maletti 21 ·

Symmetry Original rule Inverted rule q S q S CONJ VP S S w q CONJ VP PV NP-OBJ q VP x 1 → VP w q → q x 1 PV NP-OBJ twlY NP x 1 q x 2 VBD VBD q x 1 x 2 twlY NP DET-NN signed x 2 signed x 2 DET-NN AltwqyE AltwqyE Linear nondeleting XTT can be inverted Tree Transducers in MT A. Maletti 21 ·

Preservation of regularity Schematics Language model Input − → Parser − → XTT − → − → Output Parse trees best parse tree n -best parses all parses Can all be represented by regular tree language Tree Transducers in MT A. Maletti 22 ·

Preservation of regularity Schematics Input − → Parser − → Parse trees − → XTT − → . . . Parse trees best parse tree n -best parses all parses Can all be represented by regular tree language Tree Transducers in MT A. Maletti 22 ·

Preservation of regularity (cont’d) Schematics Regular language − → XTT − → Regular language? Approach Input restriction Project to output Result Linear XTT preserve regularity Tree Transducers in MT A. Maletti 23 ·

Composition Schematics Parse trees − → XTT − → . . . Example (Y AMADA , K NIGHT 2002) Reorder Insert words Translate words Tree Transducers in MT A. Maletti 24 ·

Composition Schematics Stage 1 Stage 2 Stage 3 Parse trees − → XTT − → XTT − → XTT − → . . . Example (Y AMADA , K NIGHT 2002) Reorder Insert words Translate words Tree Transducers in MT A. Maletti 24 ·

Composition Schematics Composed Parse trees − → XTT − → . . . Example (Y AMADA , K NIGHT 2002) Reorder Insert words Translate words Tree Transducers in MT A. Maletti 24 ·

Composition (cont’d) Example (A RNOLD , D AUCHET 1982) σ t 1 σ σ δ t 2 σ t 1 δ t 2 t 1 δ t 3 σ t 2 t 3 δ t 4 t 3 ⇒ ∗ ⇒ ∗ δ t 4 σ t n − 4 t n − 3 δ t n − 2 t n − 3 σ t n − 3 σ t n − 2 t n − 1 t n t n − 1 t n t n − 2 σ t n − 1 t n Tree Transducers in MT A. Maletti 25 ·

Summary E XPR S YM P RES P RES − 1 C OMP Model \ Criterion Linear nondeleting TOP ✗ ✗ ✓ ✓ ✓ Linear TOP ✗ ✗ ✓ ✓ ✗ Linear TOP R ✗ ✗ ✓ ✓ ✓ General TOP ✗ ✗ ✗ ✓ ✗ General TOP R ✓ ✗ ✗ ✓ ✗ Linear nondeleting XTOP ✓ ✓ ✓ ✓ ✗ Linear XTOP ✓ ✗ ✓ ✓ ✗ Linear XTOP R ✓ ✗ ✓ ✓ ✗ General XTOP ✓ ✗ ✗ ✓ ✗ General XTOP R ✓ ✗ ✗ ✓ ✗ Tree Transducers in MT A. Maletti 26 ·

Summary E XPR S YM P RES P RES − 1 C OMP Model \ Criterion Linear nondeleting TOP ✗ ✗ ✓ ✓ ✓ Linear TOP ✗ ✗ ✓ ✓ ✗ Linear TOP R ✗ ✗ ✓ ✓ ✓ General TOP ✗ ✗ ✗ ✓ ✗ General TOP R ✓ ✗ ✗ ✓ ✗ Linear nondeleting XTOP ✓ ✓ ✓ ✓ ✗ Linear XTOP ✓ ✗ ✓ ✓ ✗ Linear XTOP R ✓ ✗ ✓ ✓ ✗ General XTOP ✓ ✗ ✗ ✓ ✗ General XTOP R ✓ ✗ ✗ ✓ ✗ Comp. closure ln-XTOP ✓ ✓ ✓ ✓ ✓ “composable” ln-XTOP ? ? ✓ ✓ ✓ Tree Transducers in MT A. Maletti 26 ·

Implementation T IBURON [M AY , K NIGHT 2006] Implements XTOP (and tree automata; everything also weighted) Framework with command-line interface Optimized for machine translation Algorithms Application of XTOP to input tree/language Backward application of XTOP to output language Composition (for some XTOP) Example qNP.NP(DT(the) N(boy)) -> NP(N(atefl)) Tree Transducers in MT A. Maletti 27 ·

Multi Bottom-up Tree Transducers S NP-SBJ VP NML NNP VBD PP signed JJ NNP Voislav IN NP Yugoslav President for NNP Serbia SrbyA AltwqyE Alr } ys AlywgwslAfy fwyslAf En NN-PROP DET-NN PREP NP DET-NN DET-ADJ NN-PROP twlY NP PP NP NP w PV NP-OBJ NP-SBJ CONJ VP S Tree Transducers in MT A. Maletti 28 ·

Syntax Definition Extended multi bottom-up tree transducer (XMBOT) is M = ( Q , Σ , ∆ , F , R ) with finitely many rules q Σ → ∆ ∆ . . . q ′ p . . . . . . . . . x i x j x i ′ x j ′ . . . . . . x m x n x 1 x ℓ q ′ , p , q ∈ Q are now ranked states F ⊆ Q 1 final states Tree Transducers in MT A. Maletti 29 ·

Example States { f ( 1 ) , q ( 2 ) } with final state f and rules q a q b f q q q → → → e → q a a σ b b e e x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 Example (Derivation) a b b e Tree Transducers in MT A. Maletti 30 ·

Example States { f ( 1 ) , q ( 2 ) } with final state f and rules q a q b f q q q → → → e → q a a σ b b e e x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 Example (Derivation) a a b ⇒ b b q b e e e Tree Transducers in MT A. Maletti 30 ·

Example States { f ( 1 ) , q ( 2 ) } with final state f and rules q a q b f q q q → → → e → q a a σ b b e e x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 Example (Derivation) a a a b b ⇒ ⇒ q b b q b b b e e e e e Tree Transducers in MT A. Maletti 30 ·

Example States { f ( 1 ) , q ( 2 ) } with final state f and rules q a q b f q q q → → → e → q a a σ b b e e x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 Example (Derivation) a a a q a b b ⇒ ⇒ q ⇒ b b b b q b b b b b e e e e e e e Tree Transducers in MT A. Maletti 30 ·

Example States { f ( 1 ) , q ( 2 ) } with final state f and rules q a q b f q q q → → → e → q a a σ b b e e x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 Example (Derivation) a a a q q a b b a a ⇒ ⇒ q ⇒ ⇒ b b b b b b q b b b b b b b e e e e e e e e e Tree Transducers in MT A. Maletti 30 ·

Example States { f ( 1 ) , q ( 2 ) } with final state f and rules q a q b f q q q → → → e → q a a σ b b e e x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 Example (Derivation) f a a a q σ q a b b a a a a ⇒ ⇒ ⇒ q ⇒ ⇒ b b b b b b b b q b b b b b b b b b e e e e e e e e e e e Tree Transducers in MT A. Maletti 30 ·

Semantics Definition Computed transformation: τ M = { ( t , u ) ∈ T Σ × T ∆ | ∃ q ∈ F : t ⇒ ∗ q ( u ) } Tree Transducers in MT A. Maletti 31 ·

Semantics Definition Computed transformation: τ M = { ( t , u ) ∈ T Σ × T ∆ | ∃ q ∈ F : t ⇒ ∗ q ( u ) } Example τ M = {� t , σ ( t , t ) � | t ∈ T Σ } a q q b f q q q → q → → e → a a σ b b x 1 x 2 e e x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 x 1 x 2 Tree Transducers in MT A. Maletti 31 ·

Rule extraction q S S → q p S CONJ VP x 1 x 2 x 3 NP-SBJ VP w x 2 x 3 x 1 NML NNP VBD PP signed JJ NNP Voislav IN NP Yugoslav President for NNP Serbia SrbyA Alr } ys AltwqyE En NN-PROP AlywgwslAfy fwyslAf DET-NN PREP NP DET-NN DET-ADJ NN-PROP twlY NP PP NP NP w PV NP-OBJ NP-SBJ CONJ VP S Tree Transducers in MT A. Maletti 33 ·

Rule extraction q S S → q p S CONJ VP x 1 x 2 x 3 NP-SBJ VP w x 2 x 3 x 1 NML NNP VBD PP p signed JJ NNP Voislav IN NP VP PV NP-OBJ Yugoslav President for NNP q x 1 VBD → twlY NP Serbia x 1 signed DET-NN AltwqyE SrbyA Alr } ys AltwqyE En NN-PROP AlywgwslAfy fwyslAf DET-NN PREP NP DET-NN DET-ADJ NN-PROP twlY NP PP NP NP w PV NP-OBJ NP-SBJ CONJ VP S Tree Transducers in MT A. Maletti 33 ·

One-symbol normal form Definition Rule in one-symbol normal form if it contains at most one symbol Tree Transducers in MT A. Maletti 34 ·

One-symbol normal form Definition Rule in one-symbol normal form if it contains at most one symbol Example (E NGELFRIET , L ILIN , ∼ 2009) q b → q b b x 1 x 2 x 1 x 2 Tree Transducers in MT A. Maletti 34 ·

Tree Transducers in Machine Translation Andreas Maletti Universitt - PowerPoint PPT Presentation

Tree Transducers in Machine Translation Andreas Maletti Universitt Stuttgart Institute for Natural Language Processing andreas.maletti@ims.uni-stuttgart.de Szeged November 29, 2011 Tree Transducers in MT A. Maletti 1 Machine

Syntax-based Machine Translation using Multi Bottom-up Tree Transducers Andreas Maletti Fabienne

Compositions of Extended Top-down Tree Transducers Andreas Maletti March 30, 2007 Short

Learning Non-Isomorphic Tree Mappings for Machine Translation Syntax-Based Machine Translation

Compositions of Tree-to-Tree Statistical Machine Translation Models Andreas Maletti Institute

Linking Theorems for Tree Transducers Andreas Maletti maletti@ims.uni-stuttgart.de Speyer

Tree Transducers and Tree Adjoining Grammars Historical and Current Perspectives William C.

Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional

Tree Transducers Niko Paltzer Seminar Formal Grammars WS 06/07 Advisor: Marco Kuhlmann

Experiments in EnglishJapanese Tree-to-String Machine Translation Graham Neubig Nara

Equivalence of Deterministic Tree-to-String Transducers Is Decidable Helmut Seidl, Sebastian

Statistical Machine Translation Nadir Durrani 21-November-2014 Machine Translation

Categorical views on bottom-up tree transducers Tarmo Uustalu joint work with Ichiro Hasuo, Bart

Machine Translation 2 Wikipedia Machine translation, often referred to by the acronym MT, is a

Input Products for Weighted Extended Top-down Tree Transducers Andreas Maletti Universitat

Relating Tree Series Transducers and Weighted Tree Automata Andreas Maletti December 17, 2004

Machine Translation (M2M) Machine Translation (M2M) SNMP MIB to CIM MOF SNMP MIB to CIM MOF

Learning Non-Isomorphic Tree Mappings for Machine Translation Jason Eisner, Computer Science

A Gentle Introduction to Weighted Extended Top-down Tree Transducers Andreas Maletti Universitat

Natural Language Processing Machine Translation Dan Klein UC Berkeley 1 Machine Translation 2

Introd u ction to machine translation MAC H IN E TR AN SL ATION IN P YTH ON Th u shan

Natural Language Processing Machine Translation Machine Translation Dan Klein UC Berkeley

Use of the Machine Translation Module within Dj Vu X2 Quick Guidance Introduction Machine

Weighted Tree Transducers in Natural Language Processing Andreas Maletti Universitat Rovira i

Learning Tree to Word Transducers LATA 2014 Aur elien Lemay joint work with: Gr egoire