Weighted Tree Transducers in Natural Language Processing Andreas Maletti Universitat Rovira i Virgili Tarragona, Spain email: andreas.maletti@urv.cat Wrocław — May 17, 2010 Weighted Tree Transducers in NLP Andreas Maletti 1 ·
Collaborators Joint work with J OOST E NGELFRIET , L IACS , Leiden, The Netherlands UL ¨ Z OLT ´ AN F ¨ OP , University of Szeged, Hungary J ONATHAN G RAEHL , U SC , Los Angeles, C A , U SA M ARK H OPKINS , Language Weaver Inc., Los Angeles, C A , U SA K EVIN K NIGHT , U SC , Los Angeles, C A , U SA E RIC L ILIN , Universit´ e de Lille, France G IORGIO S ATTA , University of Padua, Italy H EIKO V OGLER , T U Dresden, Germany Weighted Tree Transducers in NLP Andreas Maletti 2 ·
Machine Translation 1 Weighted Tree Transducer 2 Expressive Power 3 Standard Algorithms 4 Implementation 5 Weighted Tree Transducers in NLP Andreas Maletti 3 ·
Motivation Example (Input in Catalan) Benvolguda i benvolgut membre de la comunitat universit` aria, Avui dilluns es duu a terme el darrer Consell de Govern del meu mandat com a rector; el proper dia 6 de maig, com correspon, hi haur` a una nova elecci´ o on tota la comunitat universit` aria podr` a escollir nou rector o rectora. Aquest darrer consell t´ e, naturalment, un car` acter marcadament t` ecnic; l’ordre del dia complet el trobar` as adjunt al final d’aquest text. A continuaci´ o et comento nom´ es els punts que, al meu parer, poden ser m´ es del teu inter` es. Translation (G OOGLE T RANSLATE ) to English Dear and beloved member of the university community, Today is Monday carried out by the Governing Council last of my term as rector, the next day, May 6, as appropriate, there will be another election where the entire university community can choose new rector. This last advice is, of course, a markedly technician complete agenda can be found attached to the end of this text. Then I said only the points that I believe may be of interest. Weighted Tree Transducers in NLP Andreas Maletti 4 ·
Motivation Example (Input in Catalan) Benvolguda i benvolgut membre de la comunitat universit` aria, Avui dilluns es duu a terme el darrer Consell de Govern del meu mandat com a rector; el proper dia 6 de maig, com correspon, hi haur` a una nova elecci´ o on tota la comunitat universit` aria podr` a escollir nou rector o rectora. Aquest darrer consell t´ e, naturalment, un car` acter marcadament t` ecnic; l’ordre del dia complet el trobar` as adjunt al final d’aquest text. A continuaci´ o et comento nom´ es els punts que, al meu parer, poden ser m´ es del teu inter` es. Translation (G OOGLE T RANSLATE ) to English Dear and beloved member of the university community, Today is Monday carried out by the Governing Council last of my term as rector, the next day, May 6, as appropriate, there will be another election where the entire university community can choose new rector. This last advice is, of course, a markedly technician complete agenda can be found attached to the end of this text. Then I said only the points that I believe may be of interest. Weighted Tree Transducers in NLP Andreas Maletti 4 ·
Machine Translation System Input sentence ( Benvolguda i benvolgut ... ) ⇓ Translation system ⇓ Output sentence ( Dear and beloved ... ) Weighted Tree Transducers in NLP Andreas Maletti 5 ·
Machine Translation System Input sentence ( Benvolguda i benvolgut ... ) f ⇓ Translation system ⇓ Output sentence ( Dear and beloved ... ) e Statistical translation system e = argmax p ( e | f ) e Weighted Tree Transducers in NLP Andreas Maletti 5 ·
Noisy Channel Viewpoint Input sentence ( Benvolguda i benvolgut ... ) f ⇓ Identity translation ⇐ Error signal (Noise) ⇓ Output sentence ( Dear and beloved ... ) e Weighted Tree Transducers in NLP Andreas Maletti 6 ·
Noisy Channel Viewpoint Input sentence ( Benvolguda i benvolgut ... ) f ⇓ Identity translation ⇐ Error signal (Noise) ⇓ Output sentence ( Dear and beloved ... ) e Bayes’ theorem p ( f | e ) · p ( e ) e = argmax p ( e | f ) = argmax = argmax p ( f | e ) · p ( e ) p ( f ) e e e Weighted Tree Transducers in NLP Andreas Maletti 6 ·
Components Optimization problem e = argmax p ( f | e ) · p ( e ) e Required models p ( e ) — language model p ( f | e ) — translation model Translation Language Input ⇐ Output ⇐ ⇐ model p ( f | e ) model p ( e ) Sentence f sentence e Weighted Tree Transducers in NLP Andreas Maletti 7 ·
Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Tree Transducers in NLP Andreas Maletti 8 ·
Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Tree Transducers in NLP Andreas Maletti 8 ·
Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Tree Transducers in NLP Andreas Maletti 8 ·
Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Tree Transducers in NLP Andreas Maletti 8 ·
Translation Approach Overview Foreign English Semantics Syntax Phrase Weighted Tree Transducers in NLP Andreas Maletti 8 ·
Why Syntax? Example She saw the boy with the telescope. Weighted Tree Transducers in NLP Andreas Maletti 9 ·
Why Syntax? Example She saw the boy with the telescope. S VP NP She VB NP saw NP PP the boy PREP NP with the telescope. Weighted Tree Transducers in NLP Andreas Maletti 9 ·
Why Syntax? Example She saw the boy with the telescope. S VP NP She VP PP VB NP PREP NP the boy saw with the telescope. Weighted Tree Transducers in NLP Andreas Maletti 9 ·
Syntactic Analysis Output sentence Holly picks flowers to tie them around July’s neck. Parser output S NN VP Holly VB NN ATO picks flowers TO VP to VB PP WHOBJ tie them PRP NN3 NN July’s around neck. Weighted Tree Transducers in NLP Andreas Maletti 10 ·
Syntax-based Machine Translation S NN VP Holly VB NN ATO picks flowers TO VP to VB PP WHOBJ tie them PRP NN3 NN around July’s neck. Weighted Tree Transducers in NLP Andreas Maletti 11 ·
Syntax-based Machine Translation S NN VP Holly VB NN ATO pfl¨ uckt Blumen TO VP to VB PP WHOBJ tie them PRP NN3 NN around July’s neck. Weighted Tree Transducers in NLP Andreas Maletti 11 ·
Syntax-based Machine Translation S NN VP Holly VB NN ATO pfl¨ uckt Blumen TO VP , um VB 1 PP 2 WHOBJ 3 tie them PRP NN3 NN around July’s neck. Weighted Tree Transducers in NLP Andreas Maletti 11 ·
Syntax-based Machine Translation S NN VP Holly VB NN ATO pfl¨ uckt Blumen TO VP , um PP 2 WHOBJ 3 VB 1 them PRP NN3 NN tie around July’s neck. Weighted Tree Transducers in NLP Andreas Maletti 11 ·
Syntax-based Machine Translation S NN VP Holly VB NN ATO pfl¨ uckt Blumen TO VP , um PP WHOBJ VB sie PRP NN3 NN zu binden. um Julys Hals Weighted Tree Transducers in NLP Andreas Maletti 11 ·
Table of Contents Machine Translation 1 Weighted Tree Transducer 2 Expressive Power 3 Standard Algorithms 4 Implementation 5 Weighted Tree Transducers in NLP Andreas Maletti 12 ·
Weight Structure Definition ( A , + , · , 0 , 1 ) is a (commutative) semiring if ( A , + , 0 ) and ( A , · , 1 ) commutative monoids, · distributes over + , and a · 0 = 0 for every a ∈ A . Example ( { 0 , 1 } , max , min , 0 , 1 ) B OOLEAN semiring ( R , + , · , 0 , 1 ) semiring of real numbers ( N ∪ {∞} , min , + , ∞ , 0 ) any field, ring, etc. Weighted Tree Transducers in NLP Andreas Maletti 13 ·
Syntax Definition ( Q , Σ , ∆ , I , R ) (weighted) extended (top-down) tree transducer (xtt) Q finite set of states (considered unary) Σ and ∆ ranked alphabets I : Q → A initial weight distribution R : Q ( T Σ ( X )) × T ∆ ( Q ( X )) → A rule weight assignment s.t. ◮ supp ( R ) is finite ◮ for every ( l , r ) ∈ supp ( R ) there is k ∈ N such that l ∈ Q ( C Σ ( X k )) and r ∈ T ∆ ( Q ( X k )) ◮ { l , r } �⊆ Q ( X ) for every ( l , r ) ∈ supp ( R ) References A RNOLD , D AUCHET : Bi-transductions de forˆ ets. I CALP 1976 G RAEHL , K NIGHT : Training tree transducers. H LT -N AACL 2004 Weighted Tree Transducers in NLP Andreas Maletti 14 ·
Syntax — Example S ⇒ ∗ S NP VP S ′ CONJ DT N V NP wa- [and] the boy DT N saw V NP NP the door ra’aa N N [saw] atefl albab [the boy] [the door] Question How to implement this English → Arabic translation using xtt? Weighted Tree Transducers in NLP Andreas Maletti 15 ·
Syntax — Example (cont’d) Example States { q , q S , q V , q NP } of which only q is initial q ( x 1 ) → q S ( x 1 ) ( r 1 ) q ( x 1 ) → S ( CONJ ( wa- ) , q S ( x 1 )) ( r 2 ) q S ( S ( x 1 , VP ( x 2 , x 3 ))) → S ′ ( q V ( x 2 ) , q NP ( x 1 ) , q NP ( x 3 )) ( r 3 ) q V ( V ( saw )) → V ( ra’aa ) ( r 4 ) q NP ( NP ( DT ( the ) , N ( boy ))) → NP ( N ( atefl )) ( r 5 ) q NP ( NP ( DT ( the ) , N ( door ))) → NP ( N ( albab )) ( r 6 ) Weighted Tree Transducers in NLP Andreas Maletti 16 ·
Syntax — Example (cont’d) Example Nondeterminism and epsilon rules (rules r 1 and r 2 ) 1 q → q S and q → S CONJ q S x 1 x 1 x 1 wa- x 1 Weighted Tree Transducers in NLP Andreas Maletti 17 ·
Recommend
More recommend
