Towards Probabilistic Acceptors and Transducers for Feature Structures Daniel Quernheim Institute for Natural Language Processing, University of Stuttgart daniel@ims.uni-stuttgart.de Kevin Knight Information Sciences Institute, University of Southern California knight@isi.edu Sixth Workshop on Syntax, Semantics and Structure in Statistical Translation (SSST-6) July 12, 2012 Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 1 / 21
Linguistic structures Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 2 / 21
Linguistic structures Strings surface forms, phonology, morphology Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 2 / 21
Linguistic structures Strings surface forms, phonology, morphology Trees syntax Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 2 / 21
Linguistic structures Strings surface forms, phonology, morphology Trees syntax Feature structures (= directed acyclic graphs) deep syntax (LFG etc.) semantics (abstract meaning representations) Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 2 / 21
Feature structures charge INSTANCE � � person INSTANCE THEME 1 “Pascale” NAME and INSTANCE resist INSTANCE AGENT 1 OP 1 � � arrest INSTANCE THEME THEME 1 PRED intoxicate INSTANCE THEME 1 OP 2 � � public LOCATION INSTANCE Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 3 / 21
Directed acyclic graphs CHARGE CHARGE �→ charge ( theme , pred ) AND �→ and ( op1 , op2 ) AND RESIST �→ resist ( agent , theme ) ARREST �→ arrest ( theme ) RESIST INTOXICATE INTOXICATE �→ intoxicate ( theme , location ) ARREST PUBLIC �→ public () PERSON �→ person ( name ) PERSON PUBLIC PASCALE �→ "Pascale" PASCALE Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 4 / 21
Translation pipelines Syntax-based MT pipeline fstring → translate → etree → language model → estring Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 5 / 21
Translation pipelines Syntax-based MT pipeline fstring → translate → etree → language model → estring ◮ The individual components are efficiently represented as weighted tree acceptors and transducers . Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 5 / 21
Translation pipelines Syntax-based MT pipeline nl-XTOPs − 1 FSA FSA RTG FSA fstring → translate → etree → language model → estring ◮ The individual components are efficiently represented as weighted tree acceptors and transducers . estring = B EST P ATH ( I NTERSECT ( language model , Y IELD ( B ACKWARDS ( translate , fstring )))) . Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 5 / 21
Translation pipelines (2) Semantics-based MT pipeline fstring → understand → AMR → rank → AMR → generate → etree → rank → estring Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 6 / 21
Translation pipelines (2) Semantics-based MT pipeline fstring → understand → AMR → rank → AMR → generate → etree → rank → estring ◮ No suitable automaton framework is known! Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 6 / 21
Algorithms and automata string automata tree automata graph automata k -best paths through a trees in a weighted WFSA forest EM training Forward-backward Tree transducer EM EM training Determinization of weighted string ac- of weighted tree ac- ceptors ceptors Transducer WFST composition Many transducers not composition closed under compo- sition General tools AT&T FSM, Carmel, Tiburon OpenFST Table: General-purpose algorithms for strings, trees and feature structures. Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 7 / 21
Algorithms and automata string automata tree automata graph automata ? k -best paths through a trees in a weighted WFSA forest EM training Forward-backward Tree transducer EM ? EM training Determinization of weighted string ac- of weighted tree ac- ? ceptors ceptors Transducer WFST composition Many transducers not ? composition closed under compo- sition General tools AT&T FSM, Carmel, Tiburon ? OpenFST Table: General-purpose algorithms for strings, trees and feature structures. Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 7 / 21
Algorithms and automata (2) Our goal ◮ Find an adequate automaton model for the pipeline parts ◮ Investigate algorithms and fill all the blanks! Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 8 / 21
Algorithms and automata (2) Our goal ◮ Find an adequate automaton model for the pipeline parts ◮ Investigate algorithms and fill all the blanks! Candidates ◮ Treating everything as a tree (too weak?) ◮ Unification grammars (HPSG, LFG) (too powerful?) ◮ Hyperedge replacement grammar (too powerful?) Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 8 / 21
Algorithms and automata (2) Our goal ◮ Find an adequate automaton model for the pipeline parts ◮ Investigate algorithms and fill all the blanks! Candidates ◮ Treating everything as a tree (too weak?) ◮ Unification grammars (HPSG, LFG) (too powerful?) ◮ Hyperedge replacement grammar (too powerful?) Some straightforward extension of string/tree automata? ◮ Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 8 / 21
Dag automata finite string automaton: (FSA) one input state, one input symbol, one output state . . . . . . p q σ Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 9 / 21
Dag automata finite string automaton: (FSA) one input state, one input symbol, one output state . . . . . . p q σ finite tree automaton: (FTA) one input state, one input symbol, many output states q 1 . . . . . . p q 2 σ q 3 Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 9 / 21
Dag automata finite string automaton: (FSA) one input state, one input symbol, one output state . . . . . . p q σ finite tree automaton: (FTA) one input state, one input symbol, many output states q 1 . . . . . . p q 2 σ q 3 finite dag automaton: (FDA?) many input states, one input symbol, many output states p 1 q 1 . . . . . . q 2 σ p 2 q 3 Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 9 / 21
Dag automata (2) K AMIMURA and S LUTZKI (1981, 1982) ◮ Dag acceptors and dag-to-tree transducers ◮ They proved a couple of technical properties, no algorithms Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 10 / 21
Dag automata (2) K AMIMURA and S LUTZKI (1981, 1982) ◮ Dag acceptors and dag-to-tree transducers ◮ They proved a couple of technical properties, no algorithms ◮ We investigate their model with some adjustments: ◮ not only adjacent leaves can be connected Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 10 / 21
Dag automata (2) K AMIMURA and S LUTZKI (1981, 1982) ◮ Dag acceptors and dag-to-tree transducers ◮ They proved a couple of technical properties, no algorithms ◮ We investigate their model with some adjustments: WANT ◮ not only adjacent leaves can be connected BELIEVE BOY GIRL Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 10 / 21
Dag automata (2) K AMIMURA and S LUTZKI (1981, 1982) ◮ Dag acceptors and dag-to-tree transducers ◮ They proved a couple of technical properties, no algorithms ◮ We investigate their model with some adjustments: WANT ◮ not only adjacent leaves can be connected BELIEVE ◮ top-down transducers instead of bottom-up ◮ we introduce weights (probabilities) BOY GIRL Quernheim and Knight Towards Probabilistic Acceptors and Transducers for Feature Structures July 12, 2012 10 / 21
Recommend
More recommend
