Response-based Learning for Grounded Grounded SMT Riezler, Machine - PowerPoint PPT Presentation

Response- based Learning for Response-based Learning for Grounded Grounded SMT Riezler, Machine Translation Simianer, Haas Response- based Learning Stefan Riezler, Patrick Simianer, Carolin Haas Grounded SMT Algorithms Department of Computational Linguistics Experiments Heidelberg University, Germany Discussion 1 / 23

Response-based Learning Response- based Learning for Grounded SMT Riezler, Simianer, Haas Extract supervision signal from extrinsic response to Response- based predicted structure. Learning Prediction is tried out in extrinsic task: Grounded SMT approved as positive training example in case of positive Algorithms task-based feedback, Experiments in addition to or instead of learning from given gold Discussion standard annotations. 2 / 23

Response-based Learning Response- based Learning for Grounded SMT Riezler, Simianer, Haas Extract supervision signal from extrinsic response to Response- based predicted structure. Learning Prediction is tried out in extrinsic task: Grounded SMT approved as positive training example in case of positive Algorithms task-based feedback, Experiments in addition to or instead of learning from given gold Discussion standard annotations. 2 / 23

Response-based Learning for MT Response- based Learning for Grounded SMT Riezler, Simianer, Haas Try out most probable translation in extrinsic task, and approve as reference translation in case of positive Response- based feedback. Learning Advantages over learning from references only: Grounded SMT Algorithms Reproducability : Multiple system translations can be Experiments converted into references. Reachability : References are necessarily in decoder Discussion search space (compared to independently created human reference translations). 3 / 23

Response-based Learning for MT Response- based Learning for Grounded SMT Riezler, Simianer, Haas Try out most probable translation in extrinsic task, and approve as reference translation in case of positive Response- based feedback. Learning Advantages over learning from references only: Grounded SMT Algorithms Reproducability : Multiple system translations can be Experiments converted into references. Reachability : References are necessarily in decoder Discussion search space (compared to independently created human reference translations). 3 / 23

Grounded Language Learning / Semantic Parsing Response- based Learning for Grounded SMT Grounded language learning : Successful Riezler, Simianer, Haas communication of meaning defined as successful interaction in a task Response- based ([Roy, 2002, Yu and Ballard, 2004, Yu and Siskind, 2013], Learning inter alia ). Grounded SMT Algorithms Semantic parsing : Successful execution of a meaning Experiments representation in a simulated world defined as returning Discussion the correct answer from a knowledge base (G EOQUERY , [Wong and Mooney, 2006]; A TIS [Zettlemoyer and Collins, 2009], F REEBASE [Cai and Yates, 2013], inter alia ). 4 / 23

Grounded Language Learning / Semantic Parsing Response- based Learning for Grounded SMT Grounded language learning : Successful Riezler, Simianer, Haas communication of meaning defined as successful interaction in a task Response- based ([Roy, 2002, Yu and Ballard, 2004, Yu and Siskind, 2013], Learning inter alia ). Grounded SMT Algorithms Semantic parsing : Successful execution of a meaning Experiments representation in a simulated world defined as returning Discussion the correct answer from a knowledge base (G EOQUERY , [Wong and Mooney, 2006]; A TIS [Zettlemoyer and Collins, 2009], F REEBASE [Cai and Yates, 2013], inter alia ). 4 / 23

Response-based Semantic Parsing Response- based Learning for Grounded SMT Riezler, Simianer, Haas Learn semantic parsers from question-answer pairs Response- based without recurring to annotated logical forms Learning [Kwiatowski et al., 2013, Berant et al., 2013, Grounded SMT Goldwasser and Roth, 2014]. Algorithms Experiments Term response driven learning coined by Discussion [Clarke et al., 2010]. 5 / 23

Grounding SMT in Semantic Parsing Response- based Learning for Grounded SMT Riezler, Simianer, Haas QA-scenario: Response- Question is translated successfully if correct answer based Learning is returned based only on the translation of the question. Grounded SMT Semantic parsing realization: Algorithms Translation quality defined by ability of semantic parser to Experiments construct a meaning representation from the Discussion translated query , which returns correct answer when executed against database. 6 / 23

Grounding SMT in Semantic Parsing Response- based Learning for Grounded SMT Riezler, Simianer, Haas QA-scenario: Response- Question is translated successfully if correct answer based Learning is returned based only on the translation of the question. Grounded SMT Semantic parsing realization: Algorithms Translation quality defined by ability of semantic parser to Experiments construct a meaning representation from the Discussion translated query , which returns correct answer when executed against database. 6 / 23

Response-based Learning Cycle Response- based Learning for Grounded SMT Riezler, Simianer, Haas Response- based Learning Grounded SMT Algorithms Experiments Discussion 7 / 23

Response-based Learning Cycle Response- based Learning for Grounded SMT Riezler, Simianer, Haas Advantages over learning from independent references: Response- Task-approval of system translations avoids problem of based Learning (un)reachability of references by decoder. Grounded SMT Structural and lexical variation of predicted and approved Algorithms translations broadens learning capabilities , Experiments Task-approved supervision signal allows learn optimally Discussion for task-specific aspects of translation quality . 8 / 23

Response-based Learning Cycle Response- based Learning for Grounded SMT Riezler, Simianer, Haas Advantages over learning from independent references: Response- Task-approval of system translations avoids problem of based Learning (un)reachability of references by decoder. Grounded SMT Structural and lexical variation of predicted and approved Algorithms translations broadens learning capabilities , Experiments Task-approved supervision signal allows learn optimally Discussion for task-specific aspects of translation quality . 8 / 23

Response-based Learning Cycle Response- based Learning for Grounded SMT Riezler, Simianer, Haas Advantages over learning from independent references: Response- Task-approval of system translations avoids problem of based Learning (un)reachability of references by decoder. Grounded SMT Structural and lexical variation of predicted and approved Algorithms translations broadens learning capabilities , Experiments Task-approved supervision signal allows learn optimally Discussion for task-specific aspects of translation quality . 8 / 23

Example Response- based Learning for Grounded SMT Riezler, Simianer, Haas Response- German Nenne prominente Erhebungen in den USA based Learning orig. query Name prominent elevations in the USA ✔ Grounded SMT Algorithms sys. trans Give prominent surveys in the US – Experiments sys. trans Give prominent heights in the US ✔ Discussion 9 / 23

Response-based Online Learning Response- based Execution function e ( y ) ∈ { 1 , 0 } tests whether semantic Learning for Grounded SMT parse for y receives same answer as gold standard. Riezler, Simianer, Haas Cost function c ( y ( i ) , y ) = ( 1 − BLEU ( y ( i ) , y )) based on sentence-level BLEU [Nakov et al., 2012]. Response- based y + is surrogate gold-standard translation w/ positive Learning Grounded SMT feedback, high model score s , and low cost c : Algorithms y + = � � Experiments s ( x ( i ) , y ; w ) − c ( y ( i ) , y ) arg max . Discussion y ∈ Y ( x ( i ) ): e ( y )= 1 y − opposite : negative feedback, high score and cost: y − = � � s ( x ( i ) , y ; w ) + c ( y ( i ) , y ) arg max . y ∈ Y ( x ( i ) ): e ( y )= 0 10 / 23

Response-based Online Learning Response- based Execution function e ( y ) ∈ { 1 , 0 } tests whether semantic Learning for Grounded SMT parse for y receives same answer as gold standard. Riezler, Simianer, Haas Cost function c ( y ( i ) , y ) = ( 1 − BLEU ( y ( i ) , y )) based on sentence-level BLEU [Nakov et al., 2012]. Response- based y + is surrogate gold-standard translation w/ positive Learning Grounded SMT feedback, high model score s , and low cost c : Algorithms y + = � � Experiments s ( x ( i ) , y ; w ) − c ( y ( i ) , y ) arg max . Discussion y ∈ Y ( x ( i ) ): e ( y )= 1 y − opposite : negative feedback, high score and cost: y − = � � s ( x ( i ) , y ; w ) + c ( y ( i ) , y ) arg max . y ∈ Y ( x ( i ) ): e ( y )= 0 10 / 23