4CSLL5 Advanced Computational Linguistics Introduction Phrase - PowerPoint PPT Presentation

4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Introduction Phrase Based Machine Trans Martin Emms Learning the Phrase Translation Table November 4, 2020 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans Introduction Motivation Intro and Learning ◮ Word-Based Models translate words as atomic units ◮ Phrase-Based Models translate phrases as atomic units ◮ Advantages: ◮ many-to-many translation can handle non-compositional phrases ◮ use of local context in translation ◮ the more data, the longer phrases can be learned

4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans Introduction Introduction Phrase-Based Model Compared to IBM Model ◮ recall IBM models assumed a hidden alignment between s and o , giving a formula p ( o , a | s ) and so a formula for p ( o , a , s ) as go home source he does not p ( o , a | s ) × p ( s ) ◮ phrase-based models assume hidden segmentations of s and o into K phrases ¯ s 1 : K and ¯ o 1 : K er geht ja nicht nach hause observed ◮ phrase-based models also assume a hidden mapping from the phrases ¯ s to the phrases ¯ o . This 1-to-1, and generally not order preserving. ◮ we will have a formula for formula for p ( ¯ o , τ, ¯ s ) as ◮ source is segmented in phrases p ( ¯ o , τ | ¯ s ) × p ( ¯ s ) ◮ each source phrase is translated into observed phrase ◮ observed phrases are reordered 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans Introduction Introduction Example Constructing a Phrase-Based Translation go home source he does not er geht ja nicht nach hause observed ◮ Task: translate a certain German ’observed’ sentence into ’source’ English ◮ assume s 1:5 = he does not go home and o 1:6 = er geht ja nicht nach hause ◮ possible segmentation of s 1:5 into ¯ s 1 : 4 is er geht ja nicht nach hause ¯ s 1 = s 1 : 1 = he , ¯ s 2 = s 2 : 3 = does not , ¯ s 3 = s 4 : 4 = go , ¯ s 4 = s 5 : 5 = home ◮ possible segmentation of o 1:6 into ¯ o 1 : 4 is o 1 = o 1 : 1 = er , ¯ ¯ o 2 = o 2 : 2 = geht , ¯ o 3 = o 3 : 4 = ja nicht , ¯ o 4 = o 5 : 6 = nach hause ◮ possible mapping τ from ¯ s to ¯ o is τ (1) = 1, τ (2) = 3, τ (3) = 2, τ (4) = 4

4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans Introduction Introduction Constructing a Phrase-Based Translation Constructing a Phrase-Based Translation ◮ Assume a ’phrase-table’ giving for many possible ’phrases’ ¯ o in the he observed German, possible ’phrases’ ¯ s in potential source English er geht ja nicht nach hause he is yes not after house er it are is do not to home , it goes , of course does not according to chamber , he go , is not in at home it is not home he will be is not under house er geht ja nicht nach hause it goes does not return home he goes do not do not is to are following is after all not after does not to not is not are not is not a ◮ Pick a phrase ¯ o = ’er’ in observed, choose ’he’ as ¯ s 1 in source ◮ the phrase-based translation will be built with these ingredients 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans Introduction Introduction Constructing a Phrase-Based Translation Constructing a Phrase-Based Translation he does not go he does not er ja nicht er geht ja nicht er geht ja nicht nach hause er geht ja nicht nach hause ◮ Pick a phrase ¯ o = ’geht’ in observed, choose ’go’ as ¯ s 3 in source ◮ Pick a phrase ¯ o = ’ja nicht’ in observed, choose ’does not’ as ¯ s 2 in source ◮ NB: allowed to choose ¯ o phrases out of sequence; ¯ s phrases chosen in sequence ◮ NB: phrases may have multiple words: many-to-many translation

4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans Introduction Introduction Constructing a Phrase-Based Translation ◮ just constructed one particular translation, could have constructed many, go home he does not many others using the available phrases pairs ◮ need probabilistic model which favours one over the other ◮ need to set parameters of that model er geht ja nicht nach hause → these won’t be learned by EM but instead some are (heuristically) derived from IBM models, and some just set by common sense ◮ to find high scoring translations need to manage somehow an exponential er geht ja nicht nach hause search space → ’beam search’ heuristic ◮ Pick a phrase ¯ o = ’nach hause’ in observed, choose ’home’ as ¯ s 4 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans Learning the Phrase Translation Table Learning the Phrase Translation Table Learning a Phrase Translation Table Learning ctd: alignment both ways ◮ Task: learn the model from a parallel corpus ◮ Three stages: ◮ word alignment: using IBM models or other method ◮ extraction of phrase pairs a : Ger → Eng a : Eng → Ger ◮ scoring phrase pairs do IBM model learning in both directions, and find best alignments both ways

4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans Learning the Phrase Translation Table Learning the Phrase Translation Table Learning ctd: unite alignment Learning ctd: extract consistent phrase pairs michael michael davon davon bleibt dass haus bleibt geht haus dass geht aus aus im er im er , , michael michael assumes assumes that that he he will will stay stay in in the the house house obvious 1-to-N, N-to-1 cases eg: for each training pair, merge these alignments ( that – dass ) then extract phrase pair consistent with this merge: ( assumes – geht davon aus ) next slides show a few cases ( in the – im ) 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans 4CSLL5 ’Advanced Computational Linguistics’ Phrase Based Machine Trans Learning the Phrase Translation Table Learning the Phrase Translation Table Learning ctd: extract consistent phrase pairs Learning ctd: extract consistent phrase pairs michael michael davon bleibt bleibt haus davon geht haus dass aus geht dass im aus im er , er � � � ���� ����� � � � � � , � � � � � � � ���� � � � ����� � � � � � � � � michael � � � � � � � � � � � � �� �� ����� ����� �� � �� � � � � � � � � � � � � � � � michael � � � � � � � � � � � � assumes �� �� � �� �� � � � � � � � � � � � � � assumes � �� �� � ����� ����� � � �� � � �� � � � � � � �� �� �� � � �� � � � � � � � � � � � � ����� ����� �� �� �� �� � �� � �� ����� ����� �� �� � � � � � � � � � � that � � � � � � � � � � � � that �� �� ����� ����� � � � � � � � � � � � � �� �� � � � � � � � � � � � � �� �� he � � � � � � � � � � � � � � � � � � � � � � � � he ����� ����� � � � � � � � � � � � � � � � � � � � � � � � � � � ����� ����� � � � � ����� ����� � � will � � � � � � � � � � � � � � � � � � � � � � will � � ����� ����� � � � � � � � � � � � � � � � � � � � � � � � � � � stay � � � � � � � � � � stay ����� ����� �� �� � � � � �� �� � � in in � � �� �� � � the �� �� the � � �� �� house �� �� � � house �� �� �� �� �� �� N-to-N cases: basically taping together adjacent smaller cases. N-to-N cases: ( will stay – bleibt ) + ( in the house — im haus ) eg. ( in the – im ) + ( house – haus ) → ( will stay in the house — im haus bleibt ) → ( in the house — im haus )