lexical translation models 1
play

Lexical Translation Models 1 January 24, 2013 Thursday, January - PowerPoint PPT Presentation

Jan 12, 2023 •197 likes •1.23k views

Lexical Translation Models 1 January 24, 2013 Thursday, January 24, 13 Lexical Translation How do we translate a word? Look it up in the dictionary Haus : house, home, shell, household Multiple translations Different word senses,

  1. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Start with a foreign sentence and a target length. Thursday, January 24, 13

  2. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Thursday, January 24, 13

  3. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Thursday, January 24, 13

  4. Example 0 1 2 3 4 das Haus ist klein NULL the 1 3 4 2 Thursday, January 24, 13

  5. Example 0 1 2 3 4 das Haus ist klein NULL the 1 3 4 2 Thursday, January 24, 13

  6. Example 0 1 2 3 4 das Haus ist klein NULL the house 1 3 4 2 Thursday, January 24, 13

  7. Example 0 1 2 3 4 das Haus ist klein NULL the house 1 3 4 2 Thursday, January 24, 13

  8. Example 0 1 2 3 4 das Haus ist klein NULL the house is 1 3 4 2 Thursday, January 24, 13

  9. Example 0 1 2 3 4 das Haus ist klein NULL the house is 1 3 4 2 Thursday, January 24, 13

  10. Example 0 1 2 3 4 das Haus ist klein NULL the house is small 1 3 4 2 Thursday, January 24, 13

  11. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Thursday, January 24, 13

  12. Example 0 1 2 3 4 das Haus ist klein NULL 1 3 4 2 Thursday, January 24, 13

  13. Example 0 1 2 3 4 das Haus ist klein NULL the 1 3 4 2 Thursday, January 24, 13

  14. Example 0 1 2 3 4 das Haus ist klein NULL the 1 3 4 2 Thursday, January 24, 13

  15. Example 0 1 2 3 4 das Haus ist klein NULL house the 1 3 4 2 Thursday, January 24, 13

  16. Example 0 1 2 3 4 das Haus ist klein NULL house the 1 3 4 2 Thursday, January 24, 13

  17. Example 0 1 2 3 4 das Haus ist klein NULL house the is 1 3 4 2 Thursday, January 24, 13

  18. Example 0 1 2 3 4 das Haus ist klein NULL house the is 1 3 4 2 Thursday, January 24, 13

  19. Example 0 1 2 3 4 das Haus ist klein NULL house the is small 1 3 4 2 Thursday, January 24, 13

  20. Finding the Viterbi Alignment a ⇤ = arg a 2 [0 , 1 ,...,n ] m p ( a | e , f ) max p ( e , a | f ) = arg max a 0 p ( e , a 0 | f ) P a 2 [0 , 1 ,...,n ] m = arg a 2 [0 , 1 ,...,n ] m p ( e , a | f ) max 1 n a ∗ i = arg max 1 + np ( e i | f a i ) a i =0 n = arg max a i =0 p ( e i | f a i ) Thursday, January 24, 13

  21. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  22. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  23. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  24. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  25. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  26. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  27. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  28. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  29. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  30. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  31. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  32. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  33. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  34. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  35. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  36. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  37. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  38. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  39. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  40. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  41. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  42. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  43. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  44. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  45. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  46. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  47. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  48. Finding the Viterbi Alignment 0 1 2 3 4 das Haus ist klein NULL the home is little 1 3 4 2 Thursday, January 24, 13

  49. Learning Lexical Translation Models • How do we learn the parameters p ( e | f ) • “Chicken and egg” problem • If we had the alignments, we could estimate the parameters (MLE) • If we had parameters, we could find the most likely alignments Thursday, January 24, 13

  50. EM Algorithm • pick some random (or uniform) parameters • Repeat until you get bored (~ 5 iterations for lexical translation models) • using your current parameters, compute “expected” alignments for every target word token in the training data (on board) p ( a i | e , f ) • keep track of the expected number of times f translates into e throughout the whole corpus • keep track of the expected number of times that f is used as the source of any translation • use these expected counts as if they were “real” counts in the standard MLE equation Thursday, January 24, 13

  51. EM for Model 1 Thursday, January 24, 13

  52. EM for Model 1 Thursday, January 24, 13

  53. EM for Model 1 Thursday, January 24, 13

  54. EM for Model 1 Thursday, January 24, 13

  55. EM for Model 1 Thursday, January 24, 13

  56. Convergence Thursday, January 24, 13

  57. Evaluation • Since we have a probabilistic model, we can evaluate perplexity . 1 ( e , f ) ∈ D | e | log Q ( e , f ) ∈ D p ( e | f ) − P PPL = 2 Iter ∞ Iter 1 Iter 2 Iter 3 Iter 4 ... -log likelihood - 7.66 7.21 6.84 ... -6 perplexity - 2.42 2.30 2.21 ... 2 Thursday, January 24, 13

  58. Alignment Error Rate Thursday, January 24, 13

  59. Alignment Error Rate Possible links P Thursday, January 24, 13

  60. Alignment Error Rate Possible links P Thursday, January 24, 13

  61. Alignment Error Rate Possible links Sure links S P Thursday, January 24, 13

  62. Alignment Error Rate Possible links Sure links S P Thursday, January 24, 13

  63. Alignment Error Rate Possible links Sure links S P Precision( A, P ) = | P ∩ A | | A | Thursday, January 24, 13

  64. Alignment Error Rate Possible links Sure links S P Precision( A, P ) = | P ∩ A | Recall( A, S ) = | S ∩ A | | A | | S | Thursday, January 24, 13

Recommend

Chapter 4 Word-based models Statistical Machine Translation Lexical Translation How to

Chapter 4 Word-based models Statistical Machine Translation Lexical Translation How to

Chapter 4 Word-based models Statistical Machine Translation Lexical Translation How to translate a word look up in dictionary Haus house, building, home, household, shell. Multiple translations some more frequent than others

1.2k views • 61 slides
Three models for discriminative machine Three models for discriminative machine translation using

Three models for discriminative machine Three models for discriminative machine translation using

Three models for discriminative machine Three models for discriminative machine translation using Global Lexical Selection translation using Global Lexical Selection and Sentence Reconstruction and Sentence Reconstruction Sriram Venkatapathy

645 views • 32 slides
Using Word Embeddings to Enforce Document-Level Lexical Consistency in Machine Translation Eva

Using Word Embeddings to Enforce Document-Level Lexical Consistency in Machine Translation Eva

Using Word Embeddings to Enforce Document-Level Lexical Consistency in Machine Translation Eva Martnez Garcia Carles Creus Cristina Espaa-Bonet Llus Mrquez EAMT 2017 May 30th Prague Outline Motivation 1 Lexical Consistency

808 views • 47 slides
Lexical Syntax for Dependency-based Language Models Statistical Machine Translation Incremental

Lexical Syntax for Dependency-based Language Models Statistical Machine Translation Incremental

Hany Hassan Introduction Syntax for Phrase-based SMT Supertagged Phrase-based SMT From Supertagged to Lexical Syntax for Dependency-based Language Models Statistical Machine Translation Incremental Dependency-based Language Model (IDLM)

804 views • 67 slides
4CSLL5 IBM Translation Models Martin Emms October 22, 2020 4CSLL5 IBM Translation Models IBM

4CSLL5 IBM Translation Models Martin Emms October 22, 2020 4CSLL5 IBM Translation Models IBM

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models Martin Emms October 22, 2020 4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments IBM Model 1 definitions 4CSLL5 IBM Translation Models IBM models

1.23k views • 103 slides
4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments Martin Emms

4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments Martin Emms

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments Martin Emms IBM Model 1 definitions October 22, 2020 4CSLL5 IBM Translation Models 4CSLL5 IBM

571 views • 7 slides
4CSLL5 IBM Translation Models Martin Emms October 23, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models Martin Emms October 23, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models Martin Emms October 23, 2020 4CSLL5 IBM Translation Models Parameter learning (brute force) Introduction The brute force EM algorithm defined A formula for p ( a | o , s ) Examples

759 views • 50 slides
4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models Parameter learning (efficient) How to sum alignments efficiently Efficient EM via p (( j , i ) a | o , s ) 4CSLL5 IBM

658 views • 52 slides
Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC

Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC

Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC Outline Review: EM for HMMs Machine Translation Alignment Limited Sequence Models Maximum Entropy Markov Models Conditional Random Fields

1.37k views • 100 slides
Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC

Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC

Latent Models: Sequence Models Beyond HMMs and Machine Translation Alignment CMSC 473/673 UMBC Outline Review: EM for HMMs Machine Translation Alignment Limited Sequence Models Maximum Entropy Markov Models Conditional Random Fields

1.2k views • 101 slides
Cross-Cutting Models of Lexical Semantics Joseph Reisinger and Raymond Mooney Distributional

Cross-Cutting Models of Lexical Semantics Joseph Reisinger and Raymond Mooney Distributional

Cross-Cutting Models of Lexical Semantics Joseph Reisinger and Raymond Mooney Distributional Lexical Semantics Represent meaning as a point/vector in a high-dimensional space Word relatedness correlates with some distance metric

804 views • 32 slides
Heterogeneous Lexical Resources MultiJEDI ERC 259234 Lexical Resource Lexical Resource Lexical

Heterogeneous Lexical Resources MultiJEDI ERC 259234 Lexical Resource Lexical Resource Lexical

A Robust Approach to Aligning Heterogeneous Lexical Resources MultiJEDI ERC 259234 Lexical Resource Lexical Resource Lexical Resource Why combine resources? e.g., named entities, new senses dozens of new languages

1.18k views • 64 slides
Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional

Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional

Chapter 11 Tree-based models Statistical Machine Translation Tree-Based Models Traditional statistical models operate on sequences of words Many translation problems can be best explained by pointing to syntax reordering, e.g., verb

1.39k views • 120 slides
Rule-based approach: attempts to program grammatical and lexical rules. Largely failed and

Rule-based approach: attempts to program grammatical and lexical rules. Largely failed and

A brief history of Machine Translation (MT) http://blog.systransoft.com/how-does-neural-machine-translation-work/ Rule-based approach: attempts to program grammatical and lexical rules. Largely failed and showed how limited grammar is in

207 views • 10 slides
Lexical Databases Like a dictionary Lexical properties of interest to psycholinguists

Lexical Databases Like a dictionary Lexical properties of interest to psycholinguists

A General Introduction to Lexical Databases Emmanuel Keuleers Department of Experimental Psychology Ghent University EMLAR 2015 - Utrecht, April 15-17, 2015 What can you find in a lexical database? How can you find it? Lexical

905 views • 45 slides
Data-driven sense induction for disambiguation and lexical selection in translation Marianna

Data-driven sense induction for disambiguation and lexical selection in translation Marianna

Data-driven sense induction for disambiguation and lexical selection in translation Marianna Apidianaki, University Paris 7 22 October 2008 Plan of the presentation a. Towards data-driven sense acquisition and Word Sense Disambiguation (WSD)

808 views • 47 slides
Machine Translation Classification of divergences Classical and Statistical Approaches

Machine Translation Classification of divergences Classical and Statistical Approaches

Session 4: Interlingua-based MT Dorr (1992, 1994): UNITRAN system Machine Translation Classification of divergences Classical and Statistical Approaches Lexical Conceptual Structure Translation mappings between syntactic

130 views • 12 slides
Sequence to Sequence Models for Machine Translation CMSC 723 / LING 723 / INST 725 Marine

Sequence to Sequence Models for Machine Translation CMSC 723 / LING 723 / INST 725 Marine

Sequence to Sequence Models for Machine Translation CMSC 723 / LING 723 / INST 725 Marine Carpuat Slides & figure credits: Graham Neubig Machine Translation 3 problems Translation system Input: source sentence F Output:

349 views • 20 slides
Empirical Methods in Natural Language Processing Lecture 15 Machine translation (II): Word-based

Empirical Methods in Natural Language Processing Lecture 15 Machine translation (II): Word-based

Empirical Methods in Natural Language Processing Lecture 15 Machine translation (II): Word-based models and the EM algorithm Philipp Koehn 25 February 2008 Philipp Koehn EMNLP Lecture 15 25 February 2008 1 Lexical translation How to

906 views • 32 slides
Empirical Methods in Natural Language Processing Lecture 15 Machine translation (II): Word-based

Empirical Methods in Natural Language Processing Lecture 15 Machine translation (II): Word-based

Empirical Methods in Natural Language Processing Lecture 15 Machine translation (II): Word-based models and the EM algorithm Philipp Koehn 25 February 2008 Philipp Koehn EMNLP Lecture 15 25 February 2008 1 Lexical translation How to

628 views • 16 slides
Decoding and Inference with Syntactic Translation Models Machine Translation Lecture 15

Decoding and Inference with Syntactic Translation Models Machine Translation Lecture 15

Decoding and Inference with Syntactic Translation Models Machine Translation Lecture 15 Instructor: Chris Callison-Burch TAs: Mitchell Stern, Justin Chiu Website: mt-class.org/penn jon-ga ringo-o tabeta jon-ga ringo-o tabeta ringo-o tabeta

1.16k views • 83 slides
Machine Translation/ Sequence-to-sequence Models Graham Neubig Site

Machine Translation/ Sequence-to-sequence Models Graham Neubig Site

CS11-737 Multilingual NLP Machine Translation/ Sequence-to-sequence Models Graham Neubig Site http://demo.clab.cs.cmu.edu/11737fa20/ Language Models Language models are generative models of text s ~ P(x) The Malfoys! said Hermione.

584 views • 37 slides
Syntax Directed Translation Attribute grammar and translation schemes cs4713 1 Typical

Syntax Directed Translation Attribute grammar and translation schemes cs4713 1 Typical

Syntax Directed Translation Attribute grammar and translation schemes cs4713 1 Typical implementation of languages Source Lexical Analyzer Program input Program Tokens Syntax Analyzer Parse tree / Semantic Analyzer Results

442 views • 18 slides
Computational Models of Discourse: Lexical Chains; Centering Theory Caroline Sporleder

Computational Models of Discourse: Lexical Chains; Centering Theory Caroline Sporleder

Computational Models of Discourse: Lexical Chains; Centering Theory Caroline Sporleder Universit at des Saarlandes Sommersemester 2009 06.05.2009 Caroline Sporleder csporled@coli.uni-sb.de Computational Models of Discourse Lexical Chains

1.79k views • 137 slides

More recommend