deciphering foreign language
play

Deciphering Foreign Language NLP Sujith Ravi and Kevin Knight - PowerPoint PPT Presentation

Nov 22, 2023 •120 likes •794 views

Deciphering Foreign Language NLP Sujith Ravi and Kevin Knight sravi@usc.edu,knight@isi.edu Information Sciences Institute University of Southern California Statistical Machine Translation (MT) Bilingual text Translation tables Current

  1. Deciphering Foreign Language NLP Sujith Ravi and Kevin Knight sravi@usc.edu,
knight@isi.edu Information Sciences Institute University of Southern California

  2. Statistical Machine Translation (MT) Bilingual text Translation tables Current (Spanish) Garcia y asociados associates/ MT systems asociados : 0.8 (English) Garcia and associates groups/ TRAIN (Spanish) sus grupos están en Europa grupos : 0.9 (English) its groups are in Europe : : : : : 2

  3. Statistical Machine Translation (MT) Bilingual text Translation tables Current Spanish/English (Spanish) Garcia y asociados associates/ MT systems asociados : 0.8 (English) Garcia and associates groups/ TRAIN (Spanish) sus grupos están en Europa grupos : 0.9 (English) its groups are in Europe : : : : : 2

  4. Statistical Machine Translation (MT) Bilingual text Translation tables Current Spanish/English (Spanish) Garcia y asociados associates/ MT systems asociados : 0.8 (English) Garcia and associates Japanese/German groups/ TRAIN (Spanish) sus grupos están en Europa grupos : 0.9 (English) its groups are in Europe : : : : : 2

  5. Statistical Machine Translation (MT) Bilingual text Translation tables Current Spanish/English (Spanish) Garcia y asociados associates/ MT systems asociados : 0.8 (English) Garcia and associates Japanese/German groups/ TRAIN (Spanish) sus grupos están en Europa grupos : 0.9 Malayalam/English (English) its groups are in Europe : : : : : 2

  6. Statistical Machine Translation (MT) Bilingual text Translation tables Current Spanish/English (Spanish) Garcia y asociados associates/ MT systems asociados : 0.8 (English) Garcia and associates Japanese/German groups/ TRAIN (Spanish) sus grupos están en Europa grupos : 0.9 Malayalam/English (English) its groups are in Europe : : : Swahili/German : ... : 2

  7. Statistical Machine Translation (MT) Bilingual text Translation tables Current Spanish/English (Spanish) Garcia y asociados associates/ MT systems asociados : 0.8 (English) Garcia and associates Japanese/German groups/ TRAIN (Spanish) sus grupos están en Europa grupos : 0.9 Malayalam/English (English) its groups are in Europe : : : Swahili/German : ... : BOTTLENECK 2

  8. Statistical Machine Translation (MT) Bilingual text Translation tables Current Spanish/English (Spanish) Garcia y asociados associates/ MT systems asociados : 0.8 (English) Garcia and associates Japanese/German groups/ TRAIN (Spanish) sus grupos están en Europa grupos : 0.9 Malayalam/English (English) its groups are in Europe : : : Swahili/German : ... : Can we get rid BOTTLENECK of parallel data? 2

  9. Statistical Machine Translation (MT) Bilingual text Translation tables Current Spanish/English (Spanish) Garcia y asociados associates/ MT systems asociados : 0.8 (English) Garcia and associates Japanese/German groups/ TRAIN (Spanish) sus grupos están en Europa grupos : 0.9 Malayalam/English (English) its groups are in Europe : : : Swahili/German : ... : Can we get rid BOTTLENECK of parallel data? Monolingual corpora Translation tables Spanish text associates/ asociados : 0.8 TRAIN : : English text : 2

  10. Statistical Machine Translation (MT) Bilingual text Translation tables Current Spanish/English (Spanish) Garcia y asociados associates/ MT systems asociados : 0.8 (English) Garcia and associates Japanese/German groups/ TRAIN (Spanish) sus grupos están en Europa grupos : 0.9 Malayalam/English (English) its groups are in Europe : : : Swahili/German : ... : Can we get rid BOTTLENECK of parallel data? Monolingual corpora Translation tables Spanish text associates/ PLENTY asociados : 0.8 TRAIN : : English text : PLENTY 2

  11. Machine Translation without parallel data Getting Rid of Parallel Data • MT system trained on non-parallel data ➡ useful for rare language-pairs (limited/no parallel data) 3

  12. Machine Translation without parallel data Getting Rid of Parallel Data • MT system trained on non-parallel data ➡ useful for rare language-pairs (limited/no parallel data) Monolingual corpora Translation tables Spanish text associates/ asociados : 0.8 TRAIN : : English text : 3

  13. Machine Translation without parallel data Getting Rid of Parallel Data • MT system trained on non-parallel data ➡ useful for rare language-pairs (limited/no parallel data) Monolingual corpora Translation tables Spanish text associates/ asociados : 0.8 TRAIN : : English text : • Goal: not to beat existing MT systems, instead • Can we build a reasonably good MT system from scratch without any parallel data? ➡ monolingual resources available in plenty 3

  14. Machine Translation without Related Work parallel data • Extracting bilingual lexical connections from comparable corpora ➡ exploit word context frequencies (Fung, 1995; Rapp, 1995; Koehn & Knight, 2001) ➡ Canonical Correlation Analysis (CCA) method (Haghighi & Klein, 2008) • Mining parallel sentence pairs for MT training using comparable corpora (Munteanu et al., 2004) ➡ need dictionary, some initial parallel data 4

  15. Our Contributions NLP New ✓ MT system built from scratch without parallel data ➡ novel decipherment approach for translation ➡ novel methods for training translation models from non-parallel text ➡ Bayesian training for IBM 3 translation model ✓ Novel methods to deal with large-scale vocabularies inherent in MT problems ✓ Empirical studies for MT decipherment 5

  16. Rest of this Talk • Introduction • Related Work • New Idea for Language Translation ➡ Step 1: Word Substitution ➡ Step 2: Foreign Language as a Cipher • Conclusion 6

  17. Cracking the MT Code New “When I look at an article in Spanish, I Warren Weaver (1947) say to myself, this is really English, but it has been encoded in some strange symbols. Now I will proceed to decode...” • Ciphertext: este es un sistema de cifrado complejo (Spanish) 7

  18. Cracking the MT Code New “When I look at an article in Spanish, I Warren Weaver (1947) say to myself, this is really English, but it has been encoded in some strange symbols. Now I will proceed to decode...” • Ciphertext: este es un sistema de cifrado complejo (Spanish) • (English) Plaintext: this is a complex cipher 7

  19. New MT Decipherment without Parallel Data f Spanish corpus El portal web permite la búsqueda por todo tipo de métodos. Por un lado, Wikileaks ha ordenado la documentación por diferentes categorías atendiendo a los hechos más notables. Desde el tipo de suceso (evento criminal, fuego amigo, ...

  20. New MT Decipherment without Parallel Data ? e f English Spanish corpus P(e) P(f | e) El portal web permite la búsqueda por todo English-to-Spanish English tipo de métodos. Por un lado, Wikileaks ha Translation Model Language Model ordenado la documentación por diferentes key categorías atendiendo a los hechos más notables. Desde el tipo de suceso (evento criminal, fuego amigo, ...

  21. New MT Decipherment without Parallel Data English corpus (CNN) WikiLeaks website publishes classified military documents from Iraq. The whistle-blower website WikiLeaks published nearly 400,000 classified military documents from the Iraq war on Friday, calling it the largest classified military leak in history,.... Language Model Training ? e f English Spanish corpus P(e) P(f | e) El portal web permite la búsqueda por todo English-to-Spanish English tipo de métodos. Por un lado, Wikileaks ha Translation Model Language Model ordenado la documentación por diferentes key categorías atendiendo a los hechos más notables. Desde el tipo de suceso (evento criminal, fuego amigo, ...

  22. New MT Decipherment without Parallel Data English corpus (CNN) WikiLeaks website publishes classified military documents from Iraq. The whistle-blower website WikiLeaks published nearly 400,000 classified military documents from the Iraq war on Friday, calling it the largest classified military For each f leak in history,.... • alignments = hidden Language Model Training • e translation = hidden ? e f English Spanish corpus P(e) P(f | e) El portal web permite la búsqueda por todo English-to-Spanish English tipo de métodos. Por un lado, Wikileaks ha Translation Model Language Model ordenado la documentación por diferentes key categorías atendiendo a los hechos más notables. Desde el tipo de suceso (evento criminal, fuego amigo, ...

  23. New MT Decipherment without Parallel Data Train parameters θ to maximize probability of English corpus observed foreign text f : (CNN) WikiLeaks website publishes classified argmax θ P θ (f ) ≈ argmax θ ∑ e P θ (e, f) military documents from Iraq. The whistle-blower ≈ argmax θ ∑ e P(e) . P θ (f | e) website WikiLeaks published nearly 400,000 classified military documents from the Iraq war on TRAINING Friday, calling it the largest classified military For each f leak in history,.... • alignments = hidden Language Model Training • e translation = hidden ? e f English Spanish corpus P(e) P(f | e) El portal web permite la búsqueda por todo English-to-Spanish English tipo de métodos. Por un lado, Wikileaks ha Translation Model Language Model ordenado la documentación por diferentes key categorías atendiendo a los hechos más notables. Desde el tipo de suceso (evento criminal, fuego amigo, ...

Recommend

Cryptanalysis and Language Deciphering: A Brute Force Attack on an Undeciphered Writing

Cryptanalysis and Language Deciphering: A Brute Force Attack on an Undeciphered Writing

Cryptanalysis and Language Deciphering: A Brute Force Attack on an Undeciphered Writing System Loh Jia Sheng Colin Dr Perono Cacciafoco, Francesco Loh Jia Sheng ColinDr Perono Cacciafoco, Francesco Cryptanalysis and Language Deciphering:

396 views • 37 slides
20160322 DEFENSE LANGUAGE INSTITUTE FOREIGN LANGUAGE CENTER Agenda Mission, Vision, and

20160322 DEFENSE LANGUAGE INSTITUTE FOREIGN LANGUAGE CENTER Agenda Mission, Vision, and

DEFENSE LANGUAGE INSTITUTE FOREIGN LANGUAGE CENTER 20160322 DEFENSE LANGUAGE INSTITUTE FOREIGN LANGUAGE CENTER Agenda Mission, Vision, and Values DLIFLC Overview and Profile Current Focus and Way Ahead Worldwide Presence 2

746 views • 29 slides
THE ROLE OF TECHNOLOGY IN FOREIGN LANGUAGE LEARNING AND TEACHING David P. Ellis, PhD, PMP

THE ROLE OF TECHNOLOGY IN FOREIGN LANGUAGE LEARNING AND TEACHING David P. Ellis, PhD, PMP

THE ROLE OF TECHNOLOGY IN FOREIGN LANGUAGE LEARNING AND TEACHING David P. Ellis, PhD, PMP National Foreign Language Center University of Maryland SLA - INTERNAL FACTORS Age of onset Aptitude Motivation Discipline SLA -

433 views • 17 slides
MASTER DISSERTATION Letters and Foreign Languages English Language Sciences of the language

MASTER DISSERTATION Letters and Foreign Languages English Language Sciences of the language

Mohamed Khider University of Biskra Faculty of Letters and Languages Department of Foreign Languages MASTER DISSERTATION Letters and Foreign Languages English Language Sciences of the language Submitted and Defended by: Mr.Kherbachi Abdellatif

1.32k views • 99 slides
Integrating Web 2.0 in Foreign Language Instruction Youngju Hong Assistant Professor, Curriculum

Integrating Web 2.0 in Foreign Language Instruction Youngju Hong Assistant Professor, Curriculum

Integrating Web 2.0 in Foreign Language Instruction Youngju Hong Assistant Professor, Curriculum Development Division Defense Language Institute Foreign Language Center This speech/presentation is authorized by the Defense Language Institute

589 views • 10 slides
Plan of Action Foreign Language Department 2014-2021 Planning for Program Evaluation In

Plan of Action Foreign Language Department 2014-2021 Planning for Program Evaluation In

Plan of Action Foreign Language Department 2014-2021 Planning for Program Evaluation In 2007-2008 the foreign language teachers grades 5-12 met to: - Identify questions we had about the language program - Identify ways to gather data to

461 views • 21 slides
Grammar as a Foreign Language Authors:- Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov,

Grammar as a Foreign Language Authors:- Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov,

Grammar as a Foreign Language Authors:- Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, Geoffrey Hinton Presented by:- Ved Upadhyay PaperLink:-https://papers.nips.cc/paper/5635-grammar-as-a-foreign- language.pdf Contents

1.23k views • 21 slides
When learning a foreign language, ones grammar improves if one learns to listen to the

When learning a foreign language, ones grammar improves if one learns to listen to the

When learning a foreign language, ones grammar improves if one learns to listen to the language prior to speaking it (Benson, & Heilt, 1978). Both business practitioners and academics listed listening as one of the most important

166 views • 6 slides
The Acquisition of Tense- Aspect Marker -ess/ass- by Korean as a Foreign Language (KFL)

The Acquisition of Tense- Aspect Marker -ess/ass- by Korean as a Foreign Language (KFL)

AATK Conference, Stanford University, July 1, 2012 The Acquisition of Tense- Aspect Marker -ess/ass- by Korean as a Foreign Language (KFL) Learners Sang-Seok Yoon & Yunseong Cheon University of Minnesota Learner Language

430 views • 18 slides
Traditional vs Alternative Assessments Activity 1 DEFENSE LANGUAGE INSTITUTE FOREIGN LANGUAGE

Traditional vs Alternative Assessments Activity 1 DEFENSE LANGUAGE INSTITUTE FOREIGN LANGUAGE

Traditional vs Alternative Assessments Activity 1 DEFENSE LANGUAGE INSTITUTE FOREIGN LANGUAGE CENTER Traditional Assessment Selected-response Multiple-choice test* True/False test* Matching type test*

496 views • 28 slides
RHD ~ Foreign Language Education Students Growing Into Global Citizens 3 Languages 5 Programs 5

RHD ~ Foreign Language Education Students Growing Into Global Citizens 3 Languages 5 Programs 5

RHD ~ Foreign Language Education Students Growing Into Global Citizens 3 Languages 5 Programs 5 Teachers 92 Happy Students Nous aimons parler francais. D lingu Latn First Grade Spanish Class Students learn a new language

376 views • 11 slides
Deciphering the Face Deciphering the Face Aleix M. Martinez Computational Biology Computational

Deciphering the Face Deciphering the Face Aleix M. Martinez Computational Biology Computational

Deciphering the Face Deciphering the Face Aleix M. Martinez Computational Biology Computational Biology and Cognitive Science Lab aleix@ece.osu.edu l i @ d Human-Computer Interaction Politics Human Human face face face face Art

1.04k views • 38 slides
Watch and Wonder: Foreign Languages in Todays Society Question: What is your opinion of each

Watch and Wonder: Foreign Languages in Todays Society Question: What is your opinion of each

Watch and Wonder: Foreign Languages in Todays Society Question: What is your opinion of each of these situations involving the use of foreign language in public? Airline Allegedly Removes Man for Speaking Arabic

530 views • 11 slides
Education Policies, Foreign Language and (Mobile) Technology in the field of Critical Applied

Education Policies, Foreign Language and (Mobile) Technology in the field of Critical Applied

Education Policies, Foreign Language and (Mobile) Technology in the field of Critical Applied Linguistics 1 Cludia Hilsdorf Rocha Unicamp Critical and alternative directions embrace the notion of (local) practice and distrust the grand

684 views • 39 slides
Technology Mediated Foreign Language Classrooms: How to Build Effective and Inclusive Learning

Technology Mediated Foreign Language Classrooms: How to Build Effective and Inclusive Learning

Technology Mediated Foreign Language Classrooms: How to Build Effective and Inclusive Learning Environments By Mariam Manzur Leiva Spanish Instructor & Spanish TA Program Supervisor New College of Florida The incorporation of new Digital

273 views • 15 slides
1 201.1 Deposition in Foreign Jurisdictions for Use in Texas Proceedings 201.2

1 201.1 Deposition in Foreign Jurisdictions for Use in Texas Proceedings 201.2

Sha Sharena Gillila Gilliland nd Parker County District Clerk A document issued by one court to a foreign court, requesting that the foreign court (1) take evidence from a specific person within the foreign jurisdiction or serve process

376 views • 15 slides
Sprac achau hausbi sbildung ldung TU Dres esde den Language courses can be attended by

Sprac achau hausbi sbildung ldung TU Dres esde den Language courses can be attended by

Sprac achau hausbi sbildung ldung TU Dres esde den Language courses can be attended by all foreign students of TU Dresden within the limit of their language budget. Courses are free of charge. For foreign students German as a

306 views • 6 slides
Drupal 8 Multilingual Wonderland Gabor Hojtsy Acquia Foreign language site Multilingual site

Drupal 8 Multilingual Wonderland Gabor Hojtsy Acquia Foreign language site Multilingual site

Drupal 8 Multilingual Wonderland Gabor Hojtsy Acquia Foreign language site Multilingual site Foreign language site Multilingual with translation Multilingual site Foreign language site Multilingual with translation Multilingual site

1.57k views • 42 slides
Applying Learning Technologies to Foreign Language Teaching Attilio Galimberti Local Education

Applying Learning Technologies to Foreign Language Teaching Attilio Galimberti Local Education

ERASMUS+ K2 Preventing Early School Leaving Applying Learning Technologies to Foreign Language Teaching Attilio Galimberti Local Education Authority - Milan, 20 th November 2017 1 How can we improve education? European Commission,

604 views • 59 slides
Applying CEFR to teaching and assessing Applying CEFR to teaching and assessing Chinese as a

Applying CEFR to teaching and assessing Applying CEFR to teaching and assessing Chinese as a

University Language Centres: Going for Gold Overcoming Hurdles Applying CEFR to teaching and assessing Applying CEFR to teaching and assessing Chinese as a foreign language Chinese as a foreign language A proposal from the EBCL Project

580 views • 21 slides
Deciphering*Elements*of* Human*Social*Experience* Bosiljka*Tadic*

Deciphering*Elements*of* Human*Social*Experience* Bosiljka*Tadic*

Deciphering*Elements*of* Human*Social*Experience* Bosiljka*Tadic* Jozef*Stefan*Institute*Ljubljana*and** Complexity*Science*Hub*Vienna* Contents* Brief*answers*to*Stefans*Questions**Q1,Q2,3*

84 views • 7 slides
Teaching Korean Complex Syntax in Context Using an Online Tool (GLOSS) Dohee Koo, Ph.D. Defense

Teaching Korean Complex Syntax in Context Using an Online Tool (GLOSS) Dohee Koo, Ph.D. Defense

Teaching Korean Complex Syntax in Context Using an Online Tool (GLOSS) Dohee Koo, Ph.D. Defense Language Institute Foreign Language Center This speech/presentation is authorized by the Defense Language Institute Foreign Language Center and the

94 views • 8 slides
Inclusion of learners with specific learning differences in teaching English as a foreign

Inclusion of learners with specific learning differences in teaching English as a foreign

Inclusion of learners with specific learning differences in teaching English as a foreign language Bimali Indrarathne Lancaster University Introduction My research Input processing Working memory Dyslexia and language learning Working

770 views • 20 slides
Home and School Meeting September 13, 2012 WELCOME TO NEW STAFF Robert Zakrzewski English

Home and School Meeting September 13, 2012 WELCOME TO NEW STAFF Robert Zakrzewski English

Home and School Meeting September 13, 2012 WELCOME TO NEW STAFF Robert Zakrzewski English Kacie McCormick Science Bryn Graver Math Pamela Kaneda Foreign Language Becky Jones Foreign Language Susanne McMahon Special

431 views • 30 slides

More recommend