translation model interpolation for domain adaptation in
play

Translation Model Interpolation for Domain Adaptation in TectoMT - PowerPoint PPT Presentation

Apr 23, 2023 •269 likes •561 views

Rudolf Rosa , Ondej Duek, Michal Novk, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz Translation Model Interpolation for Domain Adaptation in TectoMT Charles University in Prague Faculty of Mathematics and Physics Institute

  1. Rudolf Rosa , Ondřej Dušek, Michal Novák, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz Translation Model Interpolation for Domain Adaptation in TectoMT Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics DMTW, Praha, 3 September 2015

  2. Why domain adaptation?  not to do MT as “Europarl science”  reasonably good for e.g. economic news  not particularly good for IT helpdesk Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 2/29

  3. Why domain adaptation?  not to do MT as “Europarl science”  reasonably good for e.g. economic news  not particularly good for IT helpdesk Jak pošlu přílohu mailem? How do I send an attachment by e-mail? Click on the paperclip icon and select the file. Klikněte na ikonu sponky a vyberte soubor. Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 3/29

  4. Why model interpolation?  available parallel data  out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 4/29

  5. Why model interpolation?  available parallel data  out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences  train on out + in concatenation  effect of in negligible (overpowered by out ) Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 5/29

  6. Why model interpolation?  available parallel data  out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences  train on out + in concatenation  effect of in negligible (overpowered by out )  train on in only  too weak, low coverage, worse than out alone Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 6/29

  7. Why model interpolation?  available parallel data  out-of-domain (Europarl): 2 000 000 sentences  in-domain (IT helpdesk): 1 000 sentences  train on out + in concatenation  effect of in negligible (overpowered by out )  train on in only  too weak, low coverage, worse than out alone  train 2 translation models, one on in , one on out  final TM = (weighted) interpolation of in TM and out TM Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 7/29

  8. Why TectoMT (deep transfer MT)? ✔ better meaning preservation ✔ better generalization from small data ✗ limited support for TM interpolation ✗ no support for tuning of TM weights Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 8/29

  9. Why TectoMT (deep transfer MT)? ✔ better meaning preservation ✔ better generalization from small data ✗ limited support for TM interpolation – solved ✗ no support for tuning of TM weights – TODO Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 9/29

  10. Outline  Motivation  TectoMT (deep transfer)  Translation model interpolation in TectoMT  Evaluation  Conclusion Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 10/29

  11. TectoMT deep transfer Jak pošlu přílohu mailem? Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 11/29

  12. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 12/29

  13. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 13/29

  14. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 14/29

  15. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How adv Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 15/29

  16. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How #PersPron adv n:subj Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 16/29

  17. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How #PersPron attachment adv n:subj n:obj Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 17/29

  18. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How #PersPron attachment e-mail adv n:subj n:obj n:by+X Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 18/29

  19. TectoMT deep transfer Jak pošlu přílohu mailem? poslat v:fin Jak #PersPron příloha mail adv drop n:4 n:7 send v:fin How #PersPron attachment e-mail adv n:subj n:obj n:by+X How do I send an attachment by e-mail? Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 19/29

  20. T-lemma and formeme transfer poslat v:fin mail ? n:7 Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 20/29

  21. T-lemma and formeme transfer e-mail (n) 0.32 online (adv) 0.25 poslat address (n) 0.11 v:fin online (adj) 0.05 ... mail n:7 n:by+X 0.23 n:with+X 0.22 adv 0.13 n:via+X 0.04 ... Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 21/29

  22. T-lemma and formeme transfer e-mail (n) 0.32 online (adv) 0.25 poslat address (n) 0.11 v:fin online (adj) 0.05 ... mail e-mail n:7 n:by+X n:by+X 0.23 n:with+X 0.22 adv 0.13 n:via+X 0.04 ... Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 22/29

  23. Translation model interpolation P ( e-mail ∣ mail )= 2 3 MaxEnt mail ( e-mail ,context )+ 1 3 Static ( e-mail , mail ) Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 23/29

  24. Translation model interpolation  past: P ( e-mail ∣ mail )= 2 3 MaxEnt mail ( e-mail ,context )+ 1 3 Static ( e-mail , mail )  present: P ( e-mail ∣ mail )= 2 6 OutMaxEnt mail ( e-mail , context )+ 1 6 OutStatic ( e-mail , mail ) + 2 6 InMaxEnt mail ( e-mail , context ) + 1 6 InStatic ( e-mail , mail ) Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 24/29

  25. Translation model interpolation  future: P ( e-mail ∣ mail )= w 1 ⋅ OutMaxEnt mail ( e-mail ,context )+ w 2 ⋅ OutStatic ( e-mail , mail ) + w 3 ⋅ InMaxEnt mail ( e-mail ,context ) + w 4 ⋅ InStatic ( e-mail , mail )  interpolation weights tuned on development set Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 25/29

  26. Translation model interpolation  future: P ( e-mail ∣ mail )= w 1 ⋅ OutMaxEnt mail ( e-mail ,context )+ w 2 ⋅ OutStatic ( e-mail , mail ) + w 3 ⋅ InMaxEnt mail ( e-mail ,context ) + w 4 ⋅ InStatic ( e-mail , mail )  interpolation weights tuned on development set  not available at the moment:  tuning in TectoMT  development set Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 26/29

  27. Evaluation 40 38.9 out 38 37.0 in 36 interpol 33.7 34 32 31.3 30.6 30 28.4 28.3 28 27.1 26.5 26 24.3 24 23.3 23.0 21.5 22 21.4 20.4 20.4 20 18.5 18.5 18 16 EN→CS CS→EN EN→ES ES→EN EN→NL NL→EN Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 27/29

  28. Conclusion  TectoMT deep transfer system  IT helpdesk domain (1000 parallel sentences)  domain adaptation by TM interpolation  no tuning of interpolation weights  consistently positive results, up to +3 BLEU  future work  tuning of interpolation weights  other domain adaptation methods Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 28/29

  29. Thank you for your attention Rudolf Rosa , Ondřej Dušek, Michal Novák, Martin Popel {rosa,odusek,mnovak,popel}@ufal.mff.cuni.cz Translation Model Interpolation for Domain Adaptation in TectoMT Charles University in Prague Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics http://ufal.mff.cuni.cz/rudolf-rosa/ Rosa, Dušek, Novák, Popel: Translation Model Interpolation for Domain Adaptation in TectoMT 29/29

Recommend

Translation Model Adaptation Using Genre-Revealing Text Features Marlies van der Wees, Arianna

Translation Model Adaptation Using Genre-Revealing Text Features Marlies van der Wees, Arianna

Translation Model Adaptation Using Genre-Revealing Text Features Marlies van der Wees, Arianna Bisazza, Christof Monz Domain adaptation for SMT Prioritize translation candidates that are most relevant to a specific task Translation Model

758 views • 74 slides
Adaptation Philipp Koehn 27 October 2020 Philipp Koehn Machine Translation: Adaptation 27

Adaptation Philipp Koehn 27 October 2020 Philipp Koehn Machine Translation: Adaptation 27

Adaptation Philipp Koehn 27 October 2020 Philipp Koehn Machine Translation: Adaptation 27 October 2020 Adaptation 1 Better quality when system is adapted to a task Domain adaptation to a specific domain, e.g., information technology

962 views • 46 slides
Domain Adaptation in Statistical Machine Translation Logic, Language and Computation Bart

Domain Adaptation in Statistical Machine Translation Logic, Language and Computation Bart

Domain Adaptation in Statistical Machine Translation Logic, Language and Computation Bart Mellebeek October 28, 2013 Bart Mellebeek Domain Adaptation in Statistical Machine Translation Outline Machine Translation and AI Statistical Machine

889 views • 54 slides
What is domain adaptation?

What is domain adaptation?

A Two-Stage Approach to Domain Adaptation for Statistical Classifiers Jing Jiang & ChengXiang Zhai Department of Computer Science University of Illinois at Urbana-Champaign What is domain adaptation?

523 views • 20 slides
Domain adaptation model for retinopathy detection from cross-domain OCT images Jing Wang 1;2 ,

Domain adaptation model for retinopathy detection from cross-domain OCT images Jing Wang 1;2 ,

Domain adaptation model for retinopathy detection from cross-domain OCT images Jing Wang 1;2 , Yiwei Chen 2 , Wanyue Li1; 2 , Wen Kong 1;2 , Yi He 2 , Chuihui Jiang *3 , Guohua Shi *2;4 1 University of Science and technology of China, Hefei 230026,

348 views • 9 slides
discrepancy for unsupervised domain adaptation Hongliang Yan 2017/06/21 Domain Adaptation DA

discrepancy for unsupervised domain adaptation Hongliang Yan 2017/06/21 Domain Adaptation DA

Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation Hongliang Yan 2017/06/21 Domain Adaptation DA Training Test Problem: (Source (Target) Training and test sets are related but under

495 views • 21 slides
Domain Adaptation with Asymmetrically Relaxed Distribution Alignment Yifan Wu , Ezra Winston,

Domain Adaptation with Asymmetrically Relaxed Distribution Alignment Yifan Wu , Ezra Winston,

Domain Adaptation with Asymmetrically Relaxed Distribution Alignment Yifan Wu , Ezra Winston, Divyansh Kaushik, Zachary Lipton Carnegie Mellon University ICML 2019 1 / 8 Background - Unsupervised Domain Adaptation Unsupervised Domain

332 views • 8 slides
Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation Kaichao You 1 , Ximei Wang

Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation Kaichao You 1 , Ximei Wang

Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation Kaichao You 1 , Ximei Wang 1 , Mingsheng Long 1 , Michael I. Jordan 2 1 School of Software, Tsinghua University 1 National Engineering Lab for Big Data Software 2 University of

324 views • 18 slides
Investigations on Translation Model Adaptation Using Monolingual Data Patrik Lambert, Holger

Investigations on Translation Model Adaptation Using Monolingual Data Patrik Lambert, Holger

Investigations on Translation Model Adaptation Using Monolingual Data Patrik Lambert, Holger Schwenk Christophe Servan and Sadaf Abdul-Rauf LIUM (Computing Laboratory) University of Le Mans France WMT 2011 Lambert, Schwenk, Servan, Abdul-Rauf

485 views • 30 slides
Domain Adaptation from a Pre-trained Source Model Application on fraud detection tasks Presenter:

Domain Adaptation from a Pre-trained Source Model Application on fraud detection tasks Presenter:

Domain Adaptation from a Pre-trained Source Model Application on fraud detection tasks Presenter: Luxin Zhang (Worldline & Inria) Supervisors: Christophe Biernacki (Inria), Pascal Germain (Inria), Yacine Kessaci (Worldline) CMStatistics 2019

531 views • 17 slides
Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation 1,2 1,4 Xiang

Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation 1,2 1,4 Xiang

Implicit Class-Conditioned Domain Alignment for Unsupervised Domain Adaptation 1,2 1,4 Xiang Jiang Qicheng Lao 1,3 1 Stan Matwin Mohammad Havaei 1 Imagia 2 Dalhousie University 3 Polish Academy of Sciences 4 Mila, Universit e de Montr

1.02k views • 43 slides
Towards Assumption-free Unsupervised Domain Adaptation for Visual recognition

Towards Assumption-free Unsupervised Domain Adaptation for Visual recognition

Towards Assumption-free Unsupervised Domain Adaptation for Visual recognition Outline Background and Motivations Unsupervised Domain Adaptation - Estimating label information in

1.51k views • 47 slides
Interpolation CS3220 - Summer 2008 Jonathan Kaldor Interpolation Weve looked at the

Interpolation CS3220 - Summer 2008 Jonathan Kaldor Interpolation Weve looked at the

Interpolation CS3220 - Summer 2008 Jonathan Kaldor Interpolation Weve looked at the problem of model fitting using least squares: given some sample points, determine linear parameters for a model that best fits the data Note:

813 views • 63 slides
Instance Weighting for Domain Adaptation in NLP Jing Jiang & ChengXiang Zhai University of

Instance Weighting for Domain Adaptation in NLP Jing Jiang & ChengXiang Zhai University of

Instance Weighting for Domain Adaptation in NLP Jing Jiang & ChengXiang Zhai University of Illinois at Urbana-Champaign June 25, 2007 Domain Adaptation Many NLP tasks are cast into classification

582 views • 20 slides
Few-shot Domain Adaptation 1/12 by Causal Mechanism Transfer Domain adaptation Causal mechanism

Few-shot Domain Adaptation 1/12 by Causal Mechanism Transfer Domain adaptation Causal mechanism

Few-shot Domain Adaptation 1/12 by Causal Mechanism Transfer Domain adaptation Causal mechanism Q. When is it possible? A. Common causal Transfer assumption (TA)? mechanism as the relation. Summary Common data generating (causal) mechanism

1.03k views • 15 slides
Theoretical Analysis of Domain Adaptation Current state of the art Shai Ben-David September 14,

Theoretical Analysis of Domain Adaptation Current state of the art Shai Ben-David September 14,

Theoretical Analysis of Domain Adaptation Current state of the art Shai Ben-David September 14, 2012 Domain Adaptation Most of the statistical learning guarantees are based on assuming that the learning environment is unchanged throughout the

685 views • 64 slides
Hybrid Adaptation of Named Entity Recognition for Statistical Machine Translation Vassilina

Hybrid Adaptation of Named Entity Recognition for Statistical Machine Translation Vassilina

Hybrid Adaptation of Named Entity Recognition for Statistical Machine Translation Vassilina Nikoulina, gnes Sndor, Marc Dymetman Outline Introduction Approach NER integration within SMT NER adaptation for SMT NER prediction for

403 views • 15 slides
Hybrid Adaptation of Named Entity Recognition for Statistical Machine Translation Vassilina

Hybrid Adaptation of Named Entity Recognition for Statistical Machine Translation Vassilina

Hybrid Adaptation of Named Entity Recognition for Statistical Machine Translation Vassilina Nikoulina, gnes Sndor, Marc Dymetman Outline Introduction Approach NER integration within SMT NER adaptation for SMT NER prediction for

567 views • 15 slides
Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction Rahaf Aljundi,

Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction Rahaf Aljundi,

Lightweight Unsupervised Domain Adaptation by Convolutional Filter Reconstruction Rahaf Aljundi, Tinne Tuytelaars Unsupervised Domain Adaptation When you expect the test data (Target) to be different from your training data (Source) DA in the

683 views • 39 slides
Measuring Immediate Adaptation Performance for Neural Machine Translation Patrick Simianer , Joern

Measuring Immediate Adaptation Performance for Neural Machine Translation Patrick Simianer , Joern

Measuring Immediate Adaptation Performance for Neural Machine Translation Patrick Simianer , Joern Wuebker, John DeNero Lilt NAACL 19 Outline Motivation & Approach 1 2 Evaluation Conclusion 3 2 / 20 Motivation Online adaptation is

765 views • 52 slides
Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition Systems Stephen

Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition Systems Stephen

Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition Systems Stephen H. Shum Douglas A. Reynolds Daniel Garcia-Romero Alan McCree Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition

437 views • 25 slides
Active Online Domain Adaptation Yining Chen (Stanford) , Haipeng Luo (USC), Tengyu Ma (Stanford),

Active Online Domain Adaptation Yining Chen (Stanford) , Haipeng Luo (USC), Tengyu Ma (Stanford),

Active Online Domain Adaptation Yining Chen (Stanford) , Haipeng Luo (USC), Tengyu Ma (Stanford), Chicheng Zhang (University of Arizona) Domain shift: challenge for ML system Time Online learning: a classical framework for domain shift At

390 views • 7 slides
Part II: Interpolation and Approximation theory Contents: Review of Lagrange interpolation

Part II: Interpolation and Approximation theory Contents: Review of Lagrange interpolation

Part II: Interpolation and Approximation theory Contents: Review of Lagrange interpolation polynomials 1 Newton interpolation 2 Optimal interpolation points; Chebyshev polynomials 3 Cubic spline interpolation 4 Error analysis 5 Numerical

1.14k views • 47 slides
ISCTM Recommendations on the Translation and Adaptation of Clinician Reported Outcomes in

ISCTM Recommendations on the Translation and Adaptation of Clinician Reported Outcomes in

ISCTM Recommendations on the Translation and Adaptation of Clinician Reported Outcomes in International Clinical Trials Elizabeth Pappadopulos PHD, Global CNS Lead - Pfizer Objective The ISCTM guidelines are intended to provide practical

592 views • 17 slides

More recommend