Data Augmentation for Context-Sensitive Neural Lemmatization Using - PowerPoint PPT Presentation

Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text Toms Bergmanis, Sharon Goldwater

Lemmatization Sing Plural NOM ceļš ceļi GEN ceļa ceļu DAT ceļam ceļiem ceļš ACC ceļu ceļus INST ar ceļu ar ceļiem LOC ceļā ceļos VOC ceļ ceļi Latvian: ceļš (English: road )

Previous work: “sentence context helps to lemmatize ambiguous and unseen words” Bergmanis and Goldwater, 2018

Ambiguous words: ceļu Lemma could be: A. ceļš ( road ): NOUN, sing., ACC B. celis ( knee ): NOUN, plur., DAT C. celt ( to lift ):VERB, 1st p., sing., pres. Latvian examples `

Learning from sentences 1. Lemma annotated sentences are scarce for low resource languages 2. annotating sentences is slow 3. N types > N (contiguous) tokens

Learning from sentences 1. Lemma annotated sentences are scarce for low resource languages 2. annotating sentences is slow 3. N types > N (contiguous) tokens Chakrabarty et al., 2017

Learning from sentences 1. Lemma annotated sentences are scarce for low resource languages 2. annotating sentences is slow 3. N types > N (contiguous) tokens Garrette et al., 2013

N types > N tokens Training on 1k UDT tokens/types

Types in context algorithms get smarter , computers faster smart Bergmanis and Goldwater, 2018

Proposal: Data Augmentation ceļš Combine... UniMorph + Inflection tables ...to get types in context

Method: Data Augmentation Dzīves pēdējā ceļā pavadot mūsu ceļš UniMorph Inflection tables: ... ceļš ceļš N;NOM;SG Inflection ceļš ceļā N;LOC;SG ...

Method: Data Augmentation Dzīves pēdējā ceļā pavadot mūsu UniMorph Inflection tables: ... ceļš ceļš N;NOM;SG Context ceļš ceļā N;LOC;SG ...

Method: Data Augmentation Dzīves pēdējā ceļā pavadot mūsu ceļš UniMorph Inflection tables: ... ceļš ceļš N;NOM;SG Lemma ceļš ceļā N;LOC;SG ...

Inflection Tables: Sing Plural NOM ceļš ceļi GEN ceļa ceļu DAT ceļam ceļiem ACC ceļu ceļus INST ar ceļu ar ceļiem LOC ceļā ceļos VOC ceļ ceļi Latvian: ceļš (English: road )

Inflection Tables: Sing Plural NOM ceļš ceļi GEN ceļa ceļu DAT ceļam ceļiem ACC ceļu ceļus INST ar ceļu ar ceļiem LOC ceļā ceļos VOC ceļ ceļi celt (build) ceļot (travel) celis (knee)

Key question: If ambiguous words “enforce” the use of context: Is context still useful in the absence of ambiguous forms?

Experiments Train : 1k types from universal dependency corpus Augment : 1k, 5k, 10k types of UniMorph in Wikipedia contexts Languages: Bulgarian, Czech, Estonian, Finnish, Latvian, Polish, Romanian, Russian, Swedish, Turkish

Experiments Metric : type level macro average accuracy Test: on standard splits of universal dependency corpus

Results: Data augmentation using context

Does model learn from context? context vs no context

Afix ambiguity: wuger Lemma depends on context: A. if wuger is adjective then lemma could be wug B. if wuger is noun then lemma could be wuger English examples `

Takeaways/conclusions: Despite biased data and divergent lemmatization standards Type based data augmentation helps (+14% accuracy)

Takeaways/conclusions: Even without the ambiguous types that “enforce” the use of context Model use context to disambiguate affixes of unseen words (+5% accuracy)

Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text toms.bergmanis@gmail.com https://bitbucket.org/tomsbergmanis/data_augumentation_um_wiki

Data Augmentation for Context-Sensitive Neural Lemmatization Using - PowerPoint PPT Presentation

Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text Toms Bergmanis, Sharon Goldwater Lemmatization Sing Plural NOM ce cei GEN cea ceu DAT ceam ceiem ce ACC ceu

SwitchOut: An Efficient Data Augmentation for Neural Machine Translation Xinyi Wang , Hieu

Population Based Augmentation Efficient Learning of Augmentation Policy Schedules Daniel Ho , Eric

Data Augmentation in NLP 2020-03-21 Xiachong Feng Outline Why we need Data Augmentation?

Convolutional Neural Networks with Data Augmentation against Jitter-Based Countermeasures Eleonora

Context Sensitive Solutions Context Context Sensitive Solutions (CSS) is a collaborative approach

A6: Sensitive Data Exposure A6 Sensitive Data Exposure Sensitive data stored or transmitted

for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

Context-sensitive Analysis Attribute Grammar And Type Checking cs5363 1 Context-Sensitive

Towards More Security in Data Exchange Defining Unparsers with Context-Sensitive Encoders for

Oak Hill Parkway Oak Hill Parkway Context Sensitive Solutions CSS Workshop No. 1 October 9,

Context Sensitivity Example of a CSG Informatics 2A: Lecture 26 2 Context in Programming

and Augmentation for Deep Neural Network Training Trevor Gale Steven Eliuk Cameron Upright

Protecting Sensitive Data Implementation of a Sensitive Data Manager Recommendation Briefed

Context-sensitive languages Informatics 2A: Lecture 28 John Longley School of Informatics

Augmentation Introduction ImageNet Classification with Deep Convolutional Neural Networks,

Data Races are Bad Race: two threads access memory without synchronization and at least one is a

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Does Data Augmentation Lead to Positive Margin? Dimitris Po-Ling Loh Shashank Rajput* Zhili

Project Background Designing Walkable Urban Thoroughfares: A Context Sensitive Approach

Context-sensitive languages Informatics 2A: Lecture 28 John Longley School of Informatics

Context-sensitive languages Informatics 2A: Lecture 28 Alex Simpson School of Informatics

Context Sensitive Grammar Habeeb and Alvin ATC Seminar 30 NOV 2018 Habeeb and Alvin CSG 30

In Deep Learning Anima Anandkumar & Zachary Lipton DATA AUGMENTATION To improve

A Kernel Theory of Modern Data Augmentation Tr Tri Dao ao , Albert Gu, Alex Ratner, Virginia

Data Augmentation for Context-Sensitive Neural Lemmatization Using - PowerPoint PPT Presentation

Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text Toms Bergmanis, Sharon Goldwater Lemmatization Sing Plural NOM ce cei GEN cea ceu DAT ceam ceiem ce ACC ceu

SwitchOut: An Efficient Data Augmentation for Neural Machine Translation Xinyi Wang , Hieu

Population Based Augmentation Efficient Learning of Augmentation Policy Schedules Daniel Ho , Eric

Data Augmentation in NLP 2020-03-21 Xiachong Feng Outline Why we need Data Augmentation?

Convolutional Neural Networks with Data Augmentation against Jitter-Based Countermeasures Eleonora

Context Sensitive Solutions Context Context Sensitive Solutions (CSS) is a collaborative approach

A6: Sensitive Data Exposure A6 Sensitive Data Exposure Sensitive data stored or transmitted

for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

Context-sensitive Analysis Attribute Grammar And Type Checking cs5363 1 Context-Sensitive

Towards More Security in Data Exchange Defining Unparsers with Context-Sensitive Encoders for

Oak Hill Parkway Oak Hill Parkway Context Sensitive Solutions CSS Workshop No. 1 October 9,

Context Sensitivity Example of a CSG Informatics 2A: Lecture 26 2 Context in Programming

and Augmentation for Deep Neural Network Training Trevor Gale Steven Eliuk Cameron Upright

Protecting Sensitive Data Implementation of a Sensitive Data Manager Recommendation Briefed

Context-sensitive languages Informatics 2A: Lecture 28 John Longley School of Informatics

Augmentation Introduction ImageNet Classification with Deep Convolutional Neural Networks,

Data Races are Bad Race: two threads access memory without synchronization and at least one is a

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Does Data Augmentation Lead to Positive Margin? Dimitris Po-Ling Loh Shashank Rajput* Zhili

Project Background Designing Walkable Urban Thoroughfares: A Context Sensitive Approach

Context-sensitive languages Informatics 2A: Lecture 28 John Longley School of Informatics

Context-sensitive languages Informatics 2A: Lecture 28 Alex Simpson School of Informatics

Context Sensitive Grammar Habeeb and Alvin ATC Seminar 30 NOV 2018 Habeeb and Alvin CSG 30

In Deep Learning Anima Anandkumar &amp; Zachary Lipton DATA AUGMENTATION To improve

A Kernel Theory of Modern Data Augmentation Tr Tri Dao ao , Albert Gu, Alex Ratner, Virginia

In Deep Learning Anima Anandkumar & Zachary Lipton DATA AUGMENTATION To improve