toward active learning in data selection
play

Toward Active Learning in Data Selection: Automatic Discovery of - PowerPoint PPT Presentation

Jan 20, 2023 •283 likes •550 views

Toward Active Learning in Data Selection: Automatic Discovery of Language Features During Elicitation Jonathan Clark Robert Frederking Lori Levin Language Technologies Institute Carnegie Mellon University Pittsburgh, PA Feature Detection

  1. Toward Active Learning in Data Selection: Automatic Discovery of Language Features During Elicitation Jonathan Clark Robert Frederking Lori Levin Language Technologies Institute Carnegie Mellon University Pittsburgh, PA

  2. Feature Detection • Grammatemes * - Language features that express grammatical meanings (such as number, person, tense) • Given a set of grammatemes and a structured corpus, can we determine if these grammatemes are expressed in a particular language? • e.g. Answers “Does this language distinguish singular nouns from plural nouns?” (“And if so, how?”) * Source: Alena Böhmová, Silvie Cinková, Eva Haji č ová. Annotation on the tectogrammatical layer in the Prague Dependency Treebank. 2005.

  3. Feature Detection The dog sleeps ((num sg)…) The dogs sleep ((num dl)…) The dogs sleep ((num pl)...)

  4. 犬が寝る 犬が寝る 犬が寝る Feature Detection Bilingual Person The dog sleeps ((num sg)…) The dogs sleep ((num dl)…) The dogs sleep ((num pl)...)

  5. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Feature Detection Bilingual Person The dog sleeps Marks Plural? NO ((num sg)…) Feature The dogs sleep Detection ((num dl)…) The dogs sleep Marks Dual? NO ((num pl)...)

  6. Data Selection • Given many potential training examples, select the ones that will help the target system most • Many Uses - Seen in Speech Recognition, Speech Synthesis, and Machine Translation • Corpus Navigation: Not all data is relevant for all languages • Helps when money or time is limited • e.g. Small Domains, MT Emergencies, and Minority Languages

  7. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Data Selection Bilingual Person The dog sleeps Marks Plural? NO ((num sg)…) Feature The dogs sleep Detection ((num dl)…) The dogs sleep Marks Dual? NO ((num pl)...)

  8. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Data Selection Bilingual Person The dog sleeps Marks Plural? NO ((num sg)…) Feature The dogs sleep Detection ((num dl)…) The dogs sleep Marks Dual? NO ((num pl)...) Data Selection (Corpus Navigation)

  9. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Data Selection Bilingual Person The dog sleeps Marks Plural? NO ((num sg)…) Feature The dogs sleep Detection ((num dl)…) The dogs sleep Marks Dual? NO ((num pl)...) Data Selection (Corpus Navigation) Implicational Universal: No Plural Marking --> No Dual Marking

  10. 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る 犬が寝る Data Selection Bilingual Person The dog sleeps Marks Plural? NO Marks Plural? NO The dog sleeps ((num sg)…) ((num sg)…) Feature The dogs sleep The dogs sleep Detection ((num dl)…) ((num dl)…) The dogs sleep Marks Dual? NO The dogs sleep Marks Dual? NO ((num pl)...) ((num pl)...) Data Selection (Corpus Navigation) Implicational Universal: No Plural Marking --> No Dual Marking

  11. Elicitation Corpus Entry context: Maria bakes cookies regularly or habitually. srcsent: Maria bakes cookies .

  12. Elicitation Corpus Entry context: Maria bakes cookies regularly or habitually. srcsent: Maria bakes cookies .

  13. Elicitation Corpus Entry context: Maria bakes cookies regularly or habitually. srcsent: Maria bakes cookies . tgtsent: Maria hornea galletas . aligned: ((1,1),(2,2),(3,3),(4,4))

  14. Elicitation Corpus Entry context: Maria bakes cookies regularly or habitually. srcsent: Maria bakes cookies . tgtsent: Maria hornea galletas . aligned: ((1,1),(2,2),(3,3),(4,4)) fstruct: [f1]( [f2](actor ((gender f)(anim human)(num sg))) [f3](undergoer ((person 3) (num dl))) (tense pres)) cstruct: [n1](S1 [n2](S [n3](NP [n4](NNP Maria)) [n5](VP [n6](VBZ bakes) [n7](NP [n8](NNS cookies))))) phimap: phi(n1)=f1; phi(n3)=f2; phi(n7)=f3; headmap: h(n1)=n2; h(n2)=n5; h(n3)=n4; h(n4)=n4; h(n5)=n6; h(n6)=n6; h(n7)=n8; h(n8)=n8;

  15. Example Deduction Rule # Perfective/Imperfective Aspect (rule (sentences (A (aspect perfective)) (B (aspect progressive)))

  16. Example Deduction Rule # Perfective/Imperfective Aspect (rule (sentences (A (aspect perfective)) (B (aspect progressive))) (overlap on)

  17. Example Deduction Rule # Perfective/Imperfective Aspect (rule (sentences (A (aspect perfective)) (B (aspect progressive))) (overlap on) (if 0.6 (different (target-lex (fnode (A))) (target-lex (fnode (B)))) (then (WALS ”Perfective/Imperfective Aspect” ”Grammatical marking”)))

  18. Example Deduction Rule # Perfective/Imperfective Aspect (rule (sentences (A (aspect perfective)) (B (aspect progressive))) (overlap on) (if 0.6 (different (target-lex (fnode (A))) (target-lex (fnode (B)))) (then (WALS ”Perfective/Imperfective Aspect” ”Grammatical marking”))) (if 0.4 (same (target-lex (fnode (A))) (target-lex (fnode (B)))) (then (WALS ”Perfective/Imperfective Aspect” ”No grammatical marking”))))

  19. Feature Detection Experiment • Corpus of 60 Spanish-English 100% sentences 80% • Tried to identify 21 features 60% from the World Atlas of 40% Language Structures 20% Precision Recall F1 Baseline 12 / 21 12 / 21 12 / 21 0% Experimental Experimental 19 / 21 19 / 21 19 / 21 Baseline

  20. Toward Corpus Navigation • Not all data is relevant for every language • Performed while a linguistically naive bilingual person translates sentences in GUI • After eliciting each sentence: * Apply feature detection * Choose the most valuable sentence to elicit next • Leverages knowledge from Greenbergian Implicational Universals (from Hal Daume’s database learned from WALS)

  21. Other Applications • Learning feature-annotated closed-class morphemes • Factored MT • Selection of data for automatic grammar induction for syntactic and hybrid MT systems • Aid for linguistics field work

  22. Language Resources • Result of Corpus Navigation is: 1. A resource dense with the “right” features 2. Highly structured; each language feature is linked with sentences that illustrate it 3. Word-aligned, feature-annotated sentences useful for studying divergences and MT

  23. Toward Active Learning in Data Selection: Automatic Discovery of Language Features During Elicitation Questions? Jonathan Clark Robert Frederking Lori Levin Language Technologies Institute Carnegie Mellon University Pittsburgh, PA

  24. WALS Features for Experiment Gender Distinctions in Independent Position of Interrogative Phrases in Content Personal Pronouns Questions Nominal and Locational Predication Position of Pronominal Possessive Affixes Occurrence of Nominal Plurality Position of Tense-Aspect Affixes Inclusive/Exclusive Distinction in Independent Order of Adjective and Noun Pronouns Inclusive/Exclusive Distinction in Verbal Order of Genitive and Noun Inflection Order of Numeral and Noun Semantic Distinctions of Evidentiality Order of Subject, Object and Verb The Future Tense Order of Subject and Verb Verbal Person Marking Order of Object and Verb ‘Want’ Complement Subjects Perfective/Imperfective Aspect Zero Copula for Predicate Nominals Politeness Distinctions in Pronouns

  25. Production Predicates fnode in-order source-lex target-lex *-uhead *-ihead same present not-present

  26. Elicitation Corpus Availability • Included in LDC’s Less Commonly Taught Languages (LCTL) Language Packs • 13 languages have already been translated by the LDC • Urdu language pack used in this year’s NIST MT Eval • Bengali queued for general release this year

Recommend

Active Learning with Active Learning with Model Selection Neil Rubens Sugiyama Lab / Tokyo

Active Learning with Active Learning with Model Selection Neil Rubens Sugiyama Lab / Tokyo

Active Learning with Active Learning with Model Selection Neil Rubens Sugiyama Lab / Tokyo Institute of Technology Active Learning (NLP Motivation) NLP (common scenario) Large amounts of unlabeled data Large amounts of unlabeled

706 views • 36 slides
Information-Based Objective Functions for Active Data Selection By David J. C. Mackay Presented

Information-Based Objective Functions for Active Data Selection By David J. C. Mackay Presented

Information-Based Objective Functions for Active Data Selection By David J. C. Mackay Presented by Aditya Sanghi and Grant Watson 1 Motivation Active learning A learning algorithm which is able to interactively query for more data

1.41k views • 107 slides
Release from Active Learning / Release from Active Learning / Model Selection Dilemma: Model

Release from Active Learning / Release from Active Learning / Model Selection Dilemma: Model

IJCNN2002 May 12-17, 2002 Release from Active Learning / Release from Active Learning / Model Selection Dilemma: Model Selection Dilemma: Optimizing Sample Points and Optimizing Sample Points and Models at the Same Time Models at the Same

532 views • 18 slides
10.1 Active Learning Review What questions should be asked in order to collect best data?

10.1 Active Learning Review What questions should be asked in order to collect best data?

CS/CNS/EE 253: Advanced Topics in Machine Learning Topic: Active Learning 2: the myopic version-space shrinking strategy Lecturer: Daniel Golovin Scribe: Pete Trautman Date: 8 February 2010 10.1 Active Learning Review What questions should be

264 views • 4 slides
Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active

Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active

Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active Learning Veterans Wrap Up Introduction to Active Learning What is active learning? Brainstorm your ideas with your group and generate a

855 views • 42 slides
Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning

Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning

Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning Linguistic Annotations (ACL 08) Image Classification (CVPR 08) Current Work and Discussions Constraint-Driven Active Learning

945 views • 47 slides
ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1.

ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1.

ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1. CONSTRUCTING our TEACHING CHALLENGES in a low tech 1. ORCHESTRATION environment 1. SHOW-AND-TELL Selma Hamdani Psychology / DALC / Neuroscience

836 views • 50 slides
Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection

Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection

Learning From Data Lecture 13 Validation and Model Selection The Validation Set Model Selection Cross Validation M. Magdon-Ismail CSCI 4100/6100 recap: Regularization Regularization combats the effects of noise by putting a leash on the

349 views • 31 slides
Active Learning Passive Learning Active Learning 1. Think 1. Acquisition of knowledge Ability

Active Learning Passive Learning Active Learning 1. Think 1. Acquisition of knowledge Ability

Active Learning Passive Learning Active Learning 1. Think 1. Acquisition of knowledge Ability 2. Decision making 2. Application of knowledge 3. Explain/communicate 1. Deep learning Approach to Uni-directional 2. Interactive Learning

282 views • 5 slides
Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue

Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue

Learning From Data Lecture 3 Is Learning Feasible? Outside the Data Probability to the Rescue Learning vs. Verification Selection Bias - A Cartoon M. Magdon-Ismail CSCI 4100/6100 recap: The Perceptron Learning Algorithm y = +1 y x

538 views • 27 slides
12.1 Active Learning: A Review When learning, it may be the case that getting the true labels of

12.1 Active Learning: A Review When learning, it may be the case that getting the true labels of

CS/CNS/EE 253: Advanced Topics in Machine Learning Topic: Active Learning Review and Kernel Methods Lecturer: Andreas Krause Scribe: Jonathan Krause Date: Feb. 17, 2010 12.1 Active Learning: A Review When learning, it may be the case that getting

213 views • 5 slides
Predator and Prey: Active Learning Is Social Learning Kenneth Ronkowitz Active Learning

Predator and Prey: Active Learning Is Social Learning Kenneth Ronkowitz Active Learning

Predator and Prey: Active Learning Is Social Learning Kenneth Ronkowitz Active Learning Symposium, Rutgers University, May 15, 2018 Typically part of the curriculum in biology, phrenology, biology, anatomy, craniometry, physiognomy, crime

484 views • 14 slides
Learning Loss for Active Learning Rymarczyk D., Zieliski B., Tabor J., Sadowski M., Titov M.

Learning Loss for Active Learning Rymarczyk D., Zieliski B., Tabor J., Sadowski M., Titov M.

Learning Loss for Active Learning Rymarczyk D., Zieliski B., Tabor J., Sadowski M., Titov M. Agenda 1. Active Learning introduction 2. Base methods in AL for Deep Learning 3. Learning Loss for Active Learning 4. Our ideas for Active

802 views • 43 slides
Introduction to Machine Learning Active Learning Barnabs Pczos 1 Credits Some of the

Introduction to Machine Learning Active Learning Barnabs Pczos 1 Credits Some of the

Introduction to Machine Learning Active Learning Barnabs Pczos 1 Credits Some of the slides are taken from Nina Balcan. 2 Classic Supervised Learning Paradigm is Insufficient Nowadays Modern applications: massive amounts of raw data.

1.17k views • 73 slides
Feature Selection Gavin Brown www.cs.man.ac.uk/~gbrown The Usual Supervised Learning Approach

Feature Selection Gavin Brown www.cs.man.ac.uk/~gbrown The Usual Supervised Learning Approach

COMP61011 : Machine Learning Feature Selection Gavin Brown www.cs.man.ac.uk/~gbrown The Usual Supervised Learning Approach data + labels Learning Algorithm Model Predicted label Testing data Predicting Recurrence of Lung Cancer ofi Only a

724 views • 17 slides
Active Learning: Rethinking Our Teaching to Promote Deeper Learning Facilitated by Ken Silvestri,

Active Learning: Rethinking Our Teaching to Promote Deeper Learning Facilitated by Ken Silvestri,

Active Learning: Rethinking Our Teaching to Promote Deeper Learning Facilitated by Ken Silvestri, CFE Instructional Designer Adapted for Institute of Technology By Menaka Abraham OVERVIEW Why Active Learning What is Active Learning

542 views • 18 slides
Learning and Data Selection in Big Datasets H. S. Ghadikolaei , H. Ghauch, C. Fischione, and M.

Learning and Data Selection in Big Datasets H. S. Ghadikolaei , H. Ghauch, C. Fischione, and M.

Learning and Data Selection in Big Datasets H. S. Ghadikolaei , H. Ghauch, C. Fischione, and M. Skoglund School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Stockholm, Sweden http://www.kth.se/profile/hshokri

342 views • 8 slides
Active Learning for Regression: Active Learning for Regression: Algorithms and Applications

Active Learning for Regression: Active Learning for Regression: Algorithms and Applications

LAMDA group, Nanjing University Nov. 5, 2009. Active Learning for Regression: Active Learning for Regression: Algorithms and Applications Algorithms and Applications Masashi Sugiyama Tokyo Institute of Technology sugi@cs.titech.ac.jp

870 views • 64 slides
Data Warehousing and Machine Learning Feature Selection Thomas D. Nielsen Aalborg University

Data Warehousing and Machine Learning Feature Selection Thomas D. Nielsen Aalborg University

Data Warehousing and Machine Learning Feature Selection Thomas D. Nielsen Aalborg University Department of Computer Science Spring 2008 DWML Spring 2008 1 / 16 Feature Selection When features dont help Data generated by process

578 views • 31 slides
Efficient Nonmyopic Active Search Jiang, Malkomes, Converse, Shofner, Moseley, Garnett. ICML 2017

Efficient Nonmyopic Active Search Jiang, Malkomes, Converse, Shofner, Moseley, Garnett. ICML 2017

Efficient Nonmyopic Active Search Jiang, Malkomes, Converse, Shofner, Moseley, Garnett. ICML 2017 Active Learning unlabeled data expert / oracle active learner 2 Supervised Learning unlabeled data random sample active learner

677 views • 49 slides
Active Learning Using Discrepancy Zhenghang Cui, Issei Sato The University of Tokyo / RIKEN AIP

Active Learning Using Discrepancy Zhenghang Cui, Issei Sato The University of Tokyo / RIKEN AIP

Active Learning Using Discrepancy Zhenghang Cui, Issei Sato The University of Tokyo / RIKEN AIP Center RealML @ ICML2020 July 18, 2020 2 (Pool-based) Active Learning Given: a pool of unlabeled data a labeling oracle Decide

384 views • 6 slides
Introduction to Machine Learning Model Validation and Selection Dr. Ilija Bogunovic Learning

Introduction to Machine Learning Model Validation and Selection Dr. Ilija Bogunovic Learning

Introduction to Machine Learning Model Validation and Selection Dr. Ilija Bogunovic Learning and Adaptive Systems (las.ethz.ch) Recap: Achieving generalization Fundamental assumption: Our data set is generated independently and identically

515 views • 28 slides
Capsule Networks and Active Learning Chris Aasted, PhD Lockheed Martin Autonomous Systems

Capsule Networks and Active Learning Chris Aasted, PhD Lockheed Martin Autonomous Systems

Higher Performance with Less Data via Capsule Networks and Active Learning Chris Aasted, PhD Lockheed Martin Autonomous Systems Outline Problem Statement Capsule Networks Transfer Learning Active Learning Datasets

571 views • 30 slides
Using active learning pedagogy to enable the Presentation by Donna Whitehead teaching of

Using active learning pedagogy to enable the Presentation by Donna Whitehead teaching of

Using active learning pedagogy to enable the Presentation by Donna Whitehead teaching of enterprise and entrepreneurial skills Active learning Active learning methodology has become a preferred way to change the traditional teacher

146 views • 11 slides

More recommend