from dependency parsing
play

From Dependency Parsing to Imitation Learning CMSC 723 / LING 723 / - PowerPoint PPT Presentation

Jul 13, 2023 •449 likes •687 views

From Dependency Parsing to Imitation Learning CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Yoav Goldberg, Hal Daume III T odays topics: Addressing compounding error Improving on gold parse oracle

  1. From Dependency Parsing to Imitation Learning CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Yoav Goldberg, Hal Daume III

  2. T oday’s topics: Addressing compounding error • Improving on gold parse oracle • Research highlight: [Goldberg & Nivre, 2012] • Imitation learning for structured prediction • CIML ch 18

  3. Improving the oracle in transition-based dependency parsing • Issues with oracle we’ve used so far • Based on configuration sequence that produces gold tree • What if there are multiple sequences for a single gold tree? • How can we recover if the parser deviates from gold sequence? • Goldberg & Nivre [2012] propose an improved oracle

  4. Exercise: which of these transition sequences produces the gold tree on the left?

  5. Dependency Arc from position j to position i, Buffer Stack Arcs with dependency label l

  6. Which of these transition sequences does the oracle algorithm produce?

  7. Improving the oracle in transition-based dependency parsing • Issues with oracle we’ve used so far • Based on configuration sequence that produces gold tree • What if there are multiple sequences for a single gold tree? • How can we recover if the parser deviates from gold sequence? • Goldberg & Nivre [2012] propose an improved oracle

  8. SHIFT At test time, suppose the 4 th transition predicted is SHIFT instead of RA IOBJ What happens if we apply the oracle next?

  9. Measuring distance from gold tree • Labeled attachment loss: number of arcs in gold tree that are not found in the predicted tree Loss = 1 Loss = 3

  10. Improving the oracle in transition-based dependency parsing • Issues with oracle we’ve used so far • Based on configuration sequence that produces gold tree • What if there are multiple sequences for a single gold tree? • How can we recover if the parser deviates from gold sequence? • Goldberg & Nivre [2012] propose an improved oracle

  11. Proposed solution: 2 key changes to training algorithm Any transition that can possibly lead to a correct tree is considered correct Explore non-optimal transitions

  12. Proposed solution: 2 key changes to training algorithm

  13. Defining the cost of a transition • Loss difference between minimum loss trees achievable before and after transition • Loss for trees nicely decomposes into losses for arcs • We can compute transition cost by counting gold arcs that are no longer reachable after transition

  14. T oday’s topics Addressing compounding error • Improving on gold parse oracle • Research highlight: [Goldberg & Nivre, 2012] • Imitation learning for structured prediction • CIML ch 18

  15. Imitation Learning aka learning by demonstration • Sequential decision making problem • At each point in time 𝑢 • Receive input information 𝑦 𝑢 • Take action 𝑏 𝑢 • Suffer loss 𝑚 𝑢 • Move to next time step until time T • Goal • learn a policy function 𝑔(𝑦 𝑢 ) = 𝑧 𝑢 • That minimizes expected total loss over all trajectories enabled by f

  16. Supervised Imitation Learning

  17. Supervised Imitation Learning Problem with supervised approach: Compounding error

  18. How can we train system to make better predictions off the expert path? • We want a policy f that leads to good performance in configurations that f encounters • A chicken and egg problem • Can be addressed by iterative approach

  19. DAGGER: simple & effective imitation learning via Data AGGregation Requires interaction with expert!

  20. When is DAGGER used in practice? • Interaction with expert is not always possible • Classic use case • Expert = slow algorithm • Use DAGGER to learn a faster algorithm that imitates expert • Example: game playing where expert = brute-force search in simulation mode • But also structured prediction

  21. Sequence labeling via imitation learning • What is the “expert” here? • Given a loss function (e.g., Hamming loss) • Expert takes action that minimizes long-term loss Loss of best reachable Output prefix output starting with at time t prefix 𝑧 ∘ 𝑏 • When expert can be computed exactly, it is called an oracle • Key advantages • Can define features • No restriction to Markov features

  22. T oday’s topics • Improving on gold parse oracle • Research highlight: [Goldberg & Nivre, 2012] • Imitation learning for structured prediction • CIML ch 18

Recommend

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr

Statistical Parsing Dependency parsing ar ltekin University of Tbingen Seminar fr Sprachwissenschaft November 2016 Recap/background Dependency grammar Dependency parsing Evaluation Summary Ingredients of a parser

1.44k views • 89 slides
Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP

Dependency Parsing & Feature-based Parsing Ling571 Deep Processing Techniques for NLP February 2, 2015 Roadmap Dependency parsing Graph-based dependency parsing Maximum spanning tree CLE Algorithm Learning

899 views • 87 slides
Dependency Parsing CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan

Dependency Parsing CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan

Dependency Parsing CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan Jurafsky & James Martin Dependency Parsing Formalizing dependency trees Transition-based dependency parsing Shift-reduce parsing

676 views • 24 slides
Dependency Parsing 2 CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre,

Dependency Parsing 2 CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre,

Dependency Parsing 2 CMSC 723 / LING 723 / INST 725 Marine Carpuat Fig credits: Joakim Nivre, Dan Jurafsky & James Martin Dependency Parsing Formalizing dependency trees Transition-based dependency parsing Shift-reduce parsing

1.02k views • 52 slides
Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency

Natural Language Processing Other Syntactic Models Parsing IV Dan Klein UC Berkeley Dependency Parsing Dependency Parsing Pure dependency parsing is only cubic [Eisner 99] Lexicalized parsers can be seen as producing dependency trees

314 views • 5 slides
Last class Dependency parsing and logistic regression Dependency parsing: a fully lexicalized

Last class Dependency parsing and logistic regression Dependency parsing: a fully lexicalized

Last class Dependency parsing and logistic regression Dependency parsing: a fully lexicalized formalism; tree edges connect words in the Shay Cohen sentence based on head-dependent relationships. (based on slides by Sharon Goldwater) a

359 views • 11 slides
Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs

Robust Incremental Neural Semantic Graph Parsing Jan Buys and Phil Blunsom Dependency Parsing vs Semantic Parsing Dependency parsing models the syntactic structure between words in a sentence. Dependency Parsing vs Semantic Parsing

637 views • 31 slides
Dependency Parsing II CMSC 470 Marine Carpuat Graph-based Dependency Parsing Slides credit:

Dependency Parsing II CMSC 470 Marine Carpuat Graph-based Dependency Parsing Slides credit:

Dependency Parsing II CMSC 470 Marine Carpuat Graph-based Dependency Parsing Slides credit: Joakim Nivre Directed Spanning Trees Dependency Parsing as Finding the Maximum Spanning Tree Views parsing as finding the best directed spanning

981 views • 61 slides
Dependency Grammars and Parsing CMSC 473/673 UMBC Outline Review: PCFGs and CKY Dependency

Dependency Grammars and Parsing CMSC 473/673 UMBC Outline Review: PCFGs and CKY Dependency

Dependency Grammars and Parsing CMSC 473/673 UMBC Outline Review: PCFGs and CKY Dependency Grammars Dependency Parsing (Shift-reduce) From syntax to semantics Probabilistic Context Free Grammar 1.0 S NP VP 1.0 PP P NP .4 NP Det

862 views • 75 slides
Algorithms for NLP CS 11-711 Fall 2020 Lecture 14: Graph-based dependency parsing Emma

Algorithms for NLP CS 11-711 Fall 2020 Lecture 14: Graph-based dependency parsing Emma

Algorithms for NLP CS 11-711 Fall 2020 Lecture 14: Graph-based dependency parsing Emma Strubell Announcements No recitation on Friday (Tartan Community Day). 2 Dependency parsing 3 Dependency parsing Input buffer Transition-based

1.15k views • 77 slides
Thoughts on Learner Data and Motivation Learner Language Dependency Parsing and Dependency

Thoughts on Learner Data and Motivation Learner Language Dependency Parsing and Dependency

Thoughts on Learner Data and Dependency Parsing Niels Ott, Ramon Ziai, Julia Krivanek, Detmar Meurers Introduction and Thoughts on Learner Data and Motivation Learner Language Dependency Parsing and Dependency Annotation Approximated

812 views • 53 slides
Marina Valeeva Outline 2 1. Introduction What is Dependency Parsing? What is a

Marina Valeeva Outline 2 1. Introduction What is Dependency Parsing? What is a

GRAPH-BASED DEPENDENCY PARSING Marina Valeeva Outline 2 1. Introduction What is Dependency Parsing? What is a Dependency Tree? Projectivity vs. Non-Projectivity 2. Graph-Based Dependency Parsing 3. Models Edge Based Factorization

572 views • 40 slides
Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on

Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on

Statistical Dependency Parsing in Korean: From Corpus Generation To Automatic Parsing Workshop on Statistical Parsing of Morphologically-Rich Languages 12th International Conference on Parsing Technologies Jinho D. Choi & Martha Palmer

199 views • 19 slides
Graph-based Dependency Parsing (Chu-Liu-Edmonds algorithm) Sam Thomson (with thanks to Swabha

Graph-based Dependency Parsing (Chu-Liu-Edmonds algorithm) Sam Thomson (with thanks to Swabha

Graph-based Dependency Parsing (Chu-Liu-Edmonds algorithm) Sam Thomson (with thanks to Swabha Swayamdipta) University of Washington, CSE 490u February 22, 2017 Outline Dependency trees Three main approaches to parsing

648 views • 54 slides
Lecture 19: Dependency Grammars and Dependency Parsing Julia Hockenmaier juliahmr@illinois.edu

Lecture 19: Dependency Grammars and Dependency Parsing Julia Hockenmaier juliahmr@illinois.edu

CS447: Natural Language Processing http://courses.engr.illinois.edu/cs447 Lecture 19: Dependency Grammars and Dependency Parsing Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center Todays lecture Dependency Grammars Dependency

792 views • 54 slides
Dependency Parsing Joakim Nivre Uppsala University Department of Linguistics and Philology

Dependency Parsing Joakim Nivre Uppsala University Department of Linguistics and Philology

Dependency Parsing Joakim Nivre Uppsala University Department of Linguistics and Philology joakim.nivre@lingfil.uu.se Dependency Parsing 1(14) 1. Dependency Trees 2. Arc-Factored Models 3. Online Learning 4. Eisners Algorithm 5.

726 views • 14 slides
Dependency Parsing CMSC 470 Marine Carpuat Dependency Grammars Syntactic structure = lexical

Dependency Parsing CMSC 470 Marine Carpuat Dependency Grammars Syntactic structure = lexical

Dependency Parsing CMSC 470 Marine Carpuat Dependency Grammars Syntactic structure = lexical items linked by binary asymmetrical relations called dependencies Example Dependency Parse Dependencies (usually) form a tree: - Connected -

593 views • 23 slides
Dependency Parsing and Feature-based Parsing Ling 571 Deep Processing Techniques for

Dependency Parsing and Feature-based Parsing Ling 571 Deep Processing Techniques for

Dependency Parsing and Feature-based Parsing Ling 571 Deep Processing Techniques for NLP October 21, 2019 Shane Steinert-Threlkeld 1 Announcements Thanks for the feedback! HW3: mean 92 Handling ungrammaticality:

1.3k views • 70 slides
Soft Cross lingual Syntax Projection for Dependency Parsing Zhenghua Li, Min Zhang, Wenliang

Soft Cross lingual Syntax Projection for Dependency Parsing Zhenghua Li, Min Zhang, Wenliang

Soft Cross lingual Syntax Projection for Dependency Parsing Zhenghua Li, Min Zhang, Wenliang Chen {zhli13, minzhang, wlchen}@suda.edu.cn Soochow University, China Dependency parsing A bilingual example pmod root obj obj det subj

437 views • 41 slides
Advanced Dependency Parsing Joakim Nivre Uppsala University Linguistics and Philology Based on

Advanced Dependency Parsing Joakim Nivre Uppsala University Linguistics and Philology Based on

Advanced Dependency Parsing Joakim Nivre Uppsala University Linguistics and Philology Based on tutorials with Ryan McDonald Advanced Dependency Parsing 1(36) Introduction Plan for the Lecture 1. Graph-based vs. transition-based dependency

1.61k views • 109 slides
A Fast and Accurate Dependency Parser using Neural Networks Danqi Chen & Christopher D.

A Fast and Accurate Dependency Parser using Neural Networks Danqi Chen & Christopher D.

A Fast and Accurate Dependency Parser using Neural Networks Danqi Chen & Christopher D. Manning Qiming Chen qc2195 Apr. 8. 2015 Dependency Parsing Parsing: He has good control . Dependency Parsing Parsing: He has good control .

743 views • 35 slides
Dependency Parsing CMSC 723 / LING 723 / INST 725 M ARINE C ARPUAT marine@cs.umd.edu Slides

Dependency Parsing CMSC 723 / LING 723 / INST 725 M ARINE C ARPUAT marine@cs.umd.edu Slides

Dependency Parsing CMSC 723 / LING 723 / INST 725 M ARINE C ARPUAT marine@cs.umd.edu Slides credit: Joakim Nivre & Ryan McDonald T odays Agenda Formalizing dependency graphs Formalizing transition-based parsing most material

755 views • 26 slides
NLP Programming Tutorial 12 - Dependency Parsing Graham Neubig Nara Institute of Science and

NLP Programming Tutorial 12 - Dependency Parsing Graham Neubig Nara Institute of Science and

NLP Programming Tutorial 12 Dependency Parsing NLP Programming Tutorial 12 - Dependency Parsing Graham Neubig Nara Institute of Science and Technology (NAIST) 1 NLP Programming Tutorial 12 Dependency Parsing Interpreting Language is

244 views • 23 slides
Dependency parsing and logistic regression Shay Cohen (based on slides by Sharon Goldwater) 21

Dependency parsing and logistic regression Shay Cohen (based on slides by Sharon Goldwater) 21

Dependency parsing and logistic regression Shay Cohen (based on slides by Sharon Goldwater) 21 October 2019 Last class Dependency parsing: a fully lexicalized formalism; tree edges connect words in the sentence based on head-dependent

764 views • 43 slides

More recommend