Dependency Parsing Data structures and algorithms for Computational - PowerPoint PPT Presentation

Dependency Parsing Data structures and algorithms for Computational Linguistics III Çağrı Çöltekin ccoltekin@sfs.uni-tuebingen.de University of Tübingen Seminar für Sprachwissenschaft Winter Semester 2019–2020

Introduction Transition-based parsing WS 19–20 SfS / University of Tübingen Ç. Çöltekin, Often an artifjcial root node is used for computational convenience The arcs (relations) have labels (dependency types) Each relation defjnes one of the words as the head and the other as dependent between syntactic units The structure of the sentence is represented by asymmetric , binary relations root object subject Mary saw John a refresher Dependency grammars Evaluation Variations/improvements Graph based parsing Classifjcation 1 / 30 • No constituents, units of syntactic structure are words

Introduction Transition-based parsing WS 19–20 SfS / University of Tübingen Ç. Çöltekin, Often an artifjcial root node is used for computational convenience The arcs (relations) have labels (dependency types) Each relation defjnes one of the words as the head and the other as dependent between syntactic units root object subject Mary saw John a refresher Dependency grammars Evaluation Variations/improvements Graph based parsing Classifjcation 1 / 30 • No constituents, units of syntactic structure are words • The structure of the sentence is represented by asymmetric , binary relations

Introduction subject WS 19–20 SfS / University of Tübingen Ç. Çöltekin, Often an artifjcial root node is used for computational convenience The arcs (relations) have labels (dependency types) between syntactic units root Transition-based parsing object Mary saw John a refresher Dependency grammars Evaluation Variations/improvements Graph based parsing Classifjcation 1 / 30 • No constituents, units of syntactic structure are words • The structure of the sentence is represented by asymmetric , binary relations • Each relation defjnes one of the words as the head and the other as dependent

Introduction subject WS 19–20 SfS / University of Tübingen Ç. Çöltekin, Often an artifjcial root node is used for computational convenience between syntactic units root Transition-based parsing object Mary saw John a refresher Dependency grammars Evaluation Variations/improvements Graph based parsing Classifjcation 1 / 30 • No constituents, units of syntactic structure are words • The structure of the sentence is represented by asymmetric , binary relations • Each relation defjnes one of the words as the head and the other as dependent • The arcs (relations) have labels (dependency types)

Introduction Mary WS 19–20 SfS / University of Tübingen Ç. Çöltekin, between syntactic units root Transition-based parsing subject object saw John a refresher Dependency grammars Evaluation Variations/improvements Graph based parsing Classifjcation 1 / 30 • No constituents, units of syntactic structure are words • The structure of the sentence is represented by asymmetric , binary relations • Each relation defjnes one of the words as the head and the other as dependent • The arcs (relations) have labels (dependency types) • Often an artifjcial root node is used for computational convenience

Introduction Transition-based parsing WS 19–20 SfS / University of Tübingen Ç. Çöltekin, parsing. The above assumptions (except projectivity) are common in dependency head common assumptions, variations Dependency grammars Evaluation Variations/improvements Graph based parsing Classifjcation 2 / 30 • Single-headed : most dependency formalisms require a word to have a single • Acyclic : most dependency formalism do not allow loops in the graph • Connected : all nodes are reachable from the ‘root’ node • Projective : no crossing dependencies

Introduction Transition-based parsing WS 19–20 SfS / University of Tübingen Ç. Çöltekin, – data-driven (rules/model is learned from a treebank) – grammar-driven (hand crafted rules or constraints) – Determine the relation type – Determine the head of each word trees are limited) result is a tree) an overview Dependency parsing Evaluation Variations/improvements Graph based parsing Classifjcation 3 / 30 • Dependency parsing has many similarities with context-free parsing (e.g., the • They also have some difgerent properties (e.g., number of edges and depth of • The process involves discovering the relations between words in a sentence • Dependency parsing can be

Introduction Transition-based parsing WS 19–20 SfS / University of Tübingen Ç. Çöltekin, reduce) at each step parsing) Transition-based similar to shift-reduce parsing (used for programming language (in general, computationally more expensive) 4 / 30 Graph-based search for the best tree structure, for example There are two main approaches: common methods for data-driven parsers Dependency parsing Evaluation Variations/improvements Graph based parsing Classifjcation • fjnd minimum spanning tree (MST) • adaptations of CF chart parser (e.g., CKY) • Single pass over the sentence, determine an operation (shift or • Linear time complexity • We need an approximate method to determine the operation

Introduction Action WS 19–20 SfS / University of Tübingen Ç. Çöltekin, accept shift shift shift shift Transition-based parsing shift 5 / 30 Input bufger a refresher through an example Evaluation Stack Variations/improvements Grammar Graph based parsing Classifjcation Shift-Reduce parsing Parser states/actions S → P | S + P | S − P P → Num | P × Num | P / Num 2 + 3 × 4 2 + 3 × 4 reduce ( P → Num) P + 3 × 4 reduce ( S → P ) S + 3 × 4 S + 3 × 4 S + 3 × 4 reduce ( P → Num) S + P × 4 S + P × 4 S + P × 4 reduce ( P → P × Num) S + P reduce ( S → S + P ) S

Introduction Transition-based parsing WS 19–20 SfS / University of Tübingen Ç. Çöltekin, label phrase-structure rules) we use arc operations connecting two nodes with a actions cannot be made deterministic 6 / 30 actions are determined by a table lookup difgerences from shift-reduce parsing Transition-based parsing Evaluation Variations/improvements Graph based parsing Classifjcation • The shift-reduce parsers (for programming languages) are deterministic, • Natural language sentences are ambiguous, hence a dependency parser’s • Operations are (somewhat) difgerent: instead of reduce (using • Further operations are often defjned (e.g., to deal with non-projectivity)

Introduction Transition-based parsing WS 19–20 SfS / University of Tübingen Ç. Çöltekin, (Yamada and Matsumoto 2003; Nivre, Hall, and Nilsson 2004) using a machine learning method Shift: push the current word on to the stack Right-Arc: mark current word as a dependent of the word on top of the stack Left-Arc: mark current word as the head of the word on top of the stack Transition based parsing Evaluation Variations/improvements Graph based parsing Classifjcation 7 / 30 • Use a stack and a bufger of unprocessed words • Parsing as predicting a sequence of transitions like • Algorithm terminates when all words in the input are processed • The transitions are not naturally deterministic, best transition is predicted

Introduction stack WS 19–20 SfS / University of Tübingen Ç. Çöltekin, Transition-based parsing arcs bufger next word 8 / 30 Variations/improvements stack top Classifjcation A typical transition system Evaluation Graph based parsing ( σ | w i w j | β A ) , , Left-Arc r : ( σ | w i , w j | β , A ) ⇒ ( σ , w j | β , A ∪ { ( w j , r , w i ) } ) • pop w i , • add arc ( w j , r , w i ) to A (keep w j in the bufger) Right-Arc r : ( σ | w i , w j | β , A ) ⇒ ( σ , w i | β , A ∪ { ( w i , r , w j ) } ) • pop w i , • add arc ( w i , r , w j ) to A , • move w i to the bufger Shift: ( σ , w j | β , A ) ⇒ ( σ | w j , β , A ) • push w j to the stack • remove it from the bufger (Kübler, McDonald, and Nivre 2009, p.23)

Introduction bufger WS 19–20 SfS / University of Tübingen Ç. Çöltekin, case obl obj nsubj root Note: We need Shift for NP attachment. Note: we need Shift for NP attachment. Shift stack Transition-based parsing binoculars with her saw We Root Transition based parsing: example Evaluation Variations/improvements Graph based parsing Classifjcation 9 / 30

Introduction bufger WS 19–20 SfS / University of Tübingen Ç. Çöltekin, case obl obj nsubj root Note: We need Shift for NP attachment. Note: we need Shift for NP attachment. Left-Arc(nsubj) stack Transition-based parsing binoculars with her saw We Root Transition based parsing: example Evaluation Variations/improvements Graph based parsing Classifjcation 9 / 30

Introduction bufger WS 19–20 SfS / University of Tübingen Ç. Çöltekin, case obl obj nsubj root Note: We need Shift for NP attachment. Note: we need Shift for NP attachment. Shift stack Transition-based parsing binoculars with her saw We Root Transition based parsing: example Evaluation Variations/improvements Graph based parsing Classifjcation 9 / 30