Natural Language Processing (CSE 517): Dependency Syntax and Parsing Noah A. Smith Swabha Swayamdipta � 2018 c University of Washington { nasmith,swabha } @cs.washington.edu May 11, 2018 1 / 93
Recap: Phrase Structure S NP NP VP JJ NN DT NN NN NN VBD NP PP last year The luxury auto maker sold CD NN IN NP 1,214 cars DT NNP in the U.S. 2 / 93
Parent Annotation (Johnson, 1998) S ROOT NP S NP S VP S JJ NP NN NP DT NP NN NP NN NP NN NP last year VBD VP NP VP PP VP The luxury auto maker CD NP NN NP sold IN PP NP PP 1,214 cars in DT NP NNP NP the U.S. Increases the “vertical” Markov order: p ( children | parent , grandparent ) 3 / 93
Headedness S NP NP VP JJ NN DT NN NN NN last year NP PP VBD The luxury auto maker CD NN sold NP IN 1,214 cars DT in NNP the U.S. Suggests “horizontal” markovization: � p ( children | parent ) = p ( head | parent ) · p ( i th sibling | head , parent ) i 4 / 93
Lexicalization S sold NP maker NP year VP sold JJ last NN year DT The NN luxury NN auto NN maker last year VBD sold NP cars PP in The luxury auto maker sold CD 1,214 NN cars IN in NP U.S. 1,214 cars in DT the NNP U.S. the U.S. Each node shares a lexical head with its head child. 5 / 93
Dependencies Informally, you can think of dependency structures as a transformation of phrase-structures that ◮ maintains the word-to-word relationships induced by lexicalization, ◮ adds labels to them, and ◮ eliminates the phrase categories. There are also linguistic theories built on dependencies (Tesni` ere, 1959; Mel’ˇ cuk, 1987), as well as treebanks corresponding to those. ◮ Free(r)-word order languages (e.g., Czech) 6 / 93
Dependency Tree: Definition Let x = � x 1 , . . . , x n � be a sentence. Add a special root symbol as “ x 0 .” A dependency tree consists of a set of tuples � p, c, ℓ � , where ◮ p ∈ { 0 , . . . , n } is the index of a parent ◮ c ∈ { 1 , . . . , n } is the index of a child ◮ ℓ ∈ L is a label Different annotation schemes define different label sets L , and different constraints on the set of tuples. Most commonly: ◮ The tuple is represented as a directed edge from x p to x c with label ℓ . ◮ The directed edges form an arborescence (directed tree) with x 0 as the root (sometimes denoted root ). 7 / 93
Example S NP VP Pronoun Verb NP we wash Determiner Noun our cats Phrase-structure tree. 8 / 93
Example S NP VP Pronoun NP Verb we wash Determiner Noun our cats Phrase-structure tree with heads. 9 / 93
Example S wash NP we VP wash Pronoun we NP cats Verb wash we wash Determiner our Noun cats our cats Phrase-structure tree with heads, lexicalized. 10 / 93
Example we wash our cats “Bare bones” dependency tree. 11 / 93
Example we wash our cats who stink 12 / 93
Example we vigorously wash our cats who stink 13 / 93
Content Heads vs. Function Heads Credit: Nathan Schneider little kids were always watching birds with fish little kids were always watching birds with fish 14 / 93
Labels root pobj sbj dobj prep kids saw birds with fish Key dependency relations captured in the labels include: subject, direct object, preposition object, adjectival modifier, adverbial modifier. In this lecture, I will mostly not discuss labels, to keep the algorithms simpler. 15 / 93
Coordination Structures we vigorously wash our cats and dogs who stink The bugbear of dependency syntax. 16 / 93
Example we vigorously wash our cats and dogs who stink Make the first conjunct the head? 17 / 93
Example we vigorously wash our cats and dogs who stink Make the coordinating conjunction the head? 18 / 93
Example we vigorously wash our cats and dogs who stink Make the second conjunct the head? 19 / 93
Nonprojective Example PU ROOT TMP ATT PC VC ATT SBJ ATT A hearing is scheduled on the issue today . 20 / 93
Dependency Schemes ◮ Direct annotation. ◮ Transform the treebank: define “head rules” that can select the head child of any node in a phrase-structure tree and label the dependencies. ◮ More powerful, less local rule sets, possibly collapsing some words into arc labels. ◮ Stanford dependencies are a popular example (de Marneffe et al., 2006). ◮ Only results in projective trees. ◮ Rule based dependencies, followed by manual correction. 21 / 93
Approaches to Dependency Parsing 1. Transition-based parsing with a stack. 2. Chu-Liu-Edmonds algorithm for arborescences (directed graphs). 3. Dynamic programming with the Eisner algorithm. 22 / 93
Transition-Based Parsing ◮ Dependency tree represented as a linear sequence of transitions . 23 / 93
Transition-Based Parsing ◮ Dependency tree represented as a linear sequence of transitions . ◮ Transitions: Simple operations to be executed on a parser configuration 24 / 93
Transition-Based Parsing ◮ Dependency tree represented as a linear sequence of transitions . ◮ Transitions: Simple operations to be executed on a parser configuration ◮ Parser Configuration: stack S and a buffer B . 25 / 93
Transition-Based Parsing ◮ Dependency tree represented as a linear sequence of transitions . ◮ Transitions: Simple operations to be executed on a parser configuration ◮ Parser Configuration: stack S and a buffer B . ◮ During parsing, apply a classifier to decide which transition to take next, greedily. No backtracking. 26 / 93
Transition-Based Parsing: Transitions Parser Configuration: ◮ Initial Configuration: buffer contains x and the stack contains the root . ◮ Final Configuration: Empty buffer and the stack contains the entire tree. 27 / 93
Transition-Based Parsing: Transitions Parser Configuration: ◮ Initial Configuration: buffer contains x and the stack contains the root . ◮ Final Configuration: Empty buffer and the stack contains the entire tree. Transitions : “arc-standard” transition set (Nivre, 2004): ◮ shift the word at the front of the buffer B onto the stack S . ◮ right-arc : u = pop ( S ) ; v = pop ( S ) ; push ( S, v → u ) . ◮ left-arc : u = pop ( S ) ; v = pop ( S ) ; push ( S, v ← u ) . 28 / 93
Transition-Based Parsing: Transitions Parser Configuration: ◮ Initial Configuration: buffer contains x and the stack contains the root . ◮ Final Configuration: Empty buffer and the stack contains the entire tree. Transitions : “arc-standard” transition set (Nivre, 2004): ◮ shift the word at the front of the buffer B onto the stack S . ◮ right-arc : u = pop ( S ) ; v = pop ( S ) ; push ( S, v → u ) . ◮ left-arc : u = pop ( S ) ; v = pop ( S ) ; push ( S, v ← u ) . (For labeled parsing, add labels to the right-arc and left-arc transitions.) 29 / 93
Transition-Based Parsing: Example Buffer B : we vigorously Stack S : wash our root cats who stink Actions: 30 / 93
Transition-Based Parsing: Example Buffer B : vigorously Stack S : wash our we cats root who stink Actions: shift 31 / 93
Transition-Based Parsing: Example Buffer B : Stack S : wash our vigorously cats we who root stink Actions: shift shift 32 / 93
Transition-Based Parsing: Example Buffer B : Stack S : our wash cats vigorously who we stink root Actions: shift shift shift 33 / 93
Transition-Based Parsing: Example Stack S : Buffer B : our cats vigorously wash who stink we root Actions: shift shift shift left-arc 34 / 93
Transition-Based Parsing: Example Stack S : Buffer B : our cats who we vigorously wash stink root Actions: shift shift shift left-arc left-arc 35 / 93
Transition-Based Parsing: Example Stack S : Buffer B : our cats who stink we vigorously wash root Actions: shift shift shift left-arc left-arc shift 36 / 93
Transition-Based Parsing: Example Stack S : cats Buffer B : our who stink we vigorously wash root Actions: shift shift shift left-arc left-arc shift shift 37 / 93
Transition-Based Parsing: Example Stack S : Buffer B : our cats who stink we vigorously wash root Actions: shift shift shift left-arc left-arc shift shift left-arc 38 / 93
Transition-Based Parsing: Example Stack S : who Buffer B : our cats stink we vigorously wash root Actions: shift shift shift left-arc left-arc shift shift left-arc shift 39 / 93
Transition-Based Parsing: Example Stack S : stink who Buffer B : our cats we vigorously wash root Actions: shift shift shift left-arc left-arc shift shift left-arc shift shift 40 / 93
Transition-Based Parsing: Example Stack S : who stink Buffer B : our cats we vigorously wash root Actions: shift shift shift left-arc left-arc shift shift left-arc shift shift right-arc 41 / 93
Transition-Based Parsing: Example Stack S : our cats who stink Buffer B : we vigorously wash root Actions: shift shift shift left-arc left-arc shift shift left-arc shift shift right-arc right-arc 42 / 93
Transition-Based Parsing: Example Stack S : Buffer B : we vigorously wash our cats who stink root Actions: shift shift shift left-arc left-arc shift shift left-arc shift shift right-arc right-arc right-arc 43 / 93
Recommend
More recommend
