Transition-Based Parsing Joakim Nivre Uppsala University - PowerPoint PPT Presentation

Transition-Based Parsing Joakim Nivre Uppsala University Department of Linguistics and Philology joakim.nivre@lingfil.uu.se Transition-Based Parsing 1(19)

1. Transition Systems 2. Greedy Classifier-Based Parsing 3. Beam Search and Structured Learning 4. Non-Projective Parsing Transition-Based Parsing 2(19)

Transition Systems S = ( C , T , c s , C t ) 1. C is a set of configurations 2. T is a set of transitions, each t : C → C 3. c s is an initialization function, c s ( x ) ∈ C 4. C t ⊆ C is a set of terminal configurations Transition-Based Parsing 3(19)

Transition Systems c = (Σ , B , A ) 1. Σ is a list of nodes in V x , known as the stack 2. B is a list of nodes in V x , known as the buffer 3. A is a set of dependency arcs in V x × L × V x Transition-Based Parsing 4(19)

Transition Systems C 0 , m = ( c 0 , c 1 , . . . , c m ) 1. c 0 = c s ( x ) , 2. c m ∈ C t , 3. for every i (1 ≤ i ≤ m ), c i = t ( c i − 1 ) for some t ∈ T G c m = ( V x , A c m ) Transition-Based Parsing 5(19)

Transition Systems c s ( x = x 1 , . . . , x n ) = ([ 0 ] , [ 1 , . . . , n ] , ∅ ) Initialization: Terminal: C t = { c ∈ C | c = ([ 0 ] , [ ] , A ) } Transitions: ( σ, [ i | β ] , A ) ⇒ ([ σ | i ] , β, A ) (Shift) ([ σ | i | j ] , B , A ) ⇒ ([ σ | j ] , B , A ∪{ ( j , l , i ) } ) 1 (Left-Arc l ) ([ σ | i | j ] , B , A ) ⇒ ([ σ | i ] , B , A ∪{ ( i , l , j ) } ) (Right-Arc l ) 1 Permitted only if i � = 0. Transition-Based Parsing 6(19)

Transition Systems [ ROOT ] Σ [Economic, news, had, little, effect, on, financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , Economic] Σ [news, had, little, effect, on, financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , Economic, news] Σ [had, little, effect, on, financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , news] Σ [had, little, effect, on, financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , news, had] Σ [little, effect, on, financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had] Σ [little, effect, on, financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, little] Σ [effect, on, financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, little, effect] Σ [on, financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, effect] Σ [on, financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, effect, on] Σ [financial, markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, effect, on, financial] Σ [markets, .] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, effect, on, financial, markets] Σ [.] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, effect, on, markets] Σ [.] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, effect, on] Σ [.] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, effect] Σ [.] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had] Σ [.] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had, .] Σ [ ] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT , had] Σ [ ] B Transition-Based Parsing 7(19)

Transition Systems [ ROOT ] Σ [ ] B Transition-Based Parsing 7(19)

Transition Systems Initialization: c s ( x = x 1 , . . . , x n ) = ([ 0 ] , [ 1 , . . . , n ] , ∅ ) Terminal: C t = { c ∈ C | c = (Σ , [ ] , A ) } Transitions: ( σ, [ i | β ] , A ) ⇒ ([ σ | i ] , β, A ) (Shift) ([ σ | i ] , [ j | β ] , A ) ⇒ ( σ, [ j | β ] , A ∪{ ( j , l , i ) } ) 1 (Left-Arc l ) ([ σ | i ] , [ j | β ] , A ) ⇒ ([ σ | i | j ] , β, A ∪{ ( i , l , j ) } ) (Right-Arc l ) ([ σ | i ] , B , A ) ⇒ ( σ, B , A ) 2 (Reduce) 1 Permitted only if i � = 0 and there are no k , l ′ such that ( k , l ′ , i ) ∈ A . 2 Permitted only if there are k , l ′ such that ( k , l ′ , i ) ∈ A . Transition-Based Parsing 8(19)

Transition Systems [ ROOT ] Σ [Economic, news, had, little, effect, on, financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , Economic] Σ [news, had, little, effect, on, financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT ] Σ [news, had, little, effect, on, financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , news] Σ [had, little, effect, on, financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT ] Σ [had, little, effect, on, financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had] Σ [little, effect, on, financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had, little] Σ [effect, on, financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had] Σ [effect, on, financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had, effect] Σ [on, financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had, effect, on] Σ [financial, markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had, effect, on, financial] Σ [markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had, effect, on] Σ [markets, .] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had, effect, on, markets] Σ [.] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had, effect, on] Σ [.] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had, effect] Σ [.] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had] Σ [.] B Transition-Based Parsing 9(19)

Transition Systems [ ROOT , had, .] Σ [ ] B Transition-Based Parsing 9(19)

Greedy Classifier-Based Parsing Parse( x = ( w 0 , w 1 , . . . , w n )) 1 c ← c s ( x ) 2 while c �∈ C t t ∗ ← argmax t Score ( c , t ) 3 c ← t ∗ ( c ) 4 5 return G c Transition-Based Parsing 10(19)

Greedy Classifier-Based Parsing K � Score ( c , t ) = f k ( c , t ) · w k k = 1 Transition-Based Parsing 11(19)

Greedy Classifier-Based Parsing Unigrams Bigrams Σ 0 .pos Σ 0 .pos, B 0 .pos Σ 1 .pos Σ 0 .pos, Σ 0 .lab B 0 .pos B 0 .pos, ldep( B 0 ).lab B 1 .pos B 2 .pos B 3 .pos Trigrams Σ 0 .lab Σ 1 .pos, Σ 0 .pos, B 0 .pos ldep( Σ 0 ).lab Σ 0 .pos, B 0 .pos, B 1 .pos rdep( Σ 0 ).lab B 0 .pos, B 1 .pos, B 2 .pos ldep( B 0 ).lab B 1 .pos, B 2 .pos, B 3 .pos Σ 0 .word Σ 0 .pos, ldep( Σ 0 ).lab, rdep( Σ 0 ).lab B 0 .word B 1 .word hd( Σ 0 ).word Transition-Based Parsing 12(19)

Beam Search and Structured Learning m − 1 � Score ( c 0 , m , x ) = Score ( c i , t i ) [where t i ( c i ) = c i + 1 ] i = 0 Transition-Based Parsing 13(19)

Beam Search and Structured Learning Parse( x = ( w 0 , w 1 , . . . , w n )) 1 Beam ← {� c s ( x ) , 0 . 0 �} 2 while ∃� c , s � ∈ Beam : c �∈ C t 3 NewBeam ← ∅ 4 for every ( c , s ) ∈ Beam 5 for every t ∈ T 6 NewBeam ← NewBeam ∪ {� t ( c ) , s + Score ( c , t ) �} Beam → QBest ( NewBeam ) 7 8 return ← 1Best ( Beam ) Transition-Based Parsing 14(19)

Beam Search and Structured Learning 0 , m ) } |T | Training data: T = { ( x i , c i i = 1 1 w ← 0 2 for n : 1 .. N for i : 1 .. |T | 3 c ∗ 0 , m ← Parse ( x i , w ) 4 if c ∗ 0 , m � = c i 5 0 , m w ← Update ( w , c ∗ 0 , m , c i 6 0 , m ) 7 return w Transition-Based Parsing 15(19)

Beam Search and Structured Learning Update ( w , c ∗ 0 , m , c i 0 , m ) 1 for k : 1 .. K 2 for i : 0 .. m − 1 w k ← w k − f k ( c i , t i ) 3 4 for i : 0 .. m − 1 5 w k ← w k + f k ( c i , t i ) Transition-Based Parsing 16(19)

Non-Projective Parsing Initialization: c s ( x = x 1 , . . . , x n ) = ([ 0 ] , [ 1 , . . . , n ] , ∅ ) Terminal: C t = { c ∈ C | c = ([ 0 ] , [ ] , A ) } Transitions: ( σ, [ i | β ] , A ) ⇒ ([ σ | i ] , β, A ) (Shift) ([ σ | i | j ] , B , A ) ⇒ ([ σ | j ] , B , A ∪{ ( j , l , i ) } ) 1 (Left-Arc l ) ([ σ | i | j ] , B , A ) ⇒ ([ σ | i ] , B , A ∪{ ( i , l , j ) } ) (Right-Arc l ) ([ σ | i | k | j ] , B , A ) ⇒ ([ σ | k | j ] , B , A ∪{ ( j , l , i ) } ) 1 (Left-Arc2 l ) ([ σ | i | k | j ] , B , A ) ⇒ ([ σ | i | k ] , B , A ∪{ ( i , l , j ) } ) (Right-Arc2 l ) ([ σ | i | k 1 | k 2 | j ] , B , A ) ⇒ ([ σ | k 1 | k 2 | j ] , B , A ∪{ ( j , l , i ) } ) 1 (Left-Arc3 l ) ([ σ | i | k 1 | k 2 | j ] , B , A ) ⇒ ([ σ | i | k 1 | k 2 ] , B , A ∪{ ( i , l , j ) } ) (Right-Arc3 l ) 1 Permitted only if i � = 0. Transition-Based Parsing 17(19)