Span-Based Constituency Parsing with Provably Optimal Dynamic - PowerPoint PPT Presentation

Span-Based Constituency Parsing with Provably Optimal Dynamic Oracles James Cross and Liang Huang Oregon State University EMNLP, Austin, TX November 2, 2016

Dependency vs. Constituency S ROOT NP VP PRP MD VBP S subj dobj dobj aux I do like VP I do like eating fish VBG NP PRP MD VBP VBG NN eating NN fish search F 1 search UAS Carreras et al. (2008) cubic 91.1 Zhang & Nivre 2011 beam 92.9 Chen & Manning 2014 greedy 91.8 Shindo et al. (2012) cubic 91.1 Zhou et al. (2015) beam 93.3 Thang et al. (2015) (A*) ~cubic 91.1 Weiss et al. (2015) beam 94.0 Watanabe et al. (2015) beam 90.7 our work (ACL 2016) greedy 93.4 Vinyals et al. (2015) (WSJ) beam 90.5 Andor et al. (2016) beam 94.4 Red = Neural 2

Dependency vs. Constituency S ROOT NP VP PRP MD VBP S subj dobj dobj aux I do like VP I do like eating fish VBG NP PRP MD VBP VBG NN eating NN fish search F 1 search UAS Carreras et al. (2008) cubic 91.1 Zhang & Nivre 2011 beam 92.9 Chen & Manning 2014 greedy 91.8 Shindo et al. (2012) cubic 91.1 Zhou et al. (2015) beam 93.3 Thang et al. (2015) (A*) ~cubic 91.1 Weiss et al. (2015) beam 94.0 Watanabe et al. (2015) beam 90.7 our work (ACL 2016) greedy 93.4 Vinyals et al. (2015) (WSJ) beam 90.5 Andor et al. (2016) beam 94.4 This Work greedy 91.3 Red = Neural 2

Outline • Span-Based Constituency Parsing • Bi-Directional LSTM Span Features • Provably Optimal Dynamic Oracle • Experiments 3

Span-Based Parsing • Previous work uses tree structures on stack • We simplify to operate directly on sentence spans • Simple-to-implement linear-time parsing Stack Queue NP VP’ previous work I/PRP do/MD like/VBP eating/VBG fish/NN Stack Queue our work I/PRP do/MD like/VBP eating/VBG fish/NN 0 1 3 4 5 4

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish current I/PRP do/MD like/VBP eating/VBG fish/NN t = {} brackets 0 1 2 3 4 5 5

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish current I/PRP do/MD like/VBP eating/VBG fish/NN t = {} brackets 0 1 2 3 4 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN 4 0 1 2 3 5 5

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish current I/PRP do/MD like/VBP eating/VBG fish/NN t = {} brackets 0 1 2 3 4 5 Shift Label-NP I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 2 3 5 5

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish current I/PRP do/MD like/VBP eating/VBG fish/NN t = {} brackets 0 1 2 3 4 5 Shift Label-NP I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 2 3 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN 0 1 2 3 4 5 5

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish current I/PRP do/MD like/VBP eating/VBG fish/NN t = {} brackets 0 1 2 3 4 5 Shift Label-NP I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 2 3 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN No-Label t = { 0 NP 1 } 0 1 2 3 4 5 5

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish current I/PRP do/MD like/VBP eating/VBG fish/NN t = {} brackets 0 1 2 3 4 5 Shift Label-NP I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 2 3 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN No-Label t = { 0 NP 1 } 0 1 2 3 4 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN 0 1 2 3 4 5 5

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish current I/PRP do/MD like/VBP eating/VBG fish/NN t = {} brackets 0 1 2 3 4 5 Shift Label-NP I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 2 3 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN No-Label t = { 0 NP 1 } 0 1 2 3 4 5 Shift No-Label I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 0 1 2 3 4 5 5

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 0 1 2 3 4 5 6

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 0 1 2 3 4 5 Combine I/PRP do/MD like/VBP eating/VBG fish/NN 4 0 1 3 5 6

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 0 1 2 3 4 5 Combine No-Label I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 3 5 6

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 0 1 2 3 4 5 Combine No-Label I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 3 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN 0 1 3 4 5 6

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 0 1 2 3 4 5 Combine No-Label I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 3 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN No-Label t = { 0 NP 1 } 0 1 3 4 5 6

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 0 1 2 3 4 5 Combine No-Label I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 3 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN No-Label t = { 0 NP 1 } 0 1 3 4 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN 0 1 3 4 5 6

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 0 1 2 3 4 5 Combine No-Label I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 } 4 0 1 3 5 Shift I/PRP do/MD like/VBP eating/VBG fish/NN No-Label t = { 0 NP 1 } 0 1 3 4 5 Shift Label-NP I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 , 4 NP 5 } 0 1 3 4 5 6

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 , 4 NP 5 } 0 1 3 4 5 7

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 , 4 NP 5 } 0 1 3 4 5 Combine I/PRP do/MD like/VBP eating/VBG fish/NN 0 1 3 5 7

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 , 4 NP 5 } 0 1 3 4 5 Combine Label-S-VP t = { 0 NP 1 , 4 NP 5 , I/PRP do/MD like/VBP eating/VBG fish/NN 3 S 5 , 3 VP 5 } 0 1 3 5 7

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP Label- X Label VBG NP ` (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 , 4 NP 5 } 0 1 3 4 5 Combine Label-S-VP t = { 0 NP 1 , 4 NP 5 , I/PRP do/MD like/VBP eating/VBG fish/NN 3 S 5 , 3 VP 5 } 0 1 3 5 Combine I/PRP do/MD like/VBP eating/VBG fish/NN 0 1 5 7

S Shift Structural NP VP (even step) PRP MD VBP S Combine I do like VP ` Label- X Label VBG NP (odd step) No-Label eating NN fish I/PRP do/MD like/VBP eating/VBG fish/NN t = { 0 NP 1 , 4 NP 5 } 0 1 3 4 5 Combine Label-S-VP t = { 0 NP 1 , 4 NP 5 , I/PRP do/MD like/VBP eating/VBG fish/NN 3 S 5 , 3 VP 5 } 0 1 3 5 Combine t = { 0 NP 1 , 4 NP 5 , I/PRP do/MD like/VBP eating/VBG fish/NN Label-VP 3 S 5 , 3 VP 5 , 0 1 5 1 VP 5 } 7