On Different Approaches to Syntactic Analysis into Bi-Lexical - PowerPoint PPT Presentation

Introduction DT Setup Results Conclusions On Different Approaches to Syntactic Analysis into Bi-Lexical Dependencies An Empirical comparison of Direct, PCFG-Based, and HPSG-Based Parsers Angelina Ivanova ♠ , Stephan Oepen ♠♥ , Rebecca Dridan ♠ , Dan Flickinger ♣ , Lilja Øvrelid ♠ ♠ University of Oslo ♥ Potsdam University ♣ Stanford University The 13th International Conference on Parsing Technologies Nara, Japan, 2013 Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 1/26

Introduction DT Setup Results Conclusions Research Question How does HPSG grammar-based parsing relate to PCFG and direct dependency approaches in terms of accuracy, efficiency and domain resilience for recovering bilexical dependencies ? Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 2/26

Introduction DT Setup Results Conclusions Motivation ◮ Comparison of parsers of different frameworks is challenging; ◮ Heuristic conversion introduces fuzziness in parsing results Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 3/26

Introduction DT Setup Results Conclusions Related Work ◮ Grammar-based parser is not necessarily more accurate than PCFG-based parser (Fowler and Penn, 2010) Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 4/26

Introduction DT Setup Results Conclusions Related Work ◮ Grammar-based parser is not necessarily more accurate than PCFG-based parser (Fowler and Penn, 2010) ◮ Grammar-based parser for Dutch is more accurate and domain-resilient than direct dependency parsers (Plank and van Noord, 2010) Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 4/26

Introduction DT Setup Results Conclusions Related Work ◮ Grammar-based parser is not necessarily more accurate than PCFG-based parser (Fowler and Penn, 2010) ◮ Grammar-based parser for Dutch is more accurate and domain-resilient than direct dependency parsers (Plank and van Noord, 2010) ◮ Grammatical Relations and Stanford Bilexical Dependencies as a framework-independent parser comparison (Miyao et al., 2007) Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 4/26

Introduction DT Setup Results Conclusions Related Work ◮ Grammar-based parser is not necessarily more accurate than PCFG-based parser (Fowler and Penn, 2010) ◮ Grammar-based parser for Dutch is more accurate and domain-resilient than direct dependency parsers (Plank and van Noord, 2010) ◮ Grammatical Relations and Stanford Bilexical Dependencies as a framework-independent parser comparison (Miyao et al., 2007) ◮ CFG parsers are more accurate than direct dependency parsers on recovering bilexical Stanford Dependencies (Cer et al., 2010) Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 4/26

Introduction DT Setup Results Conclusions Experiment Setup Cross-framework parser evaluation on bilexical syntactic dependencies Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 5/26

Introduction DT Setup Results Conclusions DT Dependencies root n-nh aj-hdn sb-hd hd-cmp hd-cmp Sun- filled Mountain View didn’t impress me. n - pn le v np noger le n - pn le v vp did-n le v np* le n - pr-me le ⋆ DT: Derivation Tree–Derived Bi-Lexical Syntactic Dependencies (Ivanova et al., 2012) ⋆ Derived from English Resource Grammar (ERG; Flickinger, 2000) derivations ⋆ 48 broad HPSG constructions as dependency labels ⋆ About 1000 ERG lexical types as parts-of-speech ⋆ Deviate from PTB tokenization assumptions ⋆ Correspond to CoNLL in terms of Jaccard similarity over unlabeled dependencies Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 6/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c n-nh v-cpd c n sg ilr didn’t v n3s-bse ilr hdn bnp-qnt c n - pn le w hyphen plr v pas odlr v np* le w period plr Mountain View n - pr-me le n - pn le v np noger le impress Sun- filled me. Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le w hyphen plr v pas odlr v np* le w period plr Mountain View n - pn le v np noger le impress n - pr-me le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26

Introduction DT Setup Results Conclusions ERG derivation & DT sb-hd mc c hdn bnp-pn c hd-cmp u c aj-hdn norm c v vp did-n le hd-cmp u c didn’t n-nh v-cpd c n sg ilr v n3s-bse ilr hdn bnp-qnt c n - pn le v np* le w period plr w hyphen plr v pas odlr Mountain View impress n - pr-me le n - pn le v np noger le me. Sun- filled Ivanova et al. On Different Approaches to Syntactic Analysis IWPT 2013 7/26