Statistical Parsing Dependency parsing ar ltekin University of - PowerPoint PPT Presentation

Statistical Parsing Dependency parsing Çağrı Çöltekin University of Tübingen Seminar für Sprachwissenschaft November 2016

Recap/background Dependency grammar Dependency parsing Evaluation Summary Ingredients of a parser representations Ç. Çöltekin, SfS / University of Tübingen November 2016 1 / 45 • A grammar - useful and easy to process representations • A parsing algorithm - effjcient enumeration of possible • A disambiguation method - fjnding most likely analyses

is a distinguished start symbol Recap/background VP V saw S NP John V NP saw NP Marry Ç. Çöltekin, SfS / University of Tübingen November 2016 John | Marry V NP Dependency grammar VP NP VP S for R is a set of rules of the form S N is a set of non-terminal symbols N, S, R) Context-free parsing: grammars Summary Evaluation Dependency parsing 2 / 45 A phrase structure grammar is a tuple ( Σ , Σ is a set of terminal symbols

is a distinguished start symbol Recap/background VP V saw S NP John V NP saw NP Marry Ç. Çöltekin, SfS / University of Tübingen November 2016 John | Marry V NP Dependency grammar VP NP VP S for R is a set of rules of the form S N is a set of non-terminal symbols N, S, R) Context-free parsing: grammars Summary Evaluation Dependency parsing 2 / 45 A phrase structure grammar is a tuple ( Σ , Σ is a set of terminal symbols

Recap/background NP November 2016 SfS / University of Tübingen Ç. Çöltekin, Marry NP saw V VP John NP S saw V John | Marry V NP Dependency grammar VP NP VP S for R is a set of rules of the form N is a set of non-terminal symbols N, S, R) Context-free parsing: grammars Summary Evaluation Dependency parsing 2 / 45 A phrase structure grammar is a tuple ( Σ , Σ is a set of terminal symbols S ∈ N is a distinguished start symbol

Recap/background Dependency grammar November 2016 SfS / University of Tübingen Ç. Çöltekin, Marry NP saw V VP John NP S V NP VP 2 / 45 Dependency parsing Evaluation Summary Context-free parsing: grammars N, S, R) R is a set of rules of the form S N is a set of non-terminal symbols A phrase structure grammar is a tuple ( Σ , Σ is a set of terminal symbols S ∈ N is a distinguished start symbol A → α for A ∈ N α ∈ Σ ∪ N → NP VP → V NP → John | Marry → saw

Recap/background Dependency grammar November 2016 SfS / University of Tübingen Ç. Çöltekin, time complexity) manner (but recovering all parses can require exponential polynomial time complexity in the length of the input Context-free parsing: parsing algorithms Summary Evaluation Dependency parsing 3 / 45 • Top-down parsers start with S , and try to derive the input • Bottom-up parsers start with the input, and try to reduce it to S • Naive search (in both directions) has exponential time • Chart parsing methods (CKY, Earley) do recognition in • Chart parsers also represent ambiguity in a space effjcient

Recap/background Dependency grammar November 2016 SfS / University of Tübingen Ç. Çöltekin, set of (global) features with the independence assumption – It is also diffjcult to incorporate arbitrary/global features preferences/constraints – They cannot model structural or lexical parse Context-free parsing: disambiguation Summary Evaluation Dependency parsing 4 / 45 • PCFGs provide a fjrst approximation to fjnding most likely • But their independence assumptions are too strong: • Lexicalized grammars (or parent annotation) may help • Discriminative (re-ranking) models can incorporate richer

Recap/background Dependency grammar Dependency parsing Evaluation Summary Short divergence: deterministic parsing designed not to be ambiguous have a single (semantic) interpretation backtracking) parsing is possible with a short lookahead Ç. Çöltekin, SfS / University of Tübingen November 2016 5 / 45 • Unlike natural languages, programming languages are • Every programming language sentence (program) has to • Local ambiguity may happen, but deterministic (without

Recap/background Dependency grammar November 2016 SfS / University of Tübingen Ç. Çöltekin, appropriate CFG grammars written by hand often parser-generators (e.g., yacc) are used for converting k is the number of lookahead symbols needed (typically 1) R means rightmost derivation L means left-to-right by such parsers deterministic parsers used in compilers LR(k) grammars and shift-reduce parsing Summary Evaluation Dependency parsing 6 / 45 • Shift-reduce parsers are bottom-up, table-based, • For the classes of grammar LR(k) grammars can be parsed • Constructing an LR(k) grammar tables by hand is diffjcult,

Recap/background Dependency grammar November 2016 SfS / University of Tübingen Ç. Çöltekin, the stack pop them and push the LHS Reduce if the symbols on top of the stack match the RHS of a rule, Shift the input symbol from the bufger to the stack 7 / 45 tokens string Shift-reduce parsing Summary Evaluation Dependency parsing • A shift-reduce parser does a single pass over the input • It makes use of a stack , the lookahead and a bufger of unseen • It deterministically applies two operations: • Accepts the input, if the bufger is empty, and S is on top of

Recap/background [ term Dependency grammar shift [ term * 3 ] reduce [ term * factor ] reduce ] (?) reduce [ exp ] accept Grammar: Ç. Çöltekin, SfS / University of Tübingen November 2016 [ term * 3 ] shift [ Dependency parsing Evaluation Summary Shift-reduce parsing example Input: 2 * 3 stack * 3 ] action bufger 2 * 3 ] shift [ 2 * 3 ] reduce [ factor * 3 ] reduce [ term 8 / 45 exp → exp + term exp → term term → term * factor term → factor factor → ( exp ) factor → [0-9]+

Recap/background [ term Dependency grammar shift [ term * 3 ] reduce [ term * factor ] reduce ] (?) reduce [ exp ] accept Grammar: Ç. Çöltekin, SfS / University of Tübingen November 2016 [ term * 3 ] shift [ Dependency parsing Evaluation Summary Shift-reduce parsing example Input: 2 * 3 stack * 3 ] action bufger 2 * 3 ] shift [ 2 * 3 ] reduce [ factor * 3 ] reduce [ term 8 / 45 exp → exp + term exp → term term → term * factor term → factor factor → ( exp ) factor → [0-9]+

Recap/background [ term Dependency grammar shift [ term * 3 ] reduce [ term * factor ] reduce ] (?) reduce [ exp ] accept Grammar: Ç. Çöltekin, SfS / University of Tübingen November 2016 [ term * 3 ] shift [ Dependency parsing Evaluation Summary Shift-reduce parsing example Input: 2 * 3 stack * 3 ] action bufger 2 * 3 ] shift [ 2 * 3 ] reduce [ factor * 3 ] reduce [ term 8 / 45 exp → exp + term exp → term term → term * factor term → factor factor → ( exp ) factor → [0-9]+

Recap/background [ term Dependency grammar shift [ term * 3 ] reduce [ term * factor ] reduce ] (?) reduce [ exp ] accept Grammar: Ç. Çöltekin, SfS / University of Tübingen November 2016 [ term * 3 ] shift [ Dependency parsing Evaluation Summary Shift-reduce parsing example Input: 2 * 3 stack * 3 ] action bufger 2 * 3 ] shift [ 2 * 3 ] reduce [ factor * 3 ] reduce [ term 8 / 45 exp → exp + term exp → term term → term * factor term → factor factor → ( exp ) factor → [0-9]+

Recap/background [ term Dependency grammar [ term * 3 ] reduce [ term * factor ] reduce ] [ term * reduce [ exp ] accept Grammar: Ç. Çöltekin, SfS / University of Tübingen November 2016 3 ] shift shift (?) [ Dependency parsing Evaluation Summary Shift-reduce parsing example Input: 2 * 3 stack * 3 ] action bufger 2 * 3 ] shift [ 2 * 3 ] reduce [ factor * 3 ] reduce [ term 8 / 45 exp → exp + term exp → term term → term * factor term → factor factor → ( exp ) factor → [0-9]+

Recap/background [ term Dependency grammar [ term * 3 ] reduce [ term * factor ] reduce ] [ term * reduce [ exp ] accept Grammar: Ç. Çöltekin, SfS / University of Tübingen November 2016 3 ] shift shift (?) [ Dependency parsing Evaluation Summary Shift-reduce parsing example Input: 2 * 3 stack * 3 ] action bufger 2 * 3 ] shift [ 2 * 3 ] reduce [ factor * 3 ] reduce [ term 8 / 45 exp → exp + term exp → term term → term * factor term → factor factor → ( exp ) factor → [0-9]+

Recap/background [ term Dependency grammar [ term * 3 ] reduce [ term * factor ] reduce ] [ term * reduce [ exp ] accept Grammar: Ç. Çöltekin, SfS / University of Tübingen November 2016 3 ] shift shift (?) [ Dependency parsing Evaluation Summary Shift-reduce parsing example Input: 2 * 3 stack * 3 ] action bufger 2 * 3 ] shift [ 2 * 3 ] reduce [ factor * 3 ] reduce [ term 8 / 45 exp → exp + term exp → term term → term * factor term → factor factor → ( exp ) factor → [0-9]+

Recap/background [ term Dependency grammar [ term * 3 ] reduce [ term * factor ] reduce ] [ term * reduce [ exp ] accept Grammar: Ç. Çöltekin, SfS / University of Tübingen November 2016 3 ] shift shift (?) [ Dependency parsing Evaluation Summary Shift-reduce parsing example Input: 2 * 3 stack * 3 ] action bufger 2 * 3 ] shift [ 2 * 3 ] reduce [ factor * 3 ] reduce [ term 8 / 45 exp → exp + term exp → term term → term * factor term → factor factor → ( exp ) factor → [0-9]+