Statistical Parsing Grammars and grammar formalisms ar ltekin - PowerPoint PPT Presentation

Statistical Parsing Grammars and grammar formalisms Çağrı Çöltekin University of Tübingen Seminar für Sprachwissenschaft October 27, 2016

Recap amod NP NN natural NN languages nmod case IN nmod conj cc amod Ç. Çöltekin, SfS / University of Tübingen October 27, 2016 of PP Introduction JJ Constituency grammars Dependency grammars Grammar formalisms Finale This course is about … NP NP statistical parsing NN constituency CC and NN dependency NN 1 / 31

Recap Introduction October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, for linguistic research applications like speech recognition and machine translation applications like question answering , information extraction (semantic) interpretation of sentences hence it is useful for Why do we need syntactic parsing? Finale Grammar formalisms Dependency grammars Constituency grammars 2 / 31 • Often, syntactic analysis is an intermediate step helping • (Statistical) parsers are also used as language models for • It can be used for grammar checking , and can be a useful tool

Recap Introduction Constituency grammars Dependency grammars Grammar formalisms Finale Ingredients of a parser Ç. Çöltekin, SfS / University of Tübingen October 27, 2016 3 / 31 • A grammar • An algorithm for parsing • A method for ambiguity resolution

Recap Introduction October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, infjnite set of strings (not necessarily a natural language) of language—as in HPSG, LFG, CCG language—as in a ‘grammar (book) of English’ The term grammar is used for, Grammars Finale Grammar formalisms Dependency grammars Constituency grammars 4 / 31 • a description of the whole system/structure of a • a grammar formalism, that are often developed as theory • A formal (fjnite) specifjcation of a language as a possibly

Recap Introduction Constituency grammars Dependency grammars Grammar formalisms Finale Plan of the lecture Ç. Çöltekin, SfS / University of Tübingen October 27, 2016 5 / 31 • Constituency grammars • Dependency grammars • Brief notes on some major grammar formalisms

Recap used as synonyms October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, we will not focus on a particular grammar formalism here. Note: many grammar formalisms use constituency grammars in some way, Marry NP saw V VP John NP S context-free grammars are often Introduction or word phrases ‘constituents’, like no noun phrases words form natural groups, or computer science grammars both in linguistics, and probably the most studied Constituency grammars Finale Grammar formalisms Dependency grammars Constituency grammars 6 / 31 • Constituency grammars are • The main idea is that a group of • phrase structure grammars or

greated] Marry’ (see Müller 2016, for more examples). Recap but, presumably, no question with answer ‘John saw’ October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, Note, however, these tests are leaky, e.g., ‘[John saw] and [Peter John [saw Marry] and [said ‘hi’] Introduction 7 / 31 What is a constituency Linguists ofger a number of tests for constituency, such as Finale Grammar formalisms Dependency grammars Constituency grammars • They can answer questions: Q: ‘What did John do? → A: ‘saw Marry’ • Substitution with a pronoun forms: Q: ‘John [read the book] last week? → A: ‘John [did that] last week.’ • Fronting, topicalization: ‘John likes [reading books]’ → ‘[Reading books], John likes’ • Coordination: • …

Recap but, presumably, no question with answer ‘John saw’ October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, greated] Marry’ (see Müller 2016, for more examples). Note, however, these tests are leaky, e.g., ‘[John saw] and [Peter John [saw Marry] and [said ‘hi’] Introduction 7 / 31 What is a constituency Linguists ofger a number of tests for constituency, such as Finale Grammar formalisms Dependency grammars Constituency grammars • They can answer questions: Q: ‘What did John do? → A: ‘saw Marry’ • Substitution with a pronoun forms: Q: ‘John [read the book] last week? → A: ‘John [did that] last week.’ • Fronting, topicalization: ‘John likes [reading books]’ → ‘[Reading books], John likes’ • Coordination: • …

Recap can be derived from S with the October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, Marry NP saw V VP John NP S rewrite rules R The grammar accepts a sentence if it Introduction for R is a set of rules of the form is a distinguished start symbol S N is a set of non-terminal symbols N, S, R) Formal defjnition Finale Grammar formalisms Dependency grammars Constituency grammars 8 / 31 A phrase structure grammar is a tuple ( Σ , Σ is a set of terminal symbols

Recap can be derived from S with the October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, Marry NP saw V VP John NP S rewrite rules R The grammar accepts a sentence if it Introduction for R is a set of rules of the form is a distinguished start symbol S N is a set of non-terminal symbols N, S, R) Formal defjnition Finale Grammar formalisms Dependency grammars Constituency grammars 8 / 31 A phrase structure grammar is a tuple ( Σ , Σ is a set of terminal symbols

Recap can be derived from S with the October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, Marry NP saw V VP John NP S rewrite rules R The grammar accepts a sentence if it Introduction for R is a set of rules of the form N is a set of non-terminal symbols N, S, R) Formal defjnition Finale Grammar formalisms Dependency grammars Constituency grammars 8 / 31 A phrase structure grammar is a tuple ( Σ , Σ is a set of terminal symbols S ∈ N is a distinguished start symbol

Recap VP VP V saw NP Marry S NP VP V NP NP NP John | Marry V saw Ç. Çöltekin, SfS / University of Tübingen October 27, 2016 Introduction John S N is a set of non-terminal symbols Constituency grammars Dependency grammars Grammar formalisms Finale Formal defjnition N, S, R) rewrite rules R 8 / 31 The grammar accepts a sentence if it R is a set of rules of the form can be derived from S with the A phrase structure grammar is a tuple ( Σ , Σ is a set of terminal symbols S ∈ N is a distinguished start symbol αAβ → γ for A ∈ N α, β, γ ∈ Σ ∪ N → → → →

Recap VP VP V saw NP Marry S NP VP V NP NP NP John | Marry V saw Ç. Çöltekin, SfS / University of Tübingen October 27, 2016 Introduction John S N is a set of non-terminal symbols Constituency grammars Dependency grammars Grammar formalisms Finale Formal defjnition N, S, R) rewrite rules R 8 / 31 R is a set of rules of the form can be derived from S with the A phrase structure grammar is a tuple ( Σ , Σ is a set of terminal symbols S ∈ N is a distinguished start symbol αAβ → γ for A ∈ N α, β, γ ∈ Σ ∪ N • The grammar accepts a sentence if it → → → →

Recap V NP October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, called sentential forms or, S successive application of rewrite rules. saw V Introduction NP John | Marry 9 / 31 Finale Constituency grammars NP VP Dependency grammars S Grammar formalisms The example grammar: VP Example derivation → → → → • Phrase structure grammars derive a sentence with S ⇒ NP VP ⇒ John VP ⇒ John V NP ⇒ John saw NP ⇒ John saw Marry ∗ ⇒ John saw Marry • The intermediate forms that contain non-terminals are

Recap type 1 Context sensitive, recognized by linear-bound automaton October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, or type 3 Regular, recognized by fjnite-state automata type 2 Context free, recognized by push-down automata Introduction type 2.1 Mildly context sensitive (TAG, CCG) (HPSG, LFG) type 0 Recursively enumerable, recognized by Turing machines Chomsky hierarchy of grammars Finale Grammar formalisms Dependency grammars Constituency grammars 10 / 31 αAβ → γ γ ̸ = ϵ αAβ → αγβ, A → α A → aB A → Ba In all of the above A and B are non-terminals, a is a terminal symbol, α , β , γ are sequences of terminals and non-terminals, and ϵ is the empty string.

Recap including morphological analysis, partial parsing. October 27, 2016 SfS / University of Tübingen Ç. Çöltekin, for some syntactic phenomena. they are too powerful, hence too expensive language parsers Introduction 11 / 31 memory Some examples Finale Grammar formalisms Dependency grammars Constituency grammars • Regular grammars (fjnite-state automata) do not have any can represent a ∗ b ∗ , but not a n b n • Finite-state automata are used in many tasks in CL, • Context free grammars (push-down automata) uses a stack can represent a n b n , a n b m c m d n , but not a n b m c n d m • Context-free grammars form the basis of most natural • Context-sensitive languages can do all of the above but • Some level of context sensitiveness seems to be necessary