Lecture 14: Formal Grammars Kai-Wei Chang CS @ University of Virginia kw@kwchang.net Couse webpage: http://kwchang.net/teaching/NLP16 CS6501: NLP 1
Critical review report (due 10/20) v 1 page maximum v Pick one paper from the suggested list v Summarize the paper (use you own words) v Provide detailed comments v What can be improved v Potential future directions v Other related work v Example (see reviewer 3 at: https://goo.gl/yrBXXo) CS6501– Natural Language Processing 2
How to model language? v So far, all the models we saw formulate sentence as a sequence v Language models v POS-tagging v Morphological analysis CS6501: NLP 3
Next key concepts v In the following few weeks, we will go beyond sequence models v Syntactic parsing – model language as a recursive generating process v Often use a tree structure CS6501: NLP 4
What is grammar? v A compact way to define and describe the structure of sentences v Why we need grammar? v Number of C++ programs? C++ standard (2014) ISO/IEC 14882:2014 976 pages. 1358 pages CS6501: NLP 5
Can we define a program that generates all English sentences? From Julia Hockenmaier, Intro to NLP CS6501: NLP 6
Basic sentence structure CS6501: NLP 7
A Markov Model v I eat shshi; I eat meat; you eat banana… v Great, it covers many sentences CS6501: NLP 8
Words take different arguments v [Good] I eat sushi v [Bad] I sleep sushi v [Bad] I give sushi v Intransitive verbs (sleep): no object v Transitive verbs (eat): take one direct object v Ditransitive verbs (give): take an additional indirect object. CS6501: NLP 9
A better model CS6501: NLP 10
Language is recursive Adjectives can modify nouns. We can have unlimited modifiers (in theory) CS6501: NLP 11
We know how to model the simple one CS6501: NLP 12
Recursion can be more complex We can have another noun phrase in preposition CS6501: NLP 13
What is sentence structure v Sentence structure is hierarchical v A sentence consists of phrases (or constituents) CS6501: NLP 14
Can have complex constituents CS6501: NLP 15
Can have complex constituents v Syntactically, constituents behave like simple ones CS6501: NLP 16
Constituency v Groups of words that behave as a single unit or phrase v E.g., Noun phrases: the man, a girl with glasses v Prepositional phrases: with classes, on a table v Verb phrase: eat sushi, sleep, sleep soundly v Phrases has a head: v Other parts called dependents v E.g, the man, a girl with glasses CS6501: NLP 17
Properties of constituents v Substitution v He talks [in class] ⇒ He talks [there] v It can move around in a sentence v He talks [in class] ⇒ [In class], he talks. v Can be used as an answer: v Where does he talk? – [In class] CS6501: NLP 18
Types of dependencies v Phrases has a head: v Other parts called dependents v E.g, the man, a girl with glasses v Dependents can be arguments or adjuncts v Arguments are obligatory All arguments have to be present and cannot be v E.g., [John] likes [Mary] occupied multiple times v Adjuncts are optional Can be an arbitrary number v E.g., John runs [fast] of adjuncts v Adverbs, PPs, Adjectives… CS6501: NLP 19
How to represent the structure CS6501: NLP 20
Structure (syntax) corresponds to meaning CS6501: NLP 21
Dependency Trees v Dependency grammar describe the structure of sentences as a graph (tree) v Nodes represent words v Edges represent dependencies CS6501: NLP 22
Phrases structure trees v Can be modeled by Context-free grammars CS6501: NLP 23
Context-free grammars CS6501: NLP 24
Parse tree defined by CFG CS6501: NLP 25
Generate sentences by CFG CS6501: NLP 26
Example: Noun Phrases CS6501: NLP 27
Example: verb phrase CS6501: NLP 28
Sentences CS6501: NLP 29
Structured Prediction –beyond sequence tagging Assign values to a set of interdependent output variables Task Input Output Part-of-speech They operate Pronoun Verb Noun And Noun Tagging ships and banks. Dependency They operate Root They operate ships and banks . Parsing ships and banks. Segmentation This image This image cannot currently cannot currently be displayed. be displayed. 30
Next few lectures v Extend HMM to deal with the tree structure v Discriminative methods for tree structure Kai-Wei Chang 31
Recommend
More recommend
