Welcome back to CIS 530! PLEASE TYPE YOUR QUESTIONS IN THE CHAT IF YOUR INTERNET IS TOO SLOW TO SEE THE VIDEO, YOU CAN FIND THE SLIDES ON THE CLASS WEBSITE
New course policies 1. I’m granting everyone 10 extra late days. You can now use up to 3 late days per HW, quiz or project milestone. 2. I’m offering a HW option for the term project component of the grade. You can do 4 extra HW assignments instead of a project. 3. I’m allowing everyone to drop their lowest scoring quiz. 4. Everyone can drop their lowest scoring homework. (You can’t drop project milestones). 5. You can opt to do the course pass/fail. 50% and above is passing.
Homework Option We are creating a set of 4 additional weekly homework assignments. They will have the same deadlines as the project milestones. You may do the homework assignments individually or in pairs. HW9: Classifying Depression due – requires special data access HW10: Neural Machine Translation HW11: BERT HW12: Perspectives Detection You can do the homework individually or in pairs. HW will be graded based on leaderboard and reports (autograders may not be available).
Project Option The project is a team exercise, with teams of 4-6. Your project will be a self-designed multi-week team-based effort. Milestones: 1. Submit a formal project definition and a literature review. (due 4/8) 2. Collect your data, write an evaluation script and a baseline. (4/15) 3. Implement a published baseline. Prepare a draft of your final project presentation. (4/22) 4. Finish all your extensions to the public baseline, and submit your final report. (4/29) You need to declare whether you intend to do the project or homework option by this Wednesday using the Google form linked on Piazza. http://computational-linguistics-class.org/term-project.html
Office hours Office hours are going to be held via Zoom. TAs host a host a Zoom group meeting and post the link on Piazza. We will use the chat to manage the queue. Just like you would write your name on the whiteboard in an in-person meeting. You should write this info to add yourself to the queue: 1. Your name 2. A short version of your question 3. Whether it should be discussed publicly and privately (code help) For private questions, the TA will add you to a breakout room. For public ones, we’ll discuss them as a group so you can hear the answers to other students’ questions.
Schedule http://computational-linguistics-class.org/lectures.html#now
Reminders HOMEWORK 7 DUE DATE IS WASH YOUR HANDS TAKE CARE OF YOURSELF. DUE BY MIDNIGHT ON 3/25. MENTAL HEALTH IS HW8 WILL BE DUE 4/1. IMPORTANT TOO.
Review: Constituency Parsing JURAFSKY AND MARTIN CHAPTERS 12-14
Formal Definition of a PCFG A probabilistic context-free grammar G is defined by four parameters: N is a set of non-terminal symbols (or variables) ◦ In NLP, we often use the Penn Treebank tag set Σ is set of terminal symbols ◦ These are the words (also sometimes called the leaf nodes of the parse tree) R is a set of production rules, each of the form A → β [probability] ◦ S → NP VP [0.8] ◦ S → Aux NP VP [0.15] ◦ S → VP [0.05] S is the start symbol (a non-terminal)
Treebanks as grammar Treebanks == data Initially, building a treebank might seem like it would be a lot slower and less useful than building a grammar. Mitch Marcus However, a treebank gives us many things Reusability of the labor • • Many parsers, POS taggers, etc. Valuable resource for linguistics • Broad coverage • • Frequencies and distributional information A way to evaluate systems • 10 CS 272: STATISTICAL NLP (WINTER 2019) [Marcus et al. 1993, Computational Linguistics ]
S NP-SBJ VP . DT JJ , JJ NN VBD ADJP-PRD . That cold , empty sky was JJ PP full IN NP of NN CC NN fire and light Extracted rules S → NP VP . DT → That JJ → full NP → DT JJ , JJ NN JJ → cold IN → of VP → VBD ADJP , → , NN → fire ADJP → JJ PP JJ → empty CC → and PP → IN NP NN → sky NN → light NP → NN CC NN VBD → was
Rules with counts 40717 PP → IN NP 100 VP → VBD PP-PRD 33803 S → NP-SBJ VP 100 PRN → : NP : 22513 NP-SBJ → -NONE- 100 NP → DT JJS 21877 NP → NP PP 100 NP-CLR → NN 20740 NP → DT NN 99 NP-SBJ-1 → DT NNP 14153 S → NP-SBJ VP . 98 VP → VBN NP PP-DIR 12922 VP → TO VP 98 VP → VBD PP-TMP 11881 PP-LOC → IN NP 98 PP-TMP → VBG NP 11467 NP-SBJ → PRP 97 VP → VBD ADVP-TMP VP Compute 11378 NP → -NONE- ... Probabilities using 11291 NP → NN 10 WHNP-1 → WRB JJ MLE. ... 10 VP → VP CC VP PP-TMP 989 VP → VBG S 10 VP → VP CC VP ADVP-MNR 985 NP-SBJ → NN 10 VP → VBZ S , SBAR-ADV 983 PP-MNR → IN NP 10 VP → VBZ S ADVP-TMP 983 NP-SBJ → DT 969 VP → VBN VP 12
CKY Algorithm CKY Demo at http://lxmls.it.pt/2015/cky.html 13
Ambiguity Ambiguity can arise because of words with multiple senses or POS tags . Many kinds of ambiguity are also structural.
Attachment Ambiguity Probabilities give us a way of choosing between possible parses. S S NP VP NP VP Pronoun Verb NP Pronoun VP PP I shot Det Nominal I Verb NP in my pajamas an Nominal PP shot Det Nominal Noun in my pajamas an Noun elephant elephant
Finding best parse → Pick the parse with the highest probability. n ˆ Y T ( S ) = P ( T | S ) = argmax P ( T , S ) = P ( RHS i | LHS i ) Ts . t . S = yield ( T ) i = 1 S S VP VP Verb NP Verb NP NP Book Det Nominal Book Det Nominal Nominal the Nominal Noun the Noun Noun Noun flight P(T,S) = 2.2 * 10 -6 P(T,S) = 6.1 * 10 -7 dinner flight 16 dinner
Constituents have heads S(dumped) NP(workers) VP(dumped) NNS(workers) VBD(dumped) NP(sacks) PP(into) NNS(sacks) P NP(bin) workers dumped DT(a) NN(bin) sacks into a bin
Dependency Parsing JURAFSKY AND MARTIN CHAPTER 15
Dependency Grammars Dependency grammars depict the syntactic structure of sentences solely in terms of the words in a sentence and an associated set of directed head-dependent grammatical relations that hold among these words.
Dependency – based Constituent– based
Advantages of dependencies Ø Dependencies don’t have nodes corresponding to phrasal constituents. Instead they directly encode information that is often buried in phrase structure parses. Ø Dependency grammars are better able deal with languages that have a relatively free word order. Ø Dependency relations approximate semantic relationships between words and arguments, which is useful for many applications Ø coreference resolution Ø question answering Ø information extraction.
Dependency Formalism The dependency structures are directed graphs. G = (V, A) where V is a set of vertices and A is a set of ordered pairs of vertices (or directed arcs ). Each arc points from the head to a dependent Dependent Head Directed arcs can also be labeled with the grammatical relation that holds between the head and a dependent .
Dependency Trees Other common constraints are that dependency structure must be connected , have a designated root node, and be acyclic or planar. These result in a rooted tree called a dependency tree . A dependency tree is a digraph where: 1. There is a single designated root node that has no incoming arcs 2. Each vertex has exactly one incoming arc (except the root node) 3. There is a unique path from the root node to each vertex in V This mean that each word in the sentence has exactly one head. Head Dependent
Dependency Relations In addition having directed arcs point from the head to the dependent, arc can be labeled with the type of grammatical function involved between the words • nsubj and dobj identify the subject and direct object of the verb cancelled • nmod, det and case relations denote modifiers of the nouns flights and Houston.
Dependency Relations 26 CS 272: STATISTICAL NLP (WINTER 2019)
Dependency Relations Relation Examples with head and dependent United canceled the flight. NSUBJ United diverted the flight to Reno. DOBJ We booked her the flight to Miami. IOBJ We took the morning flight. NMOD Book the cheapest flight. AMOD JetBlue canceled 1000 flights. NUMMOD United , a unit of UAL, matched the fares. APPOS The flight was canceled. DET We flew to Denver and drove to Steamboat. CONJ We flew to Denver and drove to Steamboat. CC Book the flight through Houston. CASE
Projective vs Non-projective 29 CS 272: STATISTICAL NLP (WINTER 2019)
Dependency Treebanks Dependency Treebanks are typically created by the following methods: 1. Having human annotators build dependency structures directly 2. Using an automatic parser and then employing human annotators to correct the output 3. Automatically transforming phrase-structure treebanks into dependency structure treebanks Directly annotated dependency treebanks have been often created for morphologically rich languages such as Czech (Prague Dependency Treebank), Hindi and Finnish.
S NP-SBJ VP NNP MD VP Vinken will VB NP PP-CLR NP-TMP join DT NN IN NP NNP CD the board as DT JJ NN Nov 29 a nonexecutive director
S(join) NP-SBJ(Vinken) VP(join) NNP MD VP(join) Vinken will VB NP(board) PP-CLR(director) NP-TMP(29) join DT NN IN NP(director) NNP CD the board as DT JJ NN Nov 29 a nonexecutive director
Recommend
More recommend
