Syntactic Processing: Parts-of-Speech Tagging CSE354 - Spring 2020

Task ● Syntactic Processing ● Machine learning: h o w ? Parts-of-Speech Tagging ○ Logistic regression

Parts-of-Speech Open Class: Nouns, Verbs, Adjectives, Adverbs

Parts-of-Speech Open Class: Nouns, Verbs, Adjectives, Adverbs Function words: Determiners, conjunctions, pronouns, prepositions

Parts-of-Speech: The Penn Treebank Tagset

Parts-of-Speech: Social Media Tagset ( Gimpel et al., 2010)

POS Tagging: Applications ● Resolving ambiguity (speech: “lead”) ● Shallow searching: find noun phrases ● Speed up parsing ● Use as feature (or in place of word)

POS Tagging: Applications ● Resolving ambiguity (speech: “lead”) ● Shallow searching: find noun phrases ● Speed up parsing ● Use as feature (or in place of word) For this course: ● An introduction to language-based classification (logistic regression) ● Understand what modern deep learning methods are dealing with implicitly.

Window-based POS Tagging The book looks brief so I am happy . ?

Window-based POS Tagging The book looks brief so I am happy . D

Window-based POS Tagging The book looks brief so I am happy . D N

Window-based POS Tagging The book looks brief so I am happy . D N ?

Window-based POS Tagging The book looks brief so I am happy . D N V

Window-based POS Tagging The book looks brief so I am happy . D N V A

Window-based POS Tagging The book looks brief so I am happy . D N V ?

Window-based POS Tagging window size of 3 The book looks brief so I am happy . D N V ?

Window-based POS Tagging window size of 3 The book looks brief so I am happy . P(pos i = ‘N’|word i = “brief”) = 0.3 D N V ?

Window-based POS Tagging window size of 3 The book looks brief so I am happy . P(pos i = ‘N’|word i = “brief”) = 0.3 D N V ? P(pos i = ‘V’|word i = “brief”) = 0.4 P(pos i = ‘A’|word i = “brief”) = 0.3

Window-based POS Tagging window size of 3 The book looks brief so I am happy . P(p i =‘N’|w i =brief) = .30 D N V ? P(p i =‘V’|w i =brief) = .40 P(p i =‘A’|w i =brief) = .30

Window-based POS Tagging window size of 3 The book looks brief so I am happy . P(p i =‘N’|w i =brief,w i-1 =looks,w i+1 =so) = ?? D N V ? P(p i =‘V’|w i =brief,w i-1 =looks,w i+1 =so) = ?? P(p i =‘A’|w i =brief,w i-1 =looks,w i+1 =so) = ??

Window-based POS Tagging window size ideal result of 3 The book looks brief so I am happy . P(p i =‘N’|w i =brief,w i-1 =looks,w i+1 =so) = .005 D N V ? P(p i =‘V’|w i =brief,w i-1 =looks,w i+1 =so) = .005 P(p i =‘A’|w i =brief,w i-1 =looks,w i+1 =so) = .99

Window-based POS Tagging More likely, because we window size haven’t seen of 3 this context before. The book looks brief so I am happy . P(p i =‘N’|w i =brief,w i-1 =looks,w i+1 =so) = .3 D N V ? P(p i =‘V’|w i =brief,w i-1 =looks,w i+1 =so) = .4 P(p i =‘A’|w i =brief,w i-1 =looks,w i+1 =so) = .3

Sequential Model window size of 3 The book looks brief so I am happy . P(p i =‘N’|w i =brief,w i-1 =looks,w i+1 =so) = .3 D N V ? P(p i =‘V’|w i =brief,w i-1 =looks,w i+1 =so) = .4 P(p i =‘A’|w i =brief,w i-1 =looks,w i+1 =so) = .3 sequence order of 1

Sequential Model window size of 3 The book looks brief so I am happy . P(p i =‘N’|p i-1 =V) = .4 D N V ? P(p i =‘V’|p i-1 =V) = .10 P(p i =‘A’|p i-1 =V) = .4 sequence order of 1

Sequential Model window size of 3 The book looks brief so I am happy . P(p i =‘N’|p i-1 =V,w i =brief) = .3 D N V ? P(p i =‘V’|p i-1 =V,w i =brief) = .05 P(p i =‘A’|p i-1 =V,w i =brief) = .65 sequence order of 1

Sequence modeling -- Tasks that in which a current label is dependent on previous labels within a sequence. More generally: tasks that can leverage the order of words. Most basic example: Language Modeling -- Predicting the next word given previous.

Syntactic Processing: Parts-of-Speech Tagging CSE354 - Spring 2020 - PowerPoint PPT Presentation

Syntactic Processing: Parts-of-Speech Tagging CSE354 - Spring 2020 Task Syntactic Processing Machine learning: h o w ? Parts-of-Speech Tagging Logistic regression Parts-of-Speech Open Class: Nouns, Verbs, Adjectives,

Natural Language Processing Part of Speech Tagging Dan Klein UC Berkeley 1 2 Parts of

Natural Language Processing Parts of Speech Part of Speech Tagging Dan Klein UC

POS Tagging & Sequence Labeling Tasks CMSC 470 Marine Carpuat Parts of Speech

Part-of-Speech Tagging Part-of-Speech Tagging Berlin Chen 2003 References: 1. Speech and

Part-of-Speech Tagging Part-of-Speech Tagging Berlin Chen 2005 References: 1. Speech and

Part-of-Speech Tagging COSI 114 Computational Linguistics James Pustejovsky February 24,

Natural Language Processing and Information Retrieval Part of Speech Tagging and Named Entity

CSE 517 Natural Language Processing Winter 2017 Parts of Speech Yejin Choi [Slides adapted

CSP 517 Natural Language Processing Winter 2015 Parts of Speech Yejin Choi [Slides adapted

Part-of-Speech Tagging COSI 114 Computational Linguistics James Pustejovsky March 17, 2017

POS tagging CMSC 723 / LING 723 / INST 725 Marine Carpuat Parts of Speech Equivalence

Logistic Regression and POS Tagging CSE392 - Spring 2019 Special Topic in CS Task

Parts-of-Speech (English) Statistical NLP One basic kind of linguistic structure: syntactic

Part-of-speech tagging A simple but useful form of

Statistical Natural Language Processing Dr. Besnik Fetahu Overview POS tagging

Part-of-Speech Tagging Jimmy Lin Jimmy Lin The iSchool University of Maryland Wednesday,

Parts of Speech More Fine-Grained Classes More

Statistical Natural Language Processing Part of speech tagging ar ltekin University of

Natural Language Processing Part of Speech Tagging and Named Entity Recognition Alessandro

Natural Language Processing Parsing III Dan Klein UC Berkeley 1 Unsupervised Tagging 2

Part Of Speech (POS) Tagging Based on Foundations of Statistical NLP by C. Manning & H.

Lecture 5: Part-of-Speech Tagging Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center

Lecture 10: Part-of-Speech Tagging Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center

Speech Processing for Speech Processing for Unwritten Languages Unwritten Languages Alan W