Hacking a Way Through the Twitter Language Jungle: Syntactic Annotation, Tagging, and Parsing of English Tweets - 8 2 9 5 3 8 / l m m o t h c . . r x e i f p a a r p g l l l e a e w f / - e / l : g p n t t u h j Nathan Schneider • NLPIT , Rotterdam • June 23, 2015
Edited Text 2
Conversational Web Text #jesuischarlie <3333 wut! u da man! *fist pump* 3
ROBUSTNESS POS NER syntactic parsing semantic parsing RICHNESS 4
representation automation annotation 5
representation automation annotation 6
Outline • Twitter POS tagging • Twitter dependency parsing • What’s next? 7
Twitter non-standard spellings multi-word abbreviations hashtags Also: at-mentions, URLs, emoticons, symbols, typos, etc. 8
Twitter POS • Part-of-speech tagging for Twitter: annotation, features, and experiments. Kevin Gimpel, Nathan Schneider, Brendan O’Connor, Dipanjan Das, Daniel Mills, Jacob Eisenstein, Michael Heilman, Dani Yogatama, Jeffrey Flanigan, and Noah A. Smith. ACL-HLT 2011. • Improved part-of-speech tagging for online conversational text with word clusters. Olutobi Owoputi, Brendan O’Connor, Chris Dyer, Kevin Gimpel, Nathan Schneider, and Noah A. Smith. NAACL-HLT 2013. 9
Twitter POS: Representation lexical & punctuation common noun determiner proper noun preposition pronoun verb particle verb coordinating conjunction adjective numeral adverb interjection punctuation predeterminer / existential there complex nominal+possessive ( his, books’ ) proper noun+possessive ( Mary’s book ) nominal+verbal ( ur , ima ) proper noun+verbal ( Mary’s happy ) existential+verbal ( there’s ) 10
Twitter POS: Representation Twitter-specific hashtag #mcconnelling username @justinbieber URL/email cnn.com bob@bob.com emoticon :-) \o/ Twitter discourse marker RT <— other ily mfw ™ 11
Twitter POS: Annotation • 17 annotators 13
Twitter POS: Annotation 93.0 92.2 90.5 88.0 85.5 83.0 83.0 83.0 83.0 Inter-annotator agreement 15
Twitter POS: Automation 93.0 93.0 92.2 92.2 incl. special regexes, distributional 90.5 90.5 similarity, phonetic 89.4 normalization, tag dictionary 88.0 88.0 85.9 85.5 85.5 83.4 83.0 83.0 83.0 83.0 83.0 Stanford Our CRF, Our CRF, Inter-annotator tagger base features all features agreement 16
Can we do better? • No explicit annotation guidelines document → some conventions unclear. • Tokenization difficult due to creative emoticons. :~P \o/ • Accuracy still lags on rare/OOV words. • CRF is too slow to tag huge volumes of text. 17
Annotation Conventions • Jesse & the Rippers the California Chamber of Commerce All and only nouns within proper names ‣ tagged as proper noun • (1) this wind is serious (2) i just orgasmed over this (3) You should know, that if you come any closer … (1): determiner, (2): pronoun, ‣ (3): preposition/subordinator Gimpel et al. annotations were ‣ inconsistent, so we corrected them 18
Annotation Conventions • RT @anonuser : Tonight’s memorial for Lucas Ransom starts at 8:00 p.m. and is being held at the open space at the corner of Del Pla ... Proper noun (truncated, but we can tell ‣ from context) • (1) I need to go home man . (2) * Bbm yawn face * Man that #napflow felt so refreshing . (1): noun, (2): interjection ‣ 19
Improved Tokenizer • Rule-based , with regular expressions for faces, etc. :*O -_- <3333 • Also better URL patterns: about.me 20
Word Clusters • Brown clusters help smooth over lexicon to better accommodate rare/OOV words • We trained 1000 clusters on 56M English tweets (847M tokens) spread over 4 years 21
Word Clusters laughter Binary path Top words (by frequency) A1 111010100010 lmao lmfao lmaoo lmaooo hahahahaha lool ctfu rofl loool lmfaoo lmfaooo lmaoooo lmbo lololol A2 111010100011 haha hahaha hehe hahahaha hahah aha hehehe ahaha hah hahahah kk hahaa ahah A3 111010100100 yes yep yup nope yess yesss yessss ofcourse yeap likewise yepp yesh yw yuup yus A4 111010100101 yeah yea nah naw yeahh nooo yeh noo noooo yeaa ikr nvm yeahhh nahh nooooo A5 11101011011100 smh jk #fail #random #fact smfh #smh #winning #realtalk smdh #dead #justsaying B 011101011 u yu yuh yhu uu yuu yew y0u yuhh youh yhuu iget yoy yooh yuo yue juu dya youz yyou C 11100101111001 w fo fa fr fro ov fer fir whit abou aft serie fore fah fuh w/her w/that fron isn agains D 111101011000 facebook fb itunes myspace skype ebay tumblr bbm flickr aim msn netflix pandora E1 0011001 tryna gon finna bouta trynna boutta gne fina gonn tryina fenna qone trynaa qon E2 0011000 gonna gunna gona gna guna gnna ganna qonna gonnna gana qunna gonne goona F 0110110111 soo sooo soooo sooooo soooooo sooooooo soooooooo sooooooooo soooooooooo G1 11101011001010 ;) :p :-) xd ;-) ;d (; :3 ;p =p :-p =)) ;] xdd #gno xddd >:) ;-p >:d 8-) ;-d G2 11101011001011 :) (: =) :)) :] :’) =] ^_^ :))) ^.^ [: ;)) ((: ^__^ (= ^-^ :)))) G3 1110101100111 :( :/ -_- -.- :-( :’( d: :| :s -__- =( =/ >.< -___- :-/ </3 :\ -____- ;( /: :(( >_< =[ :[ #fml G4 111010110001 <3 xoxo <33 xo <333 #love s2 <URL-twitition.com> #neversaynever <3333 hearts/love symbols 23
Word Clusters laughter Binary path Top words (by frequency) A1 111010100010 lmao lmfao lmaoo lmaooo hahahahaha lool ctfu rofl loool lmfaoo lmfaooo lmaoooo lmbo lololol A2 111010100011 haha hahaha hehe hahahaha hahah aha hehehe ahaha hah hahahah kk hahaa ahah A3 111010100100 yes yep yup nope yess yesss yessss ofcourse yeap likewise yepp yesh yw yuup yus A4 111010100101 yeah yea nah naw yeahh nooo yeh noo noooo yeaa ikr nvm yeahhh nahh nooooo A5 11101011011100 smh jk #fail #random #fact smfh #smh #winning #realtalk smdh #dead #justsaying B 011101011 u yu yuh yhu uu yuu yew y0u yuhh youh yhuu iget yoy yooh yuo yue juu dya youz yyou C 11100101111001 w fo fa fr fro ov fer fir whit abou aft serie fore fah fuh w/her w/that fron isn agains D 111101011000 facebook fb itunes myspace skype ebay tumblr bbm flickr aim msn netflix pandora E1 0011001 tryna gon finna bouta trynna boutta gne fina gonn tryina fenna qone trynaa qon E2 0011000 gonna gunna gona gna guna gnna ganna qonna gonnna gana qunna gonne goona F 0110110111 soo sooo soooo sooooo soooooo sooooooo soooooooo sooooooooo soooooooooo G1 11101011001010 ;) :p :-) xd ;-) ;d (; :3 ;p =p :-p =)) ;] xdd #gno xddd >:) ;-p >:d 8-) ;-d G2 11101011001011 :) (: =) :)) :] :’) =] ^_^ :))) ^.^ [: ;)) ((: ^__^ (= ^-^ :)))) G3 1110101100111 :( :/ -_- -.- :-( :’( d: :| :s -__- =( =/ >.< -___- :-/ </3 :\ -____- ;( /: :(( >_< =[ :[ #fml G4 111010110001 <3 xoxo <33 xo <333 #love s2 <URL-twitition.com> #neversaynever <3333 hearts/love symbols + faces 24
Word Clusters laughter + interjections Binary path Top words (by frequency) A1 111010100010 lmao lmfao lmaoo lmaooo hahahahaha lool ctfu rofl loool lmfaoo lmfaooo lmaoooo lmbo lololol A2 111010100011 haha hahaha hehe hahahaha hahah aha hehehe ahaha hah hahahah kk hahaa ahah A3 111010100100 yes yep yup nope yess yesss yessss ofcourse yeap likewise yepp yesh yw yuup yus A4 111010100101 yeah yea nah naw yeahh nooo yeh noo noooo yeaa ikr nvm yeahhh nahh nooooo A5 11101011011100 smh jk #fail #random #fact smfh #smh #winning #realtalk smdh #dead #justsaying B 011101011 u yu yuh yhu uu yuu yew y0u yuhh youh yhuu iget yoy yooh yuo yue juu dya youz yyou C 11100101111001 w fo fa fr fro ov fer fir whit abou aft serie fore fah fuh w/her w/that fron isn agains D 111101011000 facebook fb itunes myspace skype ebay tumblr bbm flickr aim msn netflix pandora E1 0011001 tryna gon finna bouta trynna boutta gne fina gonn tryina fenna qone trynaa qon E2 0011000 gonna gunna gona gna guna gnna ganna qonna gonnna gana qunna gonne goona F 0110110111 soo sooo soooo sooooo soooooo sooooooo soooooooo sooooooooo soooooooooo G1 11101011001010 ;) :p :-) xd ;-) ;d (; :3 ;p =p :-p =)) ;] xdd #gno xddd >:) ;-p >:d 8-) ;-d G2 11101011001011 :) (: =) :)) :] :’) =] ^_^ :))) ^.^ [: ;)) ((: ^__^ (= ^-^ :)))) G3 1110101100111 :( :/ -_- -.- :-( :’( d: :| :s -__- =( =/ >.< -___- :-/ </3 :\ -____- ;( /: :(( >_< =[ :[ #fml G4 111010110001 <3 xoxo <33 xo <333 #love s2 <URL-twitition.com> #neversaynever <3333 hearts/love symbols + faces 25
Recommend
More recommend
