Ultra-Fine Entity Typing Eunsol Choi, Omer Levy, Yejin Choi, Luke Zettlemoyer ACL 2018 � 1
Entity Typing 1. Bill robbed John, and he was arrested shortly afterwards. 2. Nvidia hands out Titan V for free to AI researchers. � 2
Entity Typing 1. Bill robbed John, and he was arrested shortly afterwards. 2. Nvidia hands out Titan V for free to AI researchers. � 3
Entity Typing PER PER 1. Bill robbed John, and he was arrested shortly afterwards. 2. Nvidia hands out Titan V for free to AI researchers. ORG OBJ • Information extraction [Ling 12, YY17] • Coreference resolution [Durrett 14] • Entity linking [Durrett 14, Raiman 18] • Question answering [Yavuz 16] � 4
Scaling Up Entity Typing: Mention Coverage PER PER 1. Bill robbed John, and he was arrested shortly afterwards. 2. Nvidia hands out Titan V for free to AI researchers. ORG OBJ Prior Work This Work Bill Nvidia Bill Nvidia Titan V John Titan V John He AI researchers
Scaling Up Entity Typing: Mention Coverage PER PER 1. Bill robbed John, and he was arrested shortly afterwards. 2. Nvidia hands out Titan V for free to AI researchers. ORG OBJ Challenge 1 : Reasoning over diverse, challenging mention strings
Scaling Up Entity Typing: Type Coverage PER PER 1. Bill robbed John, and he was arrested shortly afterwards. 2. Nvidia hands out Titan V for free to AI researchers. ORG OBJ
Scaling Up Entity Typing: Type Coverage PER, Victim PER, Criminal PER, Criminal 1. Bill robbed John, and he was arrested shortly afterwards. 2. Nvidia hands out Titan V for free to AI researchers. ORG, Company OBJ, Product, Electronics PER, Researcher, Professional Any frequent nouns from dictionary is allowed as a type (10K vocabulary)
Scaling Up Entity Typing: Challenge 2 : Type Coverage Very large label space PER, criminal PER, victimPER, criminal 1. 1. Bill robbed John. He was arrested shortly afterwards. Bill robbed John. He was arrested shortly afterwards. 12/15/2017 https://homes.cs.washington.edu/~eunsol/finetype_visualization/ours_index.html ve_ ie an nn ag en ria fire era xt_ b_s rai ate ot na fer ric nk de gre gr thi eke ativ lua ent per orm em staf ar pla dic ficu drug il_lpio er atio for nce ex cul oy alu uild ow olit star ha ere ou dus ec mic_ ab eit inne dm vilia cili al_ serv opic lop wm mm blin stig ive p_a vin ecta ncti band me oup dic ufac mina rga orma ale sp ud mig ric marke demi on co uca data ounc nam part ula mmo sta spo pita ap tu fes nom spl pre lid ship unt gisla fenda vern ma aract ecord urn aim otio eat ess ne ud risi name twa roo ecti gme uratio tructur tdo war owe eat elativ atis cto age ter ball_p re nt pinio risone 2. 2. Nvidia hands out Titan V for free to AI researchers. Nvidia hands out Titan V for free to AI researchers. ove po ilo hoic sast gine uipm point pai ontr ministra ontra nda dca ide policy ateme 12/15/2017 https://homes.cs.washington.edu/~eunsol/finetype_visualization/figer_index.html dit ain rvicem book _tra professiona ar_ an wspa aid 12/15/2017 https://homes.cs.washington.edu/~eunsol/finetype_visualization/onto_index.html rts squad erie oxe bill elin maste elebrit den show seb meetin liat enterprise nc mis sum fe al ocume visio citizen iva uis r_ve actor aso ian licem mou ta _sc abit mp ocie ome roo victim year crip alit se oble /country rren state ecia tne ilwa eac land ist inist ave ontestan lec man atherin performe home /building team los mst ndin tra /city battle slat /company eporte /company up achi king musicia ec let ateri em no title apo mba eig politician goa hang um uti etwo hiev ym umb crimina tem giti case inessm cor ea nec olfe onv tuatio tru business coach gre /event esu ORG, company OBJ, product PER, researcher, professional /time bstan sh ou rrito ctio ess adc oubl ou athe company uni an nce leader male eal tain rtain s_p ell two /location player ace ual nsequen raine action resenta c_g ac sen ecut es ctio wo er wife pro rum ran album co nn song er tin tfo hurc ystem lica music ffic agu ruc rde kpl time las stm mmunica ceho object ncip soci bra pokesperso ma tertain astro islat child utho dre ov stitu pas erv urre ca writer adult har gw prize ho aceme agen car cide cientis /person /organization oar pta ers wgi work city apit erm titu hea toc mpaig singe soldie tary usa activity region olu lam /other /person elo udg re men urg og eath space dar_m erta lan semb nes zen em ders win era party int rga urnal ssu ille government m_s girl law us_l son ndin • Any frequent nouns from dictionary is allowed as a type (10K vocabulary) worker pa blicati adiu idea person organization spu cuss vem ws play mpetit ini tor _se oss et vesto alk artist emb appeni mo bby stom /art food jet nanc end ont or office cov arn nce rugg roduc oth concept clo tesm ce presiden country fere ion rat ese /location oje ge on friend ect /organization army ache em ote cash ailur eekd llag ons ea ational_ins itten_wo rcr cus fall plom reem easu boa de wo place movie up unicipal ownshi eta pe /legal oun ctio ghte our olleg duc pokesma position ng n_c upa rm docto sum event istric rim area _of_ nment_a sca /country ish rm _ag chola conflic nsi pous dy_p fact oya od ay atm oces rrori mea rrat ma ate date vote _in rganism otia ect agency tuden ns osi evic woman al_s on pos er_p location se quid group olla athlete gs_ an hav ui ruler attac ry_se gua ugh game as nci ormati pro ad erw military ge authority speake ree por ours m oo report peec riti head ctres choo opos crip art ma ape ffo nditio administration nalys dvis oad expert ubl _m matte al_ adm voca official spe ourc role fund directo nation uildin female gov ven aren ass of_ artm tem period avele visi sto institution ar ver film cip figh ced contes ens boy tatu ouri anima aw ndida rlin ha otb rule hing firm site es side mmu nesspe erc ers mploye rg orporatio oduc day pula act erv sh mmitt awye war match g_th reato eci grid an torm _p eatio ocial_grou olic rea sociati dividu https://homes.cs.washington.edu/~eunsol/finetype_visualization/figer_index.html 1/1 se ord exc xhi writin unit money ligio oth ecisio path ctu wne angem ogra ak bird onte nio e_e sport erati ut sw tifa etr ea du ace urn auc ppo plan res sw anc https://homes.cs.washington.edu/~eunsol/finetype_visualization/onto_index.html 1/1 essag llplay vers dea calizati ous ou club body an rn force ort atien plis ubjec rive mpo plan nd ffa rofi ketp cos fug ltu ady ehic sea ndma ppor act olitic arri belie na ctu no os amily own tion ad ote mp ress cce gotia com nis icip stig mpa new rticl clu tor ya gd ack ve pa sour of_e ran easo row an dic job len ric ial_ ucat ho ve ono nem gaz ua tra n_ proa emi ent stiv gat dish cin mm ron gd mpu e_n res gym ea ch et ngd on_ isk phic are ssi rie oph oun oda rde mu min cto wor lita ent rtn od ally mb an ra ine pe hw si sba tro med tis es https://homes.cs.washington.edu/~eunsol/finetype_visualization/ours_index.html 1/1
This Talk • Task: Ultra-Fine • Covers all entity mentions Entity Typing • Allows all concepts as types • Crowdsourcing ultra fine-grained typing data • New Data: • New source of distant supervision • Multitask loss for predicting ultra-fine types • New Results: • Sets state-of-the-art results on existing benchmark � 10
Outline • New Task: Ultra-Fine Entity Typing •Related Work • Crowdsourcing Ultra-Fine labels • Distant Supervision • Model and Experiments � 11
Fine grained NER He was elected over John McCain Person, Politician Pronominals Nominals Fine grained NER Named NER Entity coarse fine grained grained Type Ontology 12
Fine grained NER He was elected over John McCain Person, Politician FIGER [Ling 12] OntoNotes [Gillick 14] TypeNet [next talk] Ours Person, Politician 112 types 89 types 2K types 10K types 2 hierarchy level 3 hierarchy level 14 hierarchy level No hierarchy NER Fine grained NER coarse fine grained grained Type Ontology 13
Label Coverage Problem He was elected over [ John McCain ]. • In both, top 9 types covers over 80% of the Person, Politician evaluation data. • In OntoNotes, 52% of mentions was marked Named NER Fine grained NER as “Other”. Entity Paris Agreement Security Mortgages Oil coarse fine grained grained Type Ontology 14
Recommend
More recommend
