Interset: Reusable Tagset Conversion Daniel Zeman, Rudolf Rosa - PowerPoint PPT Presentation

Interset: Reusable Tagset Conversion Daniel Zeman, Rudolf Rosa March 20, 2020 NPFL120 Multilingual Natural Language Processing Charles University Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless otherwise stated

Part-of-Speech Tagset Conversion Interset: Reusable Tagset Conversion 1/32 • See also NPFL094 (Computational Morphology and Syntax) in Winter • There: focus on linguistic diversity • Here: focus on • Technical aspects • Difgerent expressivity • Difgerent granularity

Interset: Reusable Tagset Conversion Why Convert Tags? 2/32 • For a tool that uses tags (parser) • The meaning of the tags is signifjcant (they are not just strings) • Or the tool has been trained on a particular tagset • For a linguist who works with corpora • Reduce need to learn new tags

• Conversion tailored to a pair of tagsets • Reusable “interlingua” (Interset, Universal Dependencies) • Look at source tags + words • Look at source tags + words + context How to Convert Tags? Interset: Reusable Tagset Conversion 3/32 • Look at source tags only

• Reusable “interlingua” (Interset, Universal Dependencies) • Look at source tags + words • Look at source tags + words + context How to Convert Tags? Interset: Reusable Tagset Conversion 3/32 • Look at source tags only • Conversion tailored to a pair of tagsets

• Look at source tags + words • Look at source tags + words + context How to Convert Tags? Interset: Reusable Tagset Conversion 3/32 • Look at source tags only • Conversion tailored to a pair of tagsets • Reusable “interlingua” (Interset, Universal Dependencies)

• Look at source tags + words + context How to Convert Tags? Interset: Reusable Tagset Conversion 3/32 • Look at source tags only • Conversion tailored to a pair of tagsets • Reusable “interlingua” (Interset, Universal Dependencies) • Look at source tags + words

Interset: Reusable Tagset Conversion How to Convert Tags? 3/32 • Look at source tags only • Conversion tailored to a pair of tagsets • Reusable “interlingua” (Interset, Universal Dependencies) • Look at source tags + words • Look at source tags + words + context

• IIIT Hyderabad: all Indian languages • Indo-Aryan • Dravidian • English! • Gold Ontology • Defjnes linguistic terms • The same term may denote difgerent things in difgerent languages • Interset, Google UPOS, Universal Dependencies • Papers claiming that universal tagset does not exist Related Work Interset: Reusable Tagset Conversion 4/32 • EAGLES, PAROLE, MULTEXT • Rather wanted to standardize tags • Not to work with the tags that are already there • Very euro-centric

• Gold Ontology • Defjnes linguistic terms • The same term may denote difgerent things in difgerent languages • Interset, Google UPOS, Universal Dependencies • Papers claiming that universal tagset does not exist Related Work Interset: Reusable Tagset Conversion 4/32 • EAGLES, PAROLE, MULTEXT • Rather wanted to standardize tags • Not to work with the tags that are already there • Very euro-centric • IIIT Hyderabad: all Indian languages • Indo-Aryan • Dravidian • English!

• Interset, Google UPOS, Universal Dependencies • Papers claiming that universal tagset does not exist Related Work Interset: Reusable Tagset Conversion 4/32 • EAGLES, PAROLE, MULTEXT • Rather wanted to standardize tags • Not to work with the tags that are already there • Very euro-centric • IIIT Hyderabad: all Indian languages • Indo-Aryan • Dravidian • English! • Gold Ontology • Defjnes linguistic terms • The same term may denote difgerent things in difgerent languages

• Papers claiming that universal tagset does not exist Related Work Interset: Reusable Tagset Conversion 4/32 • EAGLES, PAROLE, MULTEXT • Rather wanted to standardize tags • Not to work with the tags that are already there • Very euro-centric • IIIT Hyderabad: all Indian languages • Indo-Aryan • Dravidian • English! • Gold Ontology • Defjnes linguistic terms • The same term may denote difgerent things in difgerent languages • Interset, Google UPOS, Universal Dependencies

Interset: Reusable Tagset Conversion Related Work 4/32 • EAGLES, PAROLE, MULTEXT • Rather wanted to standardize tags • Not to work with the tags that are already there • Very euro-centric • IIIT Hyderabad: all Indian languages • Indo-Aryan • Dravidian • English! • Gold Ontology • Defjnes linguistic terms • The same term may denote difgerent things in difgerent languages • Interset, Google UPOS, Universal Dependencies • Papers claiming that universal tagset does not exist

Prague Tags for Czech v Interset: Reusable Tagset Conversion , Z:------------- noor X@------------- ejhle II------------- jen TT------------- aby J,-X---3------- RR--6---------- NNMS1-----A---- nejméně Dg-------3A---- jsem VB-S---1P-AA--- stě ClXP3---------2 jejímuž P1ZS3FS3------- následující AGFS3-----A---- Josef 5/32

Prague Tags for Czech R6 Interset: Reusable Tagset Conversion ZIP Z:------------- NOMORPH X@------------- I II------------- T TT------------- JVX3 J,-X---3------- RR--6---------- NNMS1-----A---- DG3A Dg-------3A---- VPS1A VB-S---1P-AA--- CGXP3-2 ClXP3---------2 PSEFSZS3 P1ZS3FS3------- AVGFS3A AGFS3-----A---- NMS1A 6/32

Prague Tags for CoNLL 2006 Shared Task R R Cas=6 Interset: Reusable Tagset Conversion Z : _ Z:------------- X @ _ X@------------- I I _ II------------- T T _ TT------------- J , Num=X|Per=3 J,-X---3------- RR--6---------- NNMS1-----A---- D g Gra=3|Neg=A Dg-------3A---- V B Num=S|Per=1|Ten=P… VB-S---1P-AA--- C 1 Gen=X|Num=P|Cas=3… ClXP3---------2 P 1 Gen=Z|Num=S|Cas=3… P1ZS3FS3------- A G Gen=F|Num=S|Cas=3… AGFS3-----A---- N N Gen=M|Num=S|Cas=1… 7/32

Multext East Sps1 Interset: Reusable Tagset Conversion Z:------------- X X@------------- I II------------- Q TT------------- Css3 J,-X---3------- RR--6---------- NNMS1-----A---- Rgs Dg-------3A---- Vmip1smanyn VB-S---1P-AA--- Mcmn3y ClXP3---------2 Pr3mdsfnayn P1ZS3FS3------- Afpfsd AGFS3-----A---- Ncmsny 8/32

Majka Tagset from Brno RR--6---------- Interset: Reusable Tagset Conversion Z:------------- X@------------- k0 II------------- k9 TT------------- k8p3xS J,-X---3------- k7c6 k6d3eAxD NNMS1-----A---- Dg-------3A---- k5gXnSp1mIaIeA VB-S---1P-AA--- k4gXnPc3xC ClXP3---------2 k3gUnSc3p3hFxR P1ZS3FS3------- k2gFnSc3eA AGFS3-----A---- k1gMnSc1eA 9/32

Penn Treebank Tags for English CC CD DT EX FW IN JJ JJR JJS LS MD NN NNS NNP NNPS PDT POS PRP PRP$ RB RBR RBS RP SYM TO UH VB VBD VBG VBN VBP VBZ WDT WP WP$ WRB . , : $ # `` '' -LRB- -RRB- Interset: Reusable Tagset Conversion 10/32 • EX = existential there • FW = foreign word • IN = preposition or subordinating conjunction • TO = to • UH = interjection…

Brown Corpus Tags for English ABL ABN ABX AP AP$ AP+AP AT BE BED BED* BEDZ BEDZ* BEG BEM BEM* BEN BER BER* BEZ BEZ* CC CD CD$ CS DO DO* DO+PPSS DOD DOD* DOZ DOZ* DT DT$ DT+BEZ DT+MD DTI DTS DTS+BEZ DTX EX EX+BEZ EX+HVD EX+HVZ EX+MD FW-* FW-AT FW-AT+NN FW-BE FW-BER FW-BEZ FW-CC FW-CD FW-CS FW-DT FW-DT+BEZ FW-DTS FW-HV FW-IN FW-IN+AT FW-IN+NN FW-IN+NP FW-JJ FW-JJR FW-JJT FW-NN FW-NN$ FW-NNS FW-NP FW-NPS FW-NR FW-OD FW-PN FW-PP$ FW-PPL FW-PPL+VBZ FW-PPO FW-PPO+IN FW-PPS FW-PPSS FW-PPSS+HV FW-QL FW-RB FW-RB+CC FW-TO+VB FW-UH FW-VB … Interset: Reusable Tagset Conversion 11/32

SynTagRus Tags for Russian Dg-------2A---- Interset: Reusable Tagset Conversion II------------- INTJ TT------------- PART Jˆ------------- CONJ RR--6---------- PR 12/32 VB-P---3P-AA--- ClXX4---------- AAXP1----1A---- PSXXXXP3------- NNMS1-----A---- S ЕД МУЖ ИМ S МН РОД ОД A МН ИМ NUM ВИН V НЕСОВ ИЗЪЯВ НЕПРОШ МН 3-Л ADV СРАВ

Stuttgart-Tübingen Tagset for German ADJA ADJD ADV APPR APPRART APPO APZR ART CARD FM ITJ KOUI KOUS KON KOKOM NN NE PDS PDAT PIS PIAT PIDAT PPER PPOSS PPOSAT PRELS PRELAT PRF PWS PWAT PWAV PAV PTKZU PTKNEG PTKVZ PTKANT PTKA TRUNC VVFIN VVIMP VVINF VVIZU VVPP VAFIN VAIMP VAINF VAPP VMFIN VMINF VMPP XY $, $. $( Interset: Reusable Tagset Conversion 13/32 • Like in Penn TB: parts of speech only, but slightly more fjne-grained • No morphology (German has gender, number, case, degree, person…) • “Substantive” vs. “attributive” pronouns ( S vs. AT ) • Adposition = Präposition, Postposition, Zirkumposition

Anncorra from IIIT Hyderabad NN NST NNP PRP DEM VM VAUX JJ RB PSP RP CC WQ QF QC QO CL INTF INJ NEG UT SYM *C RDP ECH UNK Interset: Reusable Tagset Conversion 14/32 • Ambition: common tagset for all Indian languages (IE and Dravidian!) • No morphology (although the languages are rich on morphology) • Hierarchical tagset, morphology can be added at the end • And they “do not want to decrease tagging accuracy” (!) • Cloned from Penn tagset and modifjed • New categories, e.g. postposition, “quotative” • Removed traces of morphology, e.g. plural, comparative, superlative