Language tagging using the new RFC 5646 Richard Ishida W3C - PowerPoint PPT Presentation

Oct 22, 2023 •198 likes •269 views

Making the World Wide Web worldwide! Language tagging using the new RFC 5646 Richard Ishida W3C Internationalization Activity Lead Language tags Before RFC 1766 / RFC 3066 language region en en-GB ISO 639 language codes en-scouse

Making the World Wide Web worldwide! Language tagging using the new RFC 5646 Richard Ishida W3C Internationalization Activity Lead
Language tags Before RFC 1766 / RFC 3066 language region en en-GB ♦ ISO 639 language codes en-scouse ♦ ISO 3166 country codes
Language tags Now BCP 47 (includes RFC 5646) language – script – region – variant – extension – private_use ♦ only language subtag required – use other az-Cyrl tags only when needed! zh-Hant-HK ♦ codes available only from new IANA Language Subtag Registry es-419 ♦ incorporates ISO and UN codes sl-IT-nedis ♦ over 7,000 codes
Language tags Extended language subtags (extlangs) BCP 47 (includes RFC 5646) language – script – region – variant – extension – private_use extended-language ar ar-aao aao kok ar-afb ms afb sw Better as… ar-arq arq uz ar-arz arz zh etc… etc… sgn
Language tags Macrolanguages, collections, etc. zh
Language tags Use ♦ Supported by a community of experts at the IETF ♦ Registry is regularly updated through a published process allowing submissions ♦ Language tags are mostly simple, but can be flexed to meet more complex needs ♦ Extension mechanism can provide mappings to other systems ♦ Accompanied by a language tag matching specification
Language tags Further reading ♦ BCP 47 http://www.rfc-editor.org/rfc/bcp/bcp47.txt ♦ IANA Language Subtag Registry http://www.iana.org/assignments/language-subtag-registry ♦ Subtag Lookup Tool http://rishida.net/utils/subtags/ ♦ Choosing a Language Tag http://www.w3.org/International/questions/qa-choosing-language-tags

Recommend

Statistical Natural Language Processing Dr. Besnik Fetahu Overview POS tagging

Statistical Natural Language Processing Dr. Besnik Fetahu Overview POS tagging Morphology Phrase structure and ambiguities Semantics POS tagging Group words of a language into classes which show similar syntactic behavior

795 views • 36 slides

IN4080 2020 FALL NATURAL LANGUAGE PROCESSING Jan Tore Lnning 2 Tagging and sequence

1 IN4080 2020 FALL NATURAL LANGUAGE PROCESSING Jan Tore Lnning 2 Tagging and sequence labeling Lecture 7, 28 Sept Today 3 Tagged text and tag sets Tagging as sequence labeling HMM-tagging Discriminative tagging

709 views • 51 slides

Part-of-Speech Tagging Part-of-Speech Tagging Berlin Chen 2003 References: 1. Speech and

Part-of-Speech Tagging Part-of-Speech Tagging Berlin Chen 2003 References: 1. Speech and Language Processing, chapter 8 2. Foundations of Statistical Natural Language Processing, chapter 10 1 Review Tagging (part-of-speech tagging)

671 views • 38 slides

Empirical Methods in Natural Language Processing Lecture 6 Tagging (II): Transformation-Based

Empirical Methods in Natural Language Processing Lecture 6 Tagging (II): Transformation-Based Learning and Maximum Entropy Models Philipp Koehn 24 January 2008 PK EMNLP 24 January 2008 1 Tagging as supervised learning Tagging is a

326 views • 9 slides

Natural Language Processing Parsing III Dan Klein UC Berkeley 1 Unsupervised Tagging 2

Natural Language Processing Parsing III Dan Klein UC Berkeley 1 Unsupervised Tagging 2 Unsupervised Tagging? AKA part of speech induction Task: Raw sentences in Tagged sentences out Obvious thing to do: Start

559 views • 39 slides

Empirical Methods in Natural Language Processing Lecture 8 Tagging (III): Maximum Entropy Models

Empirical Methods in Natural Language Processing Lecture 8 Tagging (III): Maximum Entropy Models Philipp Koehn 31 January 2008 PK EMNLP 31 January 2008 1 POS tagging tools Three commonly used, freely available tools for tagging: TnT

402 views • 9 slides

Part-of-Speech Tagging Part-of-Speech Tagging Berlin Chen 2005 References: 1. Speech and

Part-of-Speech Tagging Part-of-Speech Tagging Berlin Chen 2005 References: 1. Speech and Language Processing , chapter 8 2. Foundations of Statistical Natural Language Processing , chapter 10 NLP-Berlin Chen 1 Review Tagging

1k views • 54 slides

Statistical Natural Language Processing Part of speech tagging ar ltekin University of

Statistical Natural Language Processing Part of speech tagging ar ltekin University of Tbingen Seminar fr Sprachwissenschaft Summer Semester 2017 POS tags and tagsets POS tagging Summer Semester 2017 SfS / University of

378 views • 26 slides

Language Processing with Perl and Prolog Chapter 8: Part-of-Speech Tagging Using Stochastic

Language Technology Language Processing with Perl and Prolog Chapter 8: Part-of-Speech Tagging Using Stochastic Techniques Pierre Nugues Lund University Pierre.Nugues@cs.lth.se http://cs.lth.se/pierre_nugues/ Pierre Nugues Language

634 views • 9 slides

Morphological and Part-of-Speech Tagging of Historical Language Data: A Comparison Stefanie

Morphological and Part-of-Speech Tagging of Historical Language Data: A Comparison Stefanie Dipper Linguistics Department Ruhr-University Bochum 5 January, 2012 Workshop on Annotation of Corpora for Research in the Humanities Heidelberg

994 views • 60 slides

Natural Language Understanding Lecture 11: Unsupervised Part-of-Speech Tagging with Neural

Introduction Maximum Entropy Models as Emissions Embeddings as Emissions Natural Language Understanding Lecture 11: Unsupervised Part-of-Speech Tagging with Neural Networks Frank Keller School of Informatics University of Edinburgh

707 views • 29 slides

Natural Language Processing Part of Speech Tagging Dan Klein UC Berkeley 1 2 Parts of

Natural Language Processing Part of Speech Tagging Dan Klein UC Berkeley 1 2 Parts of Speech Parts of Speech (English) One basic kind of linguistic structure: syntactic word classes Open class (lexical) words Nouns

529 views • 42 slides

Natural Language Processing Part of Speech Tagging and Named Entity Recognition Alessandro

Natural Language Processing Part of Speech Tagging and Named Entity Recognition Alessandro Moschitti & Olga Uryupina Department of information and communication technology University of Trento Email: moschitti@disi.unitn.it

955 views • 76 slides

Natural Language Processing Parts of Speech Part of Speech Tagging Dan Klein UC

Natural Language Processing Parts of Speech Part of Speech Tagging Dan Klein UC Berkeley Parts of Speech (English) CC conjunction, coordinating and both but either or CD numeral, cardinal mid-1890 nine-thirty 0.5 one DT

231 views • 7 slides

Language Processing with Perl and Prolog Chapter 7: Part-of-Speech Tagging Using Rules Pierre

Language Technology Language Processing with Perl and Prolog Chapter 7: Part-of-Speech Tagging Using Rules Pierre Nugues Lund University Pierre.Nugues@cs.lth.se http://cs.lth.se/pierre_nugues/ Pierre Nugues Language Processing with Perl and

420 views • 25 slides

Natural Language Processing and Information Retrieval Part of Speech Tagging and Named Entity

Natural Language Processing and Information Retrieval Part of Speech Tagging and Named Entity Recognition Alessandro Moschitti Department of information and communication technology University of Trento Email: moschitti@dit.unitn.it Parts of

950 views • 78 slides

Forewords Tagging in a nutshell Sources Slides inspired by M. Rajman and J.-C. Chappelier,

Tagging in a nutshell Tagging in a nutshell Forewords Tagging in a nutshell Sources Slides inspired by M. Rajman and J.-C. Chappelier, EPFL Vincent Claveau Vocabulary tagging, French: etiquetage IRISA - CNRS tag, Fr.

441 views • 7 slides

Traffic UTM Tagging AdWords WebMaster Tools UTM TAGGING Where does my traffic come from? UTM

Traffic UTM Tagging AdWords WebMaster Tools UTM TAGGING Where does my traffic come from? UTM TAGGING www.website.com/?utm_medium=email&utm_source=newsletter&utm_campaign=12-5-2015 utm_medium utm_content utm_source

608 views • 6 slides

Orientation Lectures 5-6 ANLP Lecture 8 Task: Language modelling Part-of-speech tagging

Orientation Lectures 5-6 ANLP Lecture 8 Task: Language modelling Part-of-speech tagging Model: Sequence model, all variables directly observed Sharon Goldwater Lecture 7 (based on slides by Philipp Koehn) Task: Text classification 1

351 views • 11 slides

POS Tagging HMMs L645 / B659 Dept. of Linguistics, Indiana University Fall 2015 1 / 17 POS

POS Tagging Definition Tagsets Automatic POS Tagging Bigram tagging MLE POS Tagging HMMs L645 / B659 Dept. of Linguistics, Indiana University Fall 2015 1 / 17 POS Tagging Def. Part of Speech Tagging Definition Tagsets Automatic POS

659 views • 17 slides

Hacking a Way Through the Twitter Language Jungle: Syntactic Annotation, Tagging, and Parsing

Hacking a Way Through the Twitter Language Jungle: Syntactic Annotation, Tagging, and Parsing of English Tweets - 8 2 9 5 3 8 / l m m o t h c . . r x e i f p a a r p g l l l e a e w f / - e / l : g p

687 views • 42 slides

Natural Language Processing with Python CS372: Spring, 20 15 Lecture 12 Categorizing and

Natural Language Processing with Python CS372: Spring, 20 15 Lecture 12 Categorizing and Tagging Words Jong C. Park Department of Computer Science Korea Advanced Institute of Science and Technology CATEGORIZING AND TAGGING WORDS Using a

688 views • 34 slides

Lecture 5: Part-of-Speech Tagging Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center

CS447: Natural Language Processing http://courses.engr.illinois.edu/cs447 Lecture 5: Part-of-Speech Tagging Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center POS tagging Tagset: NNP: proper noun POS tagger CD: numeral, JJ:

671 views • 45 slides

Lecture 10: Part-of-Speech Tagging Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center

CS447: Natural Language Processing http://courses.engr.illinois.edu/cs447 Lecture 10: Part-of-Speech Tagging Julia Hockenmaier juliahmr@illinois.edu 3324 Siebel Center POS tagging Tagset: NNP: proper noun POS tagger CD: numeral, JJ:

798 views • 58 slides