unsupervised morphological segmentation clustering
play

UNSUPERVISED MORPHOLOGICAL SEGMENTATION & CLUSTERING ICL UNI - PowerPoint PPT Presentation

Dec 29, 2023 •506 likes •762 views

UNSUPERVISED MORPHOLOGICAL SEGMENTATION & CLUSTERING ICL UNI HEIDELBERG - HS CL4LRL - KATHARINA ALLGAIER - 08.06.2016 1 OVERVIEW Introduction Morphological Segmentation (Creutz&Lagus 2005) Aims Models

  1. UNSUPERVISED MORPHOLOGICAL SEGMENTATION & CLUSTERING ICL UNI HEIDELBERG - HS CL4LRL - KATHARINA ALLGAIER - 08.06.2016 1

  2. OVERVIEW  Introduction  Morphological Segmentation (Creutz&Lagus 2005)  Aims  Models  Evaluation  Results  Affix Clustering (Moon et al 2009)  Idea  Model  Results  Conclusion 2

  3. WHAT ARE WE DOING?  Morpheme Segmentation reads = read + s  Morphemes = smallest meaning-bearing units machines = machine + s  = smallest elements of syntax translation = translate + ion  Meaning vs. Form goalkeeper = goal + keeper joystick = joy + stick  Composition vs. Perturbation 3

  4. WHAT ARE WE DOING ?  Stem vs. Affixes (Prefixes + Suffixes)  Inflectional vs. Derivational  Affix Clustering 4

  5. WHY ARE WE DOING IT?  important information  especially for highly inflected languages  (like Turkish, Finnish, Nahuatl, Japanese  agglutinative languges)  used in other CL applications  (language production, speech recognition, machine translation etc.) 5

  6. INDUCING THE MORPHOLOGICAL LEXICON OF A NATURAL LANGUAGE FROM UNANNOTATED TEXT – CREUTZ&LAGUS 2005 „ algorithm for the unsupervised learning […] of a simple morphology of a natural language “   Unsupervised morpheme segmentation with hierarchical representation  English and Finnish 6

  7. AIMS  Most accurate segmentation possible  Learn representation of the language in the data + store it in a lexicon  Based several models: Linguistica, Morfessor Baseline, Morfessor ML, Morfessor MAP 7

  8. BASELINE Morph Lexicon talk teach es  Morfessor Baseline Algorithm (Creutz&Langus) ed ing word  Similar to some unsupervised word segmentation algorithms words morf  Construct lexicon of morphs es  Each word can be constructed out of those morphs sor  AIM: find optimal + concise segmentation and lexicon  PROBLEM: frequent words stored as a whole - rare words excessively split + stored in part no representation of a morph‘s inner structure 8

  9.  Linguistica (Goldsmith 2001)  Splits word into stem + one (empty) prefix / affix ADVANTAGE: Modeling of simple word-internal syntax (morphotactics – rules on ordering of morphemes)  – grouping sets of stems & suffixes into inflectional paradigms word +s talk + ed talk + s dog + s walk + ed walk + s  DISADVANTAGE: handles highly inflecting + compounding languages poorly (alternating stems + affixes) 9

  10. IMPROVED MODEL  Morfessor Categories-ML (Creutz&Lagus) hidden states: categories (SUFF, PRE, …)  Reanalyzes segmentation of Morphessor Baseline  Maximum Likelihood Model  Words represented as HMMs  Stems, prefixes + suffixes can alternate (with some restrictions) „ noise “ category   split words whose morphs are present in the lexicon   join „ noise “ morphs with their neighbours to form proper morphs  observable states: morphs  CRITICISM: too ad hoc + information on word frequency is lost 10

  11. NEW MODEL  Morfessor Categories-MAP (Creutz&Lagus)  Induces binary hierarchical lexicon Retains inner structure of words  morphs represented as concatenation of (sub)morphs of the lexicon   Word frequency (own entry vs. Split into morphs) Prefix – Stem – Suffix – Non-morpheme  11

  12.  Maximum a posteriori framework  Words represented as HMMs Desired level of segmentation : „ finest resolution that does not contain non-morphemes “  12

  13. SEARCH ALGORITHM (GREEDY SEARCH) Initialisation of Representativ+ness segmentation stem+SUFF [Re+[present+ativ]]+[n+ess] Splitting of morphs PRE+stem+SUFF+non+SUFF [Re+[present+ativ]]+ness Joining of morphs PRE+stem+SUFF+SUFF [Re+[[pre+sent]+ativ]]+ness Splitting of morphs PRE+non+stem+SUFF+SUFF [Re+[[pre+sent]+ativ]]+ness Resegmentation of corpus + re-estimation PRE+non+stem+SUFF+SUFF of probabilitites [Re+[present+ativ]]+ness 13 Expansion to finest PRE+stem+SUFF+SUFF resolution

  14. MODEL  AIM: Finding optimal lexicon + segmentation  Maximum a posteriori estimate to be maximized: Form Meaning String of letters vs. Submorphs Frequency 14 Morph emission probability transition probability Length Right+Left Perplexity

  15.  Morph Emission Probabilities  probability that morph is emitted by the category   Depend on frequency of morph in training data  Prefix-/Suffix-Likeness (right+left perplexity)  Stem-Likeness (length)  Non-morpheme probability 15

  16. EVALUATION Goldstandard English Data Finnish Data Hutmegs Prose + news + Prose + news text  Linguistic scientific text morpheme Finnish IT Centre of segmentations Gutenberg Project Science  1.4 million Finnish Gigaword Corpus Finnish National  120 000 English Brown Corpus News Agency word forms Evaluation on 10.000, 50.000, 250.000, 12/16 million 16 words

  17. RESULTS 17

  18. UNSUPERVISED MORPHOLOGICAL SEGMENTATION AND CLUSTERING WITH DOCUMENT BOUNDARIES – MOON ET AL 2009  Simple model without heuristics /thresholds /trained parameters  Word segmentation - constrain candidate stems + affixes by document boundaries Cluster affixes of certain stems  morphologically related words   USE: interlinearised glossed texts for LRL  English + Uspanteko 18

  19. IDEA  two words in the same document are very similar in orthography  likely to be related morphologically  use document boundaries to filter out noise  constrain potential membership of word clusters 19 He suddendly drew a sharp sword … The documentation of …

  20. MODEL Candidate Generation Conflation set : „Set of word types that are related through either inflectional or derivational morphology “ 20

  21. CANDIDATE TRIE Stems  trunks like li ness  affixes branches ness hood 21

  22. MODEL Candidate Generation (D vs. G) X2 testing : Correlation betw. Affixes Candidate Conflation set : Filtering „Set of word types that are related through either inflectional or derivational morphology “ Affix Clustering Word Clustering (D vs. G) 22

  23. RESULTS 23

  24.  Thank you for your attention! 24

Recommend

Color Image Segmentation based on Automatic Morphological Clustering T. Graud, P.-Y. Strub,

Color Image Segmentation based on Automatic Morphological Clustering T. Graud, P.-Y. Strub,

Color Image Segmentation based on Automatic Morphological Clustering T. Graud, P.-Y. Strub, J. Darbon thierry.geraud @ lrde.epita.fr EPITA Research and Development Laboratory, France / ICIP, Thessaloniki, October 2001 1 Outline

469 views • 23 slides
Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat

Chapter 7: Clustering (Unsupervised Data Organization) 7.1 Hierarchical Clustering 7.2 Flat Clustering 7.3 Embedding into Vector Space for Visualization 7.4 Applications Clustering: unsupervised grouping (partitioning) of objects into

763 views • 49 slides
1 Mean Shift Algorithm Graph-Theoretic Image Segmentation Mean Shift Algorithm 1. Choose a

1 Mean Shift Algorithm Graph-Theoretic Image Segmentation Mean Shift Algorithm 1. Choose a

6.891 Last time: Segmentation and Clustering (Ch. 14) Computer Vision and Applications Supervised->Unsupervised Category Learning needs segmentation Prof. Trevor. Darrell K-Means Mean Shift Graph cuts Lecture 15:

531 views • 19 slides
Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning no labels/output, only x/input Clustering Group similar points together Machine learning taxonomy Supervised Semi-Supervised Unsupervised

366 views • 15 slides
Automatic Learning of a Morphological Model Theory and Unsupervised Approaches Unsupervised

Automatic Learning of a Morphological Model Theory and Unsupervised Approaches Unsupervised

Automatic Learning of a Morphological Model Theory and Unsupervised Approaches Unsupervised Learning - ToC Foundations Problem description General architecture Papers: Goldsmith 2001 and 2006 Goldsmith and Hu 2004

1.23k views • 43 slides
6.891 Computer Vision and Applications Prof. Trevor. Darrell Lecture 14: Unsupervised

6.891 Computer Vision and Applications Prof. Trevor. Darrell Lecture 14: Unsupervised

6.891 Computer Vision and Applications Prof. Trevor. Darrell Lecture 14: Unsupervised Category Learning Gestalt Principles Segmentation by Clustering K-Means Graph cuts Segmentation by Fitting Hough transform

1.14k views • 112 slides
UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised learning: X - y pairs, f(x) function approximation Unsupervised learning: only X, no y Exploring the space of X measurements,

440 views • 14 slides
An Unsupervised Method for Uncovering Morphological Chains Karthik Narasimhan Regina Barzilay

An Unsupervised Method for Uncovering Morphological Chains Karthik Narasimhan Regina Barzilay

An Unsupervised Method for Uncovering Morphological Chains Karthik Narasimhan Regina Barzilay Tommi Jaakkola CSAIL, Massachusetts Institute of Technology 1 Morphological Chains 2 Morphological Chains Chains to model the formation of words.

1.12k views • 74 slides
Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 11 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels for training) Group data into similar classes that Maximize

833 views • 26 slides
Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 10 Jan-Willem van de Meent Clustering Clustering Unsupervised learning (no labels for training) Group data into similar classes that Maximize

1.29k views • 69 slides
Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no output classifications (labels) l Clustering is an important type of unsupervised learning PCA was another type of unsupervised learning l The

858 views • 38 slides
Dsolve Morphological Segmentation for German using Conditional Random Fields Kay-Michael W

Dsolve Morphological Segmentation for German using Conditional Random Fields Kay-Michael W

Dsolve Morphological Segmentation for German using Conditional Random Fields Kay-Michael W urzner, Bryan Jurish { wuerzner,jurish } @bbaw.de SFCM Universit at Stuttgart 17th September 2015 Outline p Morphological analysis p Existing

395 views • 25 slides
LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a

DATA MINING LECTURE 8 The EM Algorithm Clustering Validation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

998 views • 78 slides
3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature

3D Deep Clustering a clustering framework for unsupervised learning of 3D object feature descriptors LAMBDA Workshop on Retrieval and Shape analysis Alexandros Georgiou M.Sc. in Algorithms, Logic and Discrete Mathematics 1/12 Problem

570 views • 53 slides
1 Top-down segmentation Basic ideas of grouping in human vision Figure-ground Gestalt

1 Top-down segmentation Basic ideas of grouping in human vision Figure-ground Gestalt

Segmentation by Clustering Segmentation by Clustering Reading: Chapter 14 (skip 14.5) Data reduction - obtain a compact representation for interesting image data in terms of a set of components Find components that belong together (form

291 views • 6 slides
DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING

DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING

DATA MINING LECTURE 9 The EM Algorithm Clustering Evaluation Sequence segmentation CLUSTERING What is a Clustering? In general a grouping of objects such that the objects in a group (cluster) are similar (or related) to one another and

905 views • 79 slides
Co-Segmentation of 3D Shapes via Subspace Clustering Ruizhen Hu Lubin Fan

Co-Segmentation of 3D Shapes via Subspace Clustering Ruizhen Hu Lubin Fan

Co-Segmentation of 3D Shapes via Subspace Clustering Ruizhen Hu Lubin Fan Ligang Liu Co-segmentation Input Hu et al. Co-Segmentation of 3D Shapes 2 Co-segmentation Output Hu et al. Co-Segmentation of 3D Shapes 3

1.26k views • 32 slides
UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised

UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised

UNSUPERVISED LEARNING AND CLUSTERING Jeff Robble, Brian Renzenbrink, Doug Roberts Unsupervised Procedures A procedure that uses unlabeled data in its classification process. Why would we use these? Collecting and labeling large data sets can

544 views • 28 slides
Clustering Clustering is an unsupervised classification method, i.e. unlabeled data is partitioned

Clustering Clustering is an unsupervised classification method, i.e. unlabeled data is partitioned

Clustering Clustering is an unsupervised classification method, i.e. unlabeled data is partitioned into subsets (clusters), according to a similarity measure, such thatsimilardata is grouped into the same cluster. Unlabeled Data

442 views • 22 slides
Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

UNSUPERVISED LEARNING IN R Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of clustering finding groups of homogeneous items Next up, dimensionality reduction Find structure in features

904 views • 29 slides
Clustering Lecture notes Clustering is Exploratory, unsupervised method Data in cluster is

Clustering Lecture notes Clustering is Exploratory, unsupervised method Data in cluster is

Clustering Lecture notes Clustering is Exploratory, unsupervised method Data in cluster is similar to each other, and dissimilar to other cluster data Finding structure in data Hard vs. Soft/Fuzzy vs. Exclusive Understanding and/or summarise

400 views • 26 slides
Morphology & Transducers Intro to morphological analysis of languages Motivation for

Morphology & Transducers Intro to morphological analysis of languages Motivation for

Morphology & Transducers Intro to morphological analysis of languages Motivation for morphological analysis in NLP Morphological Recognition by FSAs Transducers Unsupervised Learning (2 nd hour) Speech and Language

744 views • 31 slides
9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari + Daneil Zysman + Darren Seibert Outline Introduction to clustering K-means Bag of words (dictionary learning) Hierarchical clustering

895 views • 54 slides
Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition Systems Stephen

Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition Systems Stephen

Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition Systems Stephen H. Shum Douglas A. Reynolds Daniel Garcia-Romero Alan McCree Unsupervised Clustering Approaches for Domain Adaptation in Speaker Recognition

437 views • 25 slides

More recommend