Improving Hypernymy Extraction with Distributional Semantic Classes - PowerPoint PPT Presentation

Alexander Panchenko, Dmitry Ustalov, Stefano Faralli, Simone Paolo Ponzetto, and Chris Biemann Improving Hypernymy Extraction with Distributional Semantic Classes

Introduction May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 2/33

Introduction Hypernyms Examples of hypernymy relations apple –isa→ fruit mangosteen –isa→ fruit May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 3/33

“ This café serves fresh mangosteen juice ” Examples of applications of hypernyms question answering [Zhou et al., 2013] query expansion [Gong et al., 2005] semantic role labelling [Shi & Mihalcea, 2005] Introduction Hypernyms Examples of hypernymy relations apple#1 –isa→ fruit#2 mangosteen#0 –isa→ fruit#2 May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 4/33

Examples of applications of hypernyms question answering [Zhou et al., 2013] query expansion [Gong et al., 2005] semantic role labelling [Shi & Mihalcea, 2005] Introduction Hypernyms Examples of hypernymy relations apple#1 –isa→ fruit#2 mangosteen#0 –isa→ fruit#2 “ This café serves fresh mangosteen juice ” May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 4/33

Introduction Hypernyms Examples of hypernymy relations apple#1 –isa→ fruit#2 mangosteen#0 –isa→ fruit#2 “ This café serves fresh mangosteen juice ” Examples of applications of hypernyms question answering [Zhou et al., 2013] query expansion [Gong et al., 2005] semantic role labelling [Shi & Mihalcea, 2005] May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 4/33

2 [Snow et al., 2004]: lexical-syntactic patterns learned in a supervised way ; 3 [Weeds et al., 2014]: supervised approach with word embedding features ; 4 [Shwartz et al., 2016]: supervised approach with word and path embedding features ; 5 [Glavaš & Ponzetto, 2017, Ustalov et al., 2017]: taking into account asymmetry of hypernyms. Not taking into account word senses and global structure! Introduction Automatic extraction of hypernyms A short history of extraction methods 1 [Hearst, 1992]: lexical-syntactic patterns defjned manually ; May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 5/33

3 [Weeds et al., 2014]: supervised approach with word embedding features ; 4 [Shwartz et al., 2016]: supervised approach with word and path embedding features ; 5 [Glavaš & Ponzetto, 2017, Ustalov et al., 2017]: taking into account asymmetry of hypernyms. Not taking into account word senses and global structure! Introduction Automatic extraction of hypernyms A short history of extraction methods 1 [Hearst, 1992]: lexical-syntactic patterns defjned manually ; 2 [Snow et al., 2004]: lexical-syntactic patterns learned in a supervised way ; May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 5/33

Not taking into account word senses and global structure! Introduction Automatic extraction of hypernyms A short history of extraction methods 1 [Hearst, 1992]: lexical-syntactic patterns defjned manually ; 2 [Snow et al., 2004]: lexical-syntactic patterns learned in a supervised way ; 3 [Weeds et al., 2014]: supervised approach with word embedding features ; 4 [Shwartz et al., 2016]: supervised approach with word and path embedding features ; 5 [Glavaš & Ponzetto, 2017, Ustalov et al., 2017]: taking into account asymmetry of hypernyms. May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 5/33

Introduction Automatic extraction of hypernyms A short history of extraction methods 1 [Hearst, 1992]: lexical-syntactic patterns defjned manually ; 2 [Snow et al., 2004]: lexical-syntactic patterns learned in a supervised way ; 3 [Weeds et al., 2014]: supervised approach with word embedding features ; 4 [Shwartz et al., 2016]: supervised approach with word and path embedding features ; 5 [Glavaš & Ponzetto, 2017, Ustalov et al., 2017]: taking into account asymmetry of hypernyms. Not taking into account word senses and global structure! May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 5/33

Introduction Induction of semantic classes “ Global distributional structure ” of a language ≈ global sense clustering, e.g. panchenko.me/data/joint/nodes20000-layers7 May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 6/33

Introduction Induction of semantic classes “ Global distributional structure ” of a language ≈ global sense clustering, e.g. panchenko.me/data/joint/nodes20000-layers7 May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 7/33

No explicit evaluation of utility of hypernymy labels for hypernymy extraction. Introduction Induction of semantic classes A short history of extraction methods 1 [Lin & Pantel, 2001]: sets of similar words are clustered into concepts. 2 [Pantel & Lin, 2002]: words can belong to several clusters (representing senses) 3 [Pantel & Ravichandran, 2004]: aggregate hypernyms per cluster from from Hearst patterns May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 8/33

Introduction Induction of semantic classes A short history of extraction methods 1 [Lin & Pantel, 2001]: sets of similar words are clustered into concepts. 2 [Pantel & Lin, 2002]: words can belong to several clusters (representing senses) 3 [Pantel & Ravichandran, 2004]: aggregate hypernyms per cluster from from Hearst patterns No explicit evaluation of utility of hypernymy labels for hypernymy extraction. May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 8/33

A method for inducing sense-aware semantic classes using 1 distributional semantics; 2 A method for using the induced semantic classes for fjltering noisy hypernymy relations . Introduction Main contributions We show how distributionally-induced semantic classes can be helpful for extracting hypernyms : May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 9/33

Introduction Main contributions We show how distributionally-induced semantic classes can be helpful for extracting hypernyms : A method for inducing sense-aware semantic classes using 1 distributional semantics; 2 A method for using the induced semantic classes for fjltering noisy hypernymy relations . May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 9/33

Method May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 10/33

Method Labeled semantic classes Post-processing of hypernymy relations using distributionally induced semantic classes; A semantic class is a clusters of induced word senses labeled with hypernyms . May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 11/33

Method Outline of our approach 1 Sense-aware distributional semantic classes are induced from a text corpus ; 2 Semantic classes are used to fjlter a noisy hypernym database. May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 12/33

Method Outline of our approach 1 Sense-aware distributional semantic classes are induced from a text corpus ; 2 Semantic classes are used to fjlter a noisy hypernym database. §3 Induction of Semantic Classes §3.1 §3.2 §3.3 §3.4 Sense Ego-Networks Induced Word Senses Global Sense Graph Word Sense Induction Representing Senses Sense Graph Clustering of with Ego Networks from Text Corpus Construction Word Senes Global Sense Clusters §4 Noisy Hypernyms Labeling Sense Clusters with Hypernyms Semantic Classes Text Corpus Cleansed Hypernyms May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 12/33

Method Chinese Whispers#1 * source of the image: http://ic.pics.livejournal.com/blagin_anton/33716210/2701748/2701748_800.jpg May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 13/33

Method Chinese Whispers#2: graph clustering May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 14/33

Method Chinese Whispers#2: graph clustering May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 15/33

Method Chinese Whispers#2: graph clustering May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 16/33

Method Graph-based word sense induction May 10, 2018 Improving Hypernymy Extraction with Distributional Semantic Classes, Panchenko et al. LREC’18 17/33