Introduction Dense Sense Representations Sparse Sense Representations Future Work Unsupervised Knowledge-Free Word Sense Disambiguation Dr. Alexander Panchenko University of Hamburg, Language Technology Group 23 February, 2017 Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Overview Introduction Dense Sense Representations Sparse Sense Representations Future Work Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work About me ◮ 2008, Engineering degree (MS.) in Computer Science, Moscow State Technical University ◮ 2009, Research intern , Xerox Research Centre Europe ◮ 2013, PhD in Natural Language Processing , University of Louvain ◮ 2013, Research engineer at a start-up related to social network analysis (Digsolab) ◮ 2015, Postdoc at Technical University of Darmstadt ◮ 2017, Postdoc at University of Hamburg Topics : computational lexical semantics (semantic similarity/relatedness, semantic relations, sense induction, sense disambiguation), nlp for social network analysis, text categorization Papers, presentations, datasets : http://panchenko.me Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Publications Related to the Talk ◮ Pelevina M., Arefiev N., Biemann C., Panchenko A. (2016) Making Sense of Word Embeddings . In Proceedings of the 1st Workshop on Representation Learning for NLP. ACL 2016, Berlin, Germany. Best Paper Award ◮ Panchenko A., Simon J., Riedl M., Biemann C. (2016) Noun Sense Induction and Disambiguation using Graph-Based Distributional Semantics . In Proceedings of the KONVENS 2016, Bochum, Germany ◮ Panchenko A., Ruppert E., Faralli S., Ponzetto S. P., and Biemann C. (2017). Unsupervised Does Not Mean Uninterpretable: The Case for Word Sense Induction and Disambiguation . In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’2017). Valencia, Spain Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Motivation for Unsupervised Knowledge-Free WSD ◮ A word sense disambiguation (WSD) system: ◮ Input : word and its context. ◮ Output : a sense of this word. Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Motivation for Unsupervised Knowledge-Free WSD ◮ A word sense disambiguation (WSD) system: ◮ Input : word and its context. ◮ Output : a sense of this word. ◮ Existing approaches (Navigli, 2009): ◮ Knowledge-based approaches that rely on hand-crafted resources, such as WordNet. ◮ Supervised approaches learn from hand-labeled training data, such as SemCor. Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Motivation for Unsupervised Knowledge-Free WSD ◮ A word sense disambiguation (WSD) system: ◮ Input : word and its context. ◮ Output : a sense of this word. ◮ Existing approaches (Navigli, 2009): ◮ Knowledge-based approaches that rely on hand-crafted resources, such as WordNet. ◮ Supervised approaches learn from hand-labeled training data, such as SemCor. ◮ Problem 1: hand-crafted lexical resources and training data expensive, often inconsistent, domain-dependent. ◮ Problem 2: These methods assume a fixed sense inventory: ◮ senses emerge and disappear over time. ◮ different applications require different granularities. Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Motivation for Unsupervised Knowledge-Free WSD (cont.) ◮ An alternative route is the unsupervised knowledge-free approach . ◮ learn an interpretable sense inventory ◮ learn a disambiguation model Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Dense Sense Representations for WSD ◮ Pelevina M., Arefiev N., Biemann C., Panchenko A. Making Sense of Word Embeddings . In Proceedings of the 1st Workshop on Representation Learning for NLP. ACL 2016, Berlin, Germany. ◮ An approach to learn word sense embeddings . Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Overview of the contribution Prior methods: ◮ Induce inventory by clustering of word instances (Li and Jurafsky, 2015) ◮ Use existing inventories (Rothe and Sch¨ utze, 2015) Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Overview of the contribution Prior methods: ◮ Induce inventory by clustering of word instances (Li and Jurafsky, 2015) ◮ Use existing inventories (Rothe and Sch¨ utze, 2015) Our method: ◮ Input: word embeddings ◮ Output: word sense embeddings ◮ Word sense induction by clustering of word ego-networks ◮ Word sense disambiguation based on the induced sense representations Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Learning Word Sense Embeddings Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Word Sense Induction: Ego-Network Clustering ◮ Graph clustering using the Chinese Whispers algorithm (Biemann, 2006). Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Neighbours of Word and Sense Vectors Vector Nearest Neighbours tray, bottom, diagram, bucket, brackets, stack, table basket, list, parenthesis, cup, trays, pile, play- field, bracket, pot, drop-down, cue, plate leftmost#0, column#1, randomly#0, tableau#1, top-left0, indent#1, bracket#3, table#0 pointer#0, footer#1, cursor#1, diagram#0, grid#0 pile#1, stool#1, tray#0, basket#0, bowl#1, table#1 bucket#0, box#0, cage#0, saucer#3, mir- ror#1, birdcage#0, hole#0, pan#1, lid#0 ◮ Neighbours of the word “table” and its senses produced by our method. ◮ The neighbours of the initial vector belong to both senses . ◮ The neighbours of the sense vectors are sense-specific . Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Word Sense Disambiguation 1. Context Extraction ◮ use context words around the target word 2. Context Filtering ◮ based on context word’s relevance for disambiguation 3. Sense Choice ◮ maximize similarity between context vector and sense vector Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Word Sense Disambiguation: Example Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Introduction Dense Sense Representations Sparse Sense Representations Future Work Evaluation on SemEval 2013 Task 13 Dataset: Comparison to the State-of-the-art Model Jacc. Tau WNDCG F.NMI F.B-Cubed AI-KU (add1000) 0.176 0.609 0.205 0.033 0.317 AI-KU 0.176 0.619 0.393 0.066 0.382 AI-KU (remove5-add1000) 0.228 0.654 0.330 0.040 0.463 Unimelb (5p) 0.198 0.623 0.374 0.056 0.475 Unimelb (50k) 0.198 0.633 0.384 0.060 0.494 UoS (#WN senses) 0.171 0.600 0.298 0.046 0.186 UoS (top-3) 0.220 0.637 0.370 0.044 0.451 La Sapienza (1) 0.131 0.544 0.332 – – La Sapienza (2) 0.131 0.535 0.394 – – AdaGram, α = 0.05, 100 dim 0.274 0.644 0.318 0.058 0.470 w2v 0.197 0.615 0.291 0.011 0.615 w2v (nouns) 0.179 0.626 0.304 0.011 0.623 JBT 0.205 0.624 0.291 0.017 0.598 JBT (nouns) 0.198 0.643 0.310 0.031 0.595 TWSI (nouns) 0.215 0.651 0.318 0.030 0.573 Dr. Alexander Panchenko University of Hamburg, Language Technology Group Unsupervised Knowledge-Free Word Sense Disambiguation
Recommend
More recommend
