embeddings with almost
play

embeddings with (almost) no bilingual data Mik ikel l Art - PowerPoint PPT Presentation

Aug 19, 2023 •277 likes •1.63k views

Learning bilingual word embeddings with (almost) no bilingual data Mik ikel l Art rtetxe, Gorka Labaka, Eneko Agirre IXA NLP group University of the Basque Country (UPV/EHU) Who cares? Who cares? word embeddings are useful! Who

  1. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals Eng English-Italian Eng English-German Eng English-Finnish 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num num.

  2. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian Eng English-German Eng English-Finnish 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num num.

  3. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian Eng English-German Eng English-Finnish 5,00 ,000 25 25 num. num 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num num. wor ord translation inductio ion

  4. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian Eng English-German Eng English-Finnish 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num num. Mikolov et al. (2013a) Xing et al. (2015) Zhang et al. (2016) Artetxe et al. (2016) wor ord translation inductio ion

  5. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian Eng English-German Eng English-Finnish 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num. num Mikolov et al. (2013a) Xing et al. (2015) Zhang et al. (2016) Artetxe et al. (2016) Our method wor ord translation inductio ion

  6. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian Eng English-German English-Finnish Eng 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num. num 5,00 ,000 25 25 num num. Mikolov et al. (2013a) 34.93% 0.00% 0.00% 35.00% 0.00% 0.07% 25.91% 0.00% 0.00% Xing et al. (2015) 36.87% 0.00% 0.13% 41.27% 0.07% 0.53% 28.23% 0.07% 0.56% Zhang et al. (2016) 36.73% 0.07% 0.27% 40.80% 0.13% 0.87% 28.16% 0.14% 0.42% Artetxe et al. (2016) 39.27% 0.07% 0.40% 41.87 .87% 0.13% 0.73% 30.62 .62% 0.21% 0.77% Our method 39.67 .67% 37.27 .27% 39.40 .40% 40.87% 39.60 .60% 40.27 .27% 28.72% 28.16 .16% 26.47 .47% wor ord translation inductio ion

  7. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian Eng English-German English-Finnish Eng 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num. num 5,00 ,000 25 25 num num. Mikolov et al. (2013a) 34.93% 0.00% 0.00% 35.00% 0.00% 0.07% 25.91% 0.00% 0.00% Xing et al. (2015) 36.87% 0.00% 0.13% 41.27% 0.07% 0.53% 28.23% 0.07% 0.56% Zhang et al. (2016) 36.73% 0.07% 0.27% 40.80% 0.13% 0.87% 28.16% 0.14% 0.42% Artetxe et al. (2016) 39.27% 0.07% 0.40% 41.87 .87% 0.13% 0.73% 30.62 .62% 0.21% 0.77% Our method 39.67 .67% 37.27 .27% 39.40 .40% 40.87% 39.60 .60% 40.27 .27% 28.72% 28.16 .16% 26.47 .47% wor ord translation inductio ion

  8. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian Eng English-German English-Finnish Eng 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num. num 5,00 ,000 25 25 num num. Mikolov et al. (2013a) 34.93% 0.00% 0.00% 35.00% 0.00% 0.07% 25.91% 0.00% 0.00% Xing et al. (2015) 36.87% 0.00% 0.13% 41.27% 0.07% 0.53% 28.23% 0.07% 0.56% Zhang et al. (2016) 36.73% 0.07% 0.27% 40.80% 0.13% 0.87% 28.16% 0.14% 0.42% Artetxe et al. (2016) 39.27% 0.07% 0.40% 41.87 .87% 0.13% 0.73% 30.62 .62% 0.21% 0.77% Our method 39.67 .67% 37.27 .27% 39.40 .40% 40.87% 39.60 .60% 40.27 .27% 28.72% 28.16 .16% 26.47 .47% wor ord translation inductio ion

  9. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian Eng English-German English-Finnish Eng 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num. num 5,00 ,000 25 25 num num. Mikolov et al. (2013a) 34.93% 0.00% 0.00% 35.00% 0.00% 0.07% 25.91% 0.00% 0.00% Xing et al. (2015) 36.87% 0.00% 0.13% 41.27% 0.07% 0.53% 28.23% 0.07% 0.56% Zhang et al. (2016) 36.73% 0.07% 0.27% 40.80% 0.13% 0.87% 28.16% 0.14% 0.42% Artetxe et al. (2016) 39.27% 0.07% 0.40% 41.87 .87% 0.13% 0.73% 30.62 .62% 0.21% 0.77% Our method 39.67 .67% 37.27 .27% 39.40 .40% 40.87% 39.60 .60% 40.27 .27% 28.72% 28.16 .16% 26.47 .47% wor ord translation inductio ion

  10. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian Eng English-German English-Finnish Eng 5,00 ,000 25 25 num num. 5,00 ,000 25 25 num. num 5,00 ,000 25 25 num num. Mikolov et al. (2013a) 34.93% 0.00% 0.00% 35.00% 0.00% 0.07% 25.91% 0.00% 0.00% Xing et al. (2015) 36.87% 0.00% 0.13% 41.27% 0.07% 0.53% 28.23% 0.07% 0.56% Zhang et al. (2016) 36.73% 0.07% 0.27% 40.80% 0.13% 0.87% 28.16% 0.14% 0.42% Artetxe et al. (2016) 39.27% 0.07% 0.40% 41.87 .87% 0.13% 0.73% 30.62 .62% 0.21% 0.77% Our method 39.67 .67% 37.27 .27% 39.40 .40% 40.87% 39.60 .60% 40.27 .27% 28.72% 28.16 .16% 26.47 .47% wor ord translation inductio ion

  11. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals ⇒ Test dictionary: 1,500 word pairs Eng English-Italian 5,00 ,000 25 25 num num. Mikolov et al. (2013a) 34.93% 0.00% 0.00% Xing et al. (2015) 36.87% 0.00% 0.13% Zhang et al. (2016) 36.73% 0.07% 0.27% Artetxe et al. (2016) 39.27% 0.07% 0.40% Our method 39.67 .67% 37.27 .27% 39.40 .40% wor ord translation inductio ion

  12. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals

  13. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals cros ossli lingual wor ord si simil ilarity

  14. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals EN EN-IT IT EN EN-DE DE WS WS RG RG WS WS cros ossli lingual wor ord si simil ilarity

  15. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals EN EN-IT IT EN EN-DE DE Bi Bi. . da data WS WS RG RG WS WS cros ossli lingual wor ord si simil ilarity

  16. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals EN EN-IT IT EN EN-DE DE Bi Bi. . da data WS WS RG RG WS WS Luong et al. (2015) Europarl cros ossli lingual wor ord si simil ilarity

  17. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals EN EN-IT IT EN EN-DE DE Bi Bi. . da data WS WS RG RG WS WS Luong et al. (2015) Europarl Mikolov et al. (2013a) 5k dict Xing et al. (2015) 5k dict Zhang et al. (2016) 5k dict Artetxe et al. (2016) 5k dict cros ossli lingual wor ord si simil ilarity

  18. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals EN EN-IT IT EN EN-DE DE Bi Bi. . da data WS WS RG RG WS WS Luong et al. (2015) Europarl Mikolov et al. (2013a) 5k dict Xing et al. (2015) 5k dict Zhang et al. (2016) 5k dict Artetxe et al. (2016) 5k dict 5k dict Our method 25 dict num. cros ossli lingual wor ord si simil ilarity

  19. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals EN EN-IT IT EN EN-DE DE Bi Bi. . da data WS WS RG RG WS WS Luong et al. (2015) Europarl 33.1% 33.5% 35.6% Mikolov et al. (2013a) 5k dict 62.7% 64.3% 52.8% Xing et al. (2015) 5k dict 61.4% 70.0% 59.5% Zhang et al. (2016) 5k dict 61.6% 70.4% 59.6% Artetxe et al. (2016) 5k dict 61.7% 71.6% 59.7% 5k dict 62.4% 74.2% 61.6% .6% Our method 25 dict 62.6% 74.9% .9% 61.2% num. 62.8% .8% 73.9% 60.4% cros ossli lingual wor ord si simil ilarity

  20. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals EN EN-IT IT EN EN-DE DE Bi Bi. . da data WS WS RG RG WS WS Luong et al. (2015) Europarl 33.1% 33.5% 35.6% Mikolov et al. (2013a) 5k dict 62.7% 64.3% 52.8% Xing et al. (2015) 5k dict 61.4% 70.0% 59.5% Zhang et al. (2016) 5k dict 61.6% 70.4% 59.6% Artetxe et al. (2016) 5k dict 61.7% 71.6% 59.7% 5k dict 62.4% 74.2% 61.6% .6% Our method 25 dict 62.6% 74.9% .9% 61.2% num. 62.8% .8% 73.9% 60.4% cros ossli lingual wor ord si simil ilarity

  21. Experiments • Dataset by Dinu et al. (2015) extended to German and Finnish ⇒ Monolingual embeddings (CBOW + negative sampling) ⇒ Seed dictionary: 5,000 word pairs / 25 word pairs / numerals EN EN-IT IT EN EN-DE DE Bi Bi. . da data WS WS RG RG WS WS Luong et al. (2015) Europarl 33.1% 33.5% 35.6% Mikolov et al. (2013a) 5k dict 62.7% 64.3% 52.8% Xing et al. (2015) 5k dict 61.4% 70.0% 59.5% Zhang et al. (2016) 5k dict 61.6% 70.4% 59.6% Artetxe et al. (2016) 5k dict 61.7% 71.6% 59.7% 5k dict 62.4% 74.2% 61.6% .6% Our method 25 dict 62.6% 74.9% .9% 61.2% num. 62.8% .8% 73.9% 60.4% cros ossli lingual wor ord si simil ilarity

  22. Why does it work?

  23. Why does it work? Monolingual embeddings Dictionary Mapping Dictionary

  24. Why does it work? Monolingual embeddings small Dictionary Mapping Dictionary

  25. Why does it work? Monolingual embeddings small la large Dictionary Mapping Dictionary

  26. Why does it work? Monolingual embeddings small la large Dictionary Mapping Dictionary no o err rror

  27. Why does it work? Monolingual embeddings small la large Dictionary Mapping Dictionary no o err rror er errors

  28. Why does it work? Monolingual embeddings small la large Dictionary Mapping Dictionary no o err rror er errors Mapping Dictionary

  29. Why does it work? Monolingual embeddings small la large Dictionary Mapping Dictionary no o err rror er errors better? Mapping Dictionary

  30. Why does it work? Monolingual embeddings small large la Dictionary Mapping Dictionary no o err rror er errors better? Mapping Dictionary worse?

  31. Why does it work? Monolingual embeddings small la large Dictionary Mapping Dictionary no o err rror er errors better? Mapping Dictionary worse? Mapping Dictionary

  32. Why does it work? Monolingual embeddings small large la Dictionary Mapping Dictionary no o err rror er errors better? Mapping Dictionary worse? even en better? Mapping Dictionary

  33. Why does it work? Monolingual embeddings small large la Dictionary Mapping Dictionary no o err rror er errors better? Mapping Dictionary worse? even en better? Mapping Dictionary even en worse?

  34. Why does it work? 𝑌𝑋 𝑎

  35. Why does it work? 𝑌𝑋 𝑎 𝑋 ∗ = arg max s.t. 𝑋𝑋 𝑈 = 𝑋 𝑈 𝑋 = 𝐽 Implicit objective: ෍ max 𝑌 𝑗∗ 𝑋 ∙ 𝑎 𝑘∗ 𝑘 𝑋 𝑗

  36. Why does it work? 𝑌𝑋 𝑎 𝑋 ∗ = arg max s.t. 𝑋𝑋 𝑈 = 𝑋 𝑈 𝑋 = 𝐽 Implicit objective: ෍ max 𝑌 𝑗∗ 𝑋 ∙ 𝑎 𝑘∗ 𝑘 𝑋 𝑗 Independent from seed dictionary!

  37. Why does it work? 𝑌𝑋 𝑎 𝑋 ∗ = arg max s.t. 𝑋𝑋 𝑈 = 𝑋 𝑈 𝑋 = 𝐽 Implicit objective: ෍ max 𝑌 𝑗∗ 𝑋 ∙ 𝑎 𝑘∗ 𝑘 𝑋 𝑗

  38. Why does it work? 𝑌𝑋 𝑎 𝑋 ∗ = arg max s.t. 𝑋𝑋 𝑈 = 𝑋 𝑈 𝑋 = 𝐽 Implicit objective: ෍ max 𝑌 𝑗∗ 𝑋 ∙ 𝑎 𝑘∗ 𝑘 𝑋 𝑗

  39. Why does it work? 𝑌𝑋 𝑎 𝑋 ∗ = arg max s.t. 𝑋𝑋 𝑈 = 𝑋 𝑈 𝑋 = 𝐽 Implicit objective: ෍ max 𝑌 𝑗∗ 𝑋 ∙ 𝑎 𝑘∗ 𝑘 𝑋 𝑗

Recommend

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings Agenda Traditional NLP Word Embeddings-1 Word Embeddings-2 Text preprocessing Topic Modeling ELMo Bag-of-words model Neural

900 views • 64 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

422 views • 17 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

2k views • 24 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

828 views • 15 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

1.25k views • 75 slides
Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme:

Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme:

Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme: Index dun mot: w i Table dembeddings (lookup matrix): V Embedding: e i e i = V( w i ) Reprsentation dun mot

780 views • 47 slides
Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds

Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds

Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds University of California, San Diego Southern California Machine Learning Symposium, Caltech, 11/18/2018 Word Embeddings Language models which learn

537 views • 32 slides
What is best for spoken langage understanding: small but task-dependent embeddings or huge but

What is best for spoken langage understanding: small but task-dependent embeddings or huge but

What is best for spoken langage understanding: small but task-dependent embeddings or huge but out-of-domain embeddings Sahar Ghannay, Antoine Neuraz, Sophie Rosset 1 Goal Focus on semantic evaluation of common word embeddings

360 views • 16 slides
Subspace Embeddings and p -Regression Using Exponential Random Variables David P. Woodruff

Subspace Embeddings and p -Regression Using Exponential Random Variables David P. Woodruff

Subspace Embeddings and p -Regression Using Exponential Random Variables David P. Woodruff and Qin Zhang IBM Research Almaden COLT13, June 12, 2013 1-1 Subspace embeddings Subspace embeddings: A distribution over linear maps : R n

810 views • 44 slides
Embeddings @ Twitter Making ML easy with Embeddings !!! Sept 2018 Agenda 1 Team 2 Whats an

Embeddings @ Twitter Making ML easy with Embeddings !!! Sept 2018 Agenda 1 Team 2 Whats an

Embeddings @ Twitter Making ML easy with Embeddings !!! Sept 2018 Agenda 1 Team 2 Whats an Embedding ? 3 Why Embeddings ? 4 Embeddings Pipeline 5 Whats Next Agenda 1 Team 2 Whats an Embedding ? 3 Why Embeddings ? 4

732 views • 40 slides
Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview

Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview

Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview Word types and tokens Training contextual embeddings Embeddings from Language Models (ELMo) 1 Overview Word types and tokens Training

670 views • 40 slides
Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl Z 2 -embeddings and Tournaments

Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl Z 2 -embeddings and Tournaments

Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl Z 2 -embeddings and Tournaments Radoslav Fulek , Jan Kyn cl June 12, 2018 Drawings of Graphs Drawings of Graphs G = ( V, E ) is a simple graph . The set of vertices V is finite

596 views • 56 slides
CS 4803 / 7643: Deep Learning Guest Lecture: Embeddings and world2vec Feb. 18 th 2020 Ledell Wu

CS 4803 / 7643: Deep Learning Guest Lecture: Embeddings and world2vec Feb. 18 th 2020 Ledell Wu

CS 4803 / 7643: Deep Learning Guest Lecture: Embeddings and world2vec Feb. 18 th 2020 Ledell Wu Research Engineer, Facebook AI ledell@fb.com 1 Outline Word Embeddings word2vec Graph Embeddings Applications world2vec

769 views • 41 slides
Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to generate vector embeddings? One approach: feedforward neural language models Training a neural language model just to get word embeddings is expensive!

569 views • 38 slides
Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to

Dense Word Embeddings CMSC 470 Marine Carpuat Slides credit: Jurasky & Martin How to generate vector embeddings? One approach: feedforward neural language models Training a neural language model just to get word embeddings is expensive!

537 views • 35 slides
Embeddings for KB and text representation, extraction and question answering. Jason Weston

Embeddings for KB and text representation, extraction and question answering. Jason Weston

Embeddings for multi-relational data Pros and cons of embedding models Embeddings for KB and text representation, extraction and question answering. Jason Weston & Antoine Bordes & Sumit Chopra Facebook AI Research External

595 views • 24 slides
Name Embeddings and Online News Analysis Speaker: Junting Ye Department: Computer

Name Embeddings and Online News Analysis Speaker: Junting Ye Department: Computer

Name Embeddings and Online News Analysis Speaker: Junting Ye Department: Computer Science Advisor: Prof. Steven Skiena Outline Overview Name Embeddings Nationality Classification Ethnicity & Gender

509 views • 15 slides
Extensions of Embeddings in the 0 2 Turing Degrees. Antonio Montalb an. U. of Chicago

Extensions of Embeddings in the 0 2 Turing Degrees. Antonio Montalb an. U. of Chicago

Decidability results Our results Extensions of Embeddings in the 0 2 Turing Degrees. Antonio Montalb an. U. of Chicago Nanjing, May 2008 Joint work with Rod Downey, Noam Greenberg and Andy Lewis. Extensions of Embeddings in the 0

404 views • 24 slides
Word Embeddings Tutorial HILA GONEN PHD STUDENT AT YOAV GOLDBERGS LAB BAR ILAN UNIVERSITY

Word Embeddings Tutorial HILA GONEN PHD STUDENT AT YOAV GOLDBERGS LAB BAR ILAN UNIVERSITY

Word Embeddings Tutorial HILA GONEN PHD STUDENT AT YOAV GOLDBERGS LAB BAR ILAN UNIVERSITY 5/3/18 Outline NLP Intro Word representations and word embeddings Word2vec models Visualizing word embeddings Word2vec in

1.18k views • 63 slides
Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings Maksim Tkachenko

Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings Maksim Tkachenko

Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings Maksim Tkachenko and Chong Cher Chia and Hady W. Lauw Singapore Management University Word Embeddings Dense vectors of words Unsupervised training: GloVe,

522 views • 19 slides
IN5550: Neural Methods in Natural Language Processing Lecture 11/1 Contextualized embeddings

IN5550: Neural Methods in Natural Language Processing Lecture 11/1 Contextualized embeddings

IN5550: Neural Methods in Natural Language Processing Lecture 11/1 Contextualized embeddings Andrey Kutuzov University of Oslo 14 April 2020 1 Contents Brief Recap 1 Problems of static word embeddings 2 Solution: contextualized

372 views • 24 slides
Pattern-based Solutions to Limitations of Leading Word Embeddings Roy Schwartz University of

Pattern-based Solutions to Limitations of Leading Word Embeddings Roy Schwartz University of

Pattern-based Solutions to Limitations of Leading Word Embeddings Roy Schwartz University of Washington NLP Seminar, February 8 th , 2016 Joint work with Roi Reichart and Ari Rappoport Background Word embeddings are great! Problem

872 views • 84 slides
Deep Image-Text Embeddings Learning Deep Structure-Preserving Image-Text Embeddings (CVPR 2016)

Deep Image-Text Embeddings Learning Deep Structure-Preserving Image-Text Embeddings (CVPR 2016)

CS688 Paper Presentation 1 Deep Image-Text Embeddings Learning Deep Structure-Preserving Image-Text Embeddings (CVPR 2016) Woobin Im ( ) 2016-11-08 Sentence-to-image Retrieval Retrieval system Query text A cat next to a blue chair

594 views • 28 slides
Word, Sense and Contextualized Embeddings: Vector Representations of Meaning in NLP Jose

Word, Sense and Contextualized Embeddings: Vector Representations of Meaning in NLP Jose

Word, Sense and Contextualized Embeddings: Vector Representations of Meaning in NLP Jose Camacho-Collados Cardiff University, 18 March 2019 1 Outline Background Vector Space Models (word embeddings) Lexical resources Sense

1.33k views • 98 slides

More recommend