improving unsupervised acoustic word embeddings using
play

Improving Unsupervised Acoustic Word Embeddings using Speaker and - PowerPoint PPT Presentation

Feb 21, 2023 •354 likes •602 views

Improving Unsupervised Acoustic Word Embeddings using Speaker and Gender Information Lisa van Staden, Herman Kamper 31 January 2020 Zero-Resource Speech Processing Popular methods for speech processing rely on transcribed speech. Obtaining

  1. Improving Unsupervised Acoustic Word Embeddings using Speaker and Gender Information Lisa van Staden, Herman Kamper 31 January 2020

  2. Zero-Resource Speech Processing Popular methods for speech processing rely on transcribed speech. Obtaining transcriptions is expensive and not always possible. 1

  3. Zero-Resource Speech Processing Popular methods for speech processing rely on transcribed speech. Obtaining transcriptions is expensive and not always possible. 1

  4. Tasks in Zero-Resource Processing We don’t always need to predict text labels: • Query-by-Example Search: search speech using speech. • Unsupervised Term Discovery: Discover repeating patterns in speech. 2

  5. Tasks in Zero-Resource Processing We don’t always need to predict text labels: • Query-by-Example Search: search speech using speech. • Unsupervised Term Discovery: Discover repeating patterns in speech. 2

  6. Tasks in Zero-Resource Processing We don’t always need to predict text labels: • Query-by-Example Search: search speech using speech. • Unsupervised Term Discovery: Discover repeating patterns in speech. 2

  7. Speech Segment Comparison These tasks require comparing speech segments. The conventional method is Dynamic Time Warping. • Computationally expensive. 3

  8. Speech Segment Comparison These tasks require comparing speech segments. The conventional method is Dynamic Time Warping. • Computationally expensive. 3

  9. Acoustic Word Embeddings We want to map speech to these representation without using labels. 4

  10. Acoustic Word Embeddings We want to map speech to these representation without using labels. 4

  11. Speaker and Gender Information Acoustic properties of speech from difgerent speakers/genders difger. We want embeddings to be robust. 5 cat pan pan pun cat bat Speaker A Speaker B Female Male Male

  12. RNN (Correspondence) Autoencoder 6 x 1 x 2 x T GRU GRU GRU GRU embedding GRU GRU Encoder Decoder x 1 ' / y 1 ' x 2 ' / y 2 ' x T ' / y T '

  13. Speaker/Gender Conditioning 7 Speaker\Gender x 1 x 2 x T GRU GRU GRU GRU embedding GRU GRU Encoder Decoder x 1 ' / y 1 ' x 2 ' / y 2 ' x T ' / y T '

  14. 8 Adversarial Training Turn A X Encoder Turn B p Embedding Classifier Decoder X ' /Y '

  15. Speaker/Gender Classifier 9 z p Linear ReLU Dropout Softmax

  16. Evaluating Quality of AWEs Use the same-difgerent task to evaluate AWEs: • Measure if AWEs are similar given a threshold. • Calculate area under Precision vs Recall curve. 10

  17. Results 11 30.49 30.18 29.72 English 30 28.98 Xitsonga 25.53 25.38 25.19 25 22.72 22.52 Average Precision (%) 20 15 12.78 11.65 11.22 10 5 0 AE-Baseline AE-Top-1 AE-Top-2 CAE-Baseline CAE-Top-1 CAE-Top-2 Model Type

  18. Evaluate Speaker and Gender Predictability Analyse if the speaker and gender information has decreased: • Use speaker/gender classifier model. • Evaluate accuracy. 12

  19. Average Precision vs Speaker/Gender Predictability AE CAE 13 27.0 32.0 26.8 31.5 Average Precision Average Precision 26.6 31.0 26.4 26.2 30.5 26.0 30.0 72 74 76 78 80 82 84 68 70 72 74 76 78 80 82 84 Speaker Predictability Speaker Predictability 27.0 32.0 26.8 31.5 Average Precision Average Precision 26.6 31.0 26.4 26.2 30.5 26.0 30.0 89.5 90.0 90.5 91.0 91.5 92.0 92.5 93.0 93.5 88 89 90 91 92 93 Gender Predictability Gender Predictability

  20. Conclusions • English data shows marginal improvement by incorporating speaker information. • Best Xitsonga model shows 22% improvement. • It’s diffjcult to remove speaker and gender information. • Future work ... 14

  21. Conclusions • English data shows marginal improvement by incorporating speaker information. • Best Xitsonga model shows 22% improvement. • It’s diffjcult to remove speaker and gender information. • Future work ... 14

  22. Conclusions • English data shows marginal improvement by incorporating speaker information. • Best Xitsonga model shows 22% improvement. • It’s diffjcult to remove speaker and gender information. • Future work ... 14

  23. Conclusions • English data shows marginal improvement by incorporating speaker information. • Best Xitsonga model shows 22% improvement. • It’s diffjcult to remove speaker and gender information. • Future work ... 14

Recommend

Unsupervised speech processing using acoustic word embeddings Herman Kamper School of

Unsupervised speech processing using acoustic word embeddings Herman Kamper School of

Unsupervised speech processing using acoustic word embeddings Herman Kamper School of Informatics, University of Edinburgh TTI at Chicago MLSLP 2016: Spotlight invited talk Unsupervised speech processing Speech recognition applications

912 views • 68 slides
Deep convolutional acoustic word embeddings using word-pair side information Herman Kamper 1 ,

Deep convolutional acoustic word embeddings using word-pair side information Herman Kamper 1 ,

Deep convolutional acoustic word embeddings using word-pair side information Herman Kamper 1 , Weiran Wang 2 , Karen Livescu 2 1 CSTR and ILCC, School of Informatics, University of Edinburgh, UK 2 Toyota Technological Institute at Chicago, USA

1.03k views • 76 slides
Evaluation methods for unsupervised word embeddings EMNLP 2015 Tobias Schnabel, Igor Labutov,

Evaluation methods for unsupervised word embeddings EMNLP 2015 Tobias Schnabel, Igor Labutov,

Evaluation methods for unsupervised word embeddings EMNLP 2015 Tobias Schnabel, Igor Labutov, David Mimno and Thorsten Joachims Cornell University September 19th, 2015 Evaluation methods for unsupervised word embeddings Motivation How

762 views • 26 slides
Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings

Word Embeddings Natural Language Processing VU (706.230) - Andi Rexha 02/04/2020 Word Embeddings Agenda Traditional NLP Word Embeddings-1 Word Embeddings-2 Text preprocessing Topic Modeling ELMo Bag-of-words model Neural

900 views • 64 slides
Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings Maksim Tkachenko

Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings Maksim Tkachenko

Searching for the X-Factor: Exploring Corpus Subjectivity for Word Embeddings Maksim Tkachenko and Chong Cher Chia and Hady W. Lauw Singapore Management University Word Embeddings Dense vectors of words Unsupervised training: GloVe,

522 views • 19 slides
USING PSEUDO-SENSES FOR IMPROVING THE EXTRACTION OF SYNONYMS FROM WORD EMBEDDINGS Olivier Ferret

USING PSEUDO-SENSES FOR IMPROVING THE EXTRACTION OF SYNONYMS FROM WORD EMBEDDINGS Olivier Ferret

USING PSEUDO-SENSES FOR IMPROVING THE EXTRACTION OF SYNONYMS FROM WORD EMBEDDINGS Olivier Ferret CONTEXT AND OBJECTIVES Context semantic specialization of word embeddings most approaches following Retrofitting [Faruqui et al.,

341 views • 11 slides
Removing Nuisance Variables from Acoustic Word Embeddings Obtaining transcriptions is expensive

Removing Nuisance Variables from Acoustic Word Embeddings Obtaining transcriptions is expensive

Lisa van Staden Removing Nuisance Variables from Acoustic Word Embeddings Obtaining transcriptions is expensive and not always possible. Popular methods for speech processing rely on transcribed speech. 1 Low-Resource Speech and Language

631 views • 7 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

422 views • 17 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

2k views • 24 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

1.25k views • 75 slides
Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word

Word Embeddings CS 6956: Deep Learning for NLP Overview Representing meaning Word embeddings: Early work Word embeddings via language models Word2vec and Glove Evaluating embeddings Design choices and open questions 1

828 views • 15 slides
Improving the Compositionality of Word Embeddings M ASTER T HESIS Supervisors: Author: dr.

Improving the Compositionality of Word Embeddings M ASTER T HESIS Supervisors: Author: dr.

Improving the Compositionality of Word Embeddings M ASTER T HESIS Supervisors: Author: dr. Evangelos K ANOULAS Thijs S CHEEPERS dr. Efstratios G AVVES Truely understanding A far out goal for Artificial Intelligence What is your name? Such a

746 views • 71 slides
Acoustic word embeddings for ASR error detection Sahar Ghannay, Yannick Estve, Nathalie Camelin

Acoustic word embeddings for ASR error detection Sahar Ghannay, Yannick Estve, Nathalie Camelin

Acoustic word embeddings for ASR error detection Sahar Ghannay, Yannick Estve, Nathalie Camelin and Paul Delglise LIUM, IICC, Universit du Maine Le Mans, France INTERSPEECH 2016, SAN FRANCISCO 10/09/2016 1. Introduction Introduction 2.

697 views • 16 slides
Motivation Current Scenario : Rising interest in vector space word embeddings and their use, given

Motivation Current Scenario : Rising interest in vector space word embeddings and their use, given

IMPROVING WORD EMBEDDINGS USING MULTIPLE WORD PROTOTYPES CS671A Course Project : Under Prof. Amitabha Mukerjee Anurendra Kumar Nishant Rai 15th October Indian Institute of Technology Kanpur Motivation Current Scenario : Rising interest in

505 views • 16 slides
Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme:

Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme:

Word embeddings Rappel Embeddings ( pas Word Embeddings ) Est une lookup table Formalisme: Index dun mot: w i Table dembeddings (lookup matrix): V Embedding: e i e i = V( w i ) Reprsentation dun mot

780 views • 47 slides
Improving Distributional Similarity with Lessons Learned from Word Embeddings Authors: Omar Levy,

Improving Distributional Similarity with Lessons Learned from Word Embeddings Authors: Omar Levy,

Improving Distributional Similarity with Lessons Learned from Word Embeddings Authors: Omar Levy, Yoav Goldberg, Ido Dagan Presentation: Collin Gress Motivation u We want to do NLP tasks. How do we represent words? Motivation u We want to do

935 views • 66 slides
Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds

Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds

Mixed membership word embeddings: Corpus-specific embeddings without big data James Foulds University of California, San Diego Southern California Machine Learning Symposium, Caltech, 11/18/2018 Word Embeddings Language models which learn

537 views • 32 slides
Word Embeddings through Hellinger PCA Rmi Lebret and Ronan Collobert Idiap Research Institute /

Word Embeddings through Hellinger PCA Rmi Lebret and Ronan Collobert Idiap Research Institute /

Word Embeddings through Hellinger PCA Rmi Lebret and Ronan Collobert Idiap Research Institute / EPFL EACL, 29 April 2014 Word Embeddings Continuous vector-space models. Represent word meanings with vectors capturing semantic +

831 views • 46 slides
Word Embeddings Tutorial HILA GONEN PHD STUDENT AT YOAV GOLDBERGS LAB BAR ILAN UNIVERSITY

Word Embeddings Tutorial HILA GONEN PHD STUDENT AT YOAV GOLDBERGS LAB BAR ILAN UNIVERSITY

Word Embeddings Tutorial HILA GONEN PHD STUDENT AT YOAV GOLDBERGS LAB BAR ILAN UNIVERSITY 5/3/18 Outline NLP Intro Word representations and word embeddings Word2vec models Visualizing word embeddings Word2vec in

1.18k views • 63 slides
Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview

Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview

Word Embeddings Revisited: Contextual Embeddings CS 6956: Deep Learning for NLP Overview Word types and tokens Training contextual embeddings Embeddings from Language Models (ELMo) 1 Overview Word types and tokens Training

670 views • 40 slides
Symmetric Pattern Based Word Embeddings for Improved Word Similarity Prediction Roy Schwartz + ,

Symmetric Pattern Based Word Embeddings for Improved Word Similarity Prediction Roy Schwartz + ,

Symmetric Pattern Based Word Embeddings for Improved Word Similarity Prediction Roy Schwartz + , Roi Reichart * and Ari Rappoport + + The Hebrew University, * Technion IIT CoNLL 2015 Symmetric Pattern Based Word Embeddings for 2 Improved Word

917 views • 49 slides
#TagSpace: Semantic Embeddings from Hashtags Jason Weston, Sumit Chopra, Keith Adams 2014

#TagSpace: Semantic Embeddings from Hashtags Jason Weston, Sumit Chopra, Keith Adams 2014

#TagSpace: Semantic Embeddings from Hashtags Jason Weston, Sumit Chopra, Keith Adams 2014 Jack Lanchantin Motivation Word and document embeddings are difficult to learn Most current techniques use unsupervised methods word2vec

540 views • 19 slides
Pattern-based Solutions to Limitations of Leading Word Embeddings Roy Schwartz University of

Pattern-based Solutions to Limitations of Leading Word Embeddings Roy Schwartz University of

Pattern-based Solutions to Limitations of Leading Word Embeddings Roy Schwartz University of Washington NLP Seminar, February 8 th , 2016 Joint work with Roi Reichart and Ari Rappoport Background Word embeddings are great! Problem

872 views • 84 slides
Word, Sense and Contextualized Embeddings: Vector Representations of Meaning in NLP Jose

Word, Sense and Contextualized Embeddings: Vector Representations of Meaning in NLP Jose

Word, Sense and Contextualized Embeddings: Vector Representations of Meaning in NLP Jose Camacho-Collados Cardiff University, 18 March 2019 1 Outline Background Vector Space Models (word embeddings) Lexical resources Sense

1.33k views • 98 slides

More recommend