gaussian mixture latent vector grammars
play

Gaussian Mixture Latent Vector Grammars Yanpeng Zhao Liwen Zhang - PowerPoint PPT Presentation

Aug 14, 2022 •114 likes •422 views

Gaussian Mixture Latent Vector Grammars Yanpeng Zhao Liwen Zhang Kewei Tu School of Information Science and Technology ShanghaiTech University ACL 2018 PCFG Parsing & Limitations Constituency Parsing Grammars Prob.

  1. Gaussian Mixture Latent Vector Grammars Yanpeng Zhao Liwen Zhang Kewei Tu School of Information Science and Technology ShanghaiTech University ACL 2018

  2. PCFG Parsing & Limitations Constituency Parsing Grammars Prob. S → NP VP 1.0 VP → V NP 0.2 S NP → DT NP 0.5 PCFGs NP VP NP → NP NP 0.3 … … He found me P V NP P → He 1.0 P → me 1.0 P V → found 1.0 He found me … … � 2

  3. PCFG Parsing & Limitations Constituency Parsing Grammars Prob. S → NP VP 1.0 VP → V NP 0.2 S NP → DT NP 0.5 PCFGs NP VP NP → NP NP 0.3 … … He found me P V NP P → He 1.0 P → me 1.0 P V → found 1.0 He found me … … S T 1 = NP VP P V NP P He found me � 3

  4. PCFG Parsing & Limitations Constituency Parsing Grammars Prob. S → NP VP 1.0 VP → V NP 0.2 S NP → DT NP 0.5 PCFGs NP VP NP → NP NP 0.3 … … He found me P V NP P → He 1.0 P → me 1.0 P V → found 1.0 He found me … … S S T 1 = T 2 = NP VP NP VP P V NP P V NP P P He found me me found He � 4

  5. PCFG Parsing & Limitations Constituency Parsing Grammars Prob. S → NP VP 1.0 VP → V NP 0.2 S NP → DT NP 0.5 PCFGs NP VP NP → NP NP 0.3 … … He found me P V NP P → He 1.0 P → me 1.0 P V → found 1.0 He found me … … S S P ( T 1 ) = P ( T 2 ) T 1 = T 2 = NP VP NP VP P V NP P V NP He found me 6 = me found He P P limitations He found me me found He � 5

  6. Tree Annotation & Lexicalization SˆROOT S-found NPˆS VPˆS-BD-Z NP-He VP-found P-Z V-BD-Z NPˆVP-O P-He V-found NP-me P-O P-me He found me He found me [Johnson. 1998; Klein et al. 2003] [Collins. 1997; Charniak. 2000] Tree Annotation Lexicalization � 6

  7. Latent Variable Grammars (LVGs) [Matsuzaki et al. 2005; Petrov et al. 2007] S[ x 1 ] S 0 NP[ x 2 ] VP[ x 4 ] NP 2 VP 0 P[ x 3 ] V[ x 5 ] NP[ x 6 ] P 0 V 1 NP 2 P[ x 7 ] P 3 He found me He found me Annotated parse tree A subtype parse tree • Discrete latent variables x 1 , x 2 , x 3 . . . • Model a finite number of nonterminal subtypes � 7

  8. Refining Syntactic Category SˆROOT S-found S[ x 1 ] NPˆS VPˆS-BD-Z NP-He VP-found NP[ x 2 ] VP[ x 4 ] P-Z V-BD-Z NPˆVP-O P-He V-found NP-me P[ x 3 ] V[ x 5 ] NP[ x 6 ] P-O P-me P[ x 7 ] He found me He found me He found me Tree annotation Lexicalization LVGs [Klein et al. 2003] [Charniak. 2000] [Petrov et al. 2007] F1: 85.7 F1: 89.6 F1: 90.1 � 8

  9. Latent Vector Grammars (LVeGs) S[ x 1 ] S [0 . 05 , 0 . 2] NP[ x 2 ] VP[ x 4 ] NP [0 . 4 , 1 . 7] VP [3 . 4 , 0 . 9] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P [0 . 3 , 2 . 1] V [0 . 1 , 0 . 2] NP [1 . 3 , 0 . 7] P[ x 7 ] P [0 . 5 , 1 . 4] He found me He found me Annotated parse tree A subtype parse tree • Continuous latent vectors x 1 , x 2 , x 3 . . . • Model an infinite number of nonterminal subtypes � 9

  10. LVGs vs LVeGs (Latent Variable Grammars) (Latent Vector Grammars) S[ x 1 ] S[ x 1 ] NP[ x 2 ] VP[ x 4 ] NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] P[ x 7 ] He found me He found me � 10

  11. LVGs vs LVeGs (Latent Variable Grammars) (Latent Vector Grammars) S[ x 1 ] S[ x 1 ] NP[ x 2 ] VP[ x 4 ] NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] P[ x 7 ] He found me He found me Discrete latent variables Continuous latent vectors x 1 , x 2 , x 3 . . . x 1 , x 2 , x 3 . . . Annotated rule NP[ x 2 ] � P[ x 3 ] Annotated rule NP[ x 2 ] � P[ x 3 ] � 11

  12. LVGs vs LVeGs (Latent Variable Grammars) (Latent Vector Grammars) S[ x 1 ] S[ x 1 ] NP[ x 2 ] VP[ x 4 ] NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] P[ x 7 ] He found me He found me Discrete latent variables Continuous latent vectors x 1 , x 2 , x 3 . . . x 1 , x 2 , x 3 . . . Annotated rule Annotated rule NP[ x 2 ] � P[ x 3 ] NP[ x 2 ] � P[ x 3 ] A finite set of subtype rules An infinite set of subtype rules Subtype rule Subtype rule NP [2 . 3 , 1 . 7] � P [0 . 4 , 1 . 3] NP 0 � P 1 � 12

  13. LVGs vs LVeGs (Latent Variable Grammars) (Latent Vector Grammars) S[ x 1 ] S[ x 1 ] NP[ x 2 ] VP[ x 4 ] NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] P[ x 7 ] He found me He found me Discrete latent variables Continuous latent vectors x 1 , x 2 , x 3 . . . x 1 , x 2 , x 3 . . . Annotated rule Annotated rule NP[ x 2 ] � P[ x 3 ] NP[ x 2 ] � P[ x 3 ] A finite set of subtype rules An infinite set of subtype rules Subtype rule Subtype rule NP [2 . 3 , 1 . 7] � P [0 . 4 , 1 . 3] NP 0 � P 1 Prob. of the subtype rule: Weight density of the subtype rule: W NP � P ( x 2 = [2 . 3 , 1 . 7] , x 3 = [0 . 4 , 1 . 3]) W NP � P ( x 2 = 0 , x 3 = 1) � 13

  14. LVGs as Special Case of LVeGs S[ x 1 ] Discrete Variables NP[ x 2 ] VP[ x 4 ] P[ x 3 ] V[ x 5 ] NP[ x 6 ] P[ x 7 ] One-hot Vectors He found me � 14

Recommend

Kernel Methods for Regression Support Vector Regression Gaussian Mixture Regression Gaussian

Kernel Methods for Regression Support Vector Regression Gaussian Mixture Regression Gaussian

MACHINE LEARNING MACHINE LEARNING MACHINE LEARNING Kernel Methods for Regression Support Vector Regression Gaussian Mixture Regression Gaussian Process Regression 1 MACHINE LEARNING 2012 MACHINE LEARNING MACHINE LEARNING Problem

945 views • 71 slides
MLE 04-09-2019 For Gaussian and Mixture Gaussian Models Instructor - Sriram Ganapathy

MLE 04-09-2019 For Gaussian and Mixture Gaussian Models Instructor - Sriram Ganapathy

E9 205 Machine Learning for Signal Processing MLE 04-09-2019 For Gaussian and Mixture Gaussian Models Instructor - Sriram Ganapathy (sriramg@iisc.ac.in) Teaching Assistant - Prachi Singh (prachisingh@iisc.ac.in). Finding the parameters of the

390 views • 14 slides
Multivariate Gaussian Mean vector: Covariance matrix: 2 1 Conditioning a Gaussian Joint

Multivariate Gaussian Mean vector: Covariance matrix: 2 1 Conditioning a Gaussian Joint

Readings: K&F: 6.1, 6.2, 6.3, 14.1, 14.2, 14.3, 14.4, Kalman Filters Gaussian MNs Graphical Models 10708 Carlos Guestrin Carnegie Mellon University December 1 st , 2008 1 Multivariate Gaussian Mean vector: Covariance matrix: 2 1

257 views • 11 slides
Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Mixture Models: definition Mixture models: Linear supper-position of mixtures or components | =

1.04k views • 39 slides
Using Gaussian Mixture Models to Detect Figurative Language in Context Linlin Li and Caroline

Using Gaussian Mixture Models to Detect Figurative Language in Context Linlin Li and Caroline

Introduction Using Gaussian Mixture Model to Detect Figurative Language Evaluating the GMM Approach Conclusion Using Gaussian Mixture Models to Detect Figurative Language in Context Linlin Li and Caroline Sporleder Cluster of Excellence, MMCI

411 views • 21 slides
Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested

Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested

Case Study: Approximate Bayesian Inference for Latent Gaussian Models by Using Integrated Nested Laplace Approximations BIOSTAT830: Graphical Models December 08, 2016 Introduction - INLA Inference for latent Gaussian Markov random field

605 views • 35 slides
Deep Gaussian Mixture Models Cinzia Viroli (University of Bologna, Italy) joint with Geoff

Deep Gaussian Mixture Models Cinzia Viroli (University of Bologna, Italy) joint with Geoff

Deep Gaussian Mixture Models Cinzia Viroli (University of Bologna, Italy) joint with Geoff McLachlan (University of Queensland, Australia) JOCLAD 2018, Lisbona, April 5th, 2018 Outline Deep Learning Mixture Models Deep Gaussian Mixture

915 views • 66 slides
Applied Machine Learning Expectation Maximization for Mixture of Gaussians Siamak Ravanbakhsh

Applied Machine Learning Expectation Maximization for Mixture of Gaussians Siamak Ravanbakhsh

Applied Machine Learning Expectation Maximization for Mixture of Gaussians Siamak Ravanbakhsh COMP 551 (Fall 2020) Learning objectives what is a latent variable model? Gaussian mixture model the intuition behind the Expectation-Maximization

530 views • 18 slides
Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models

Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models Yarin Gal Mark van der Wilk Carl E. Rasmussen yg279@cam.ac.uk June 25th, 2014 Outline Gaussian process regression and latent variable

1.18k views • 54 slides
Piecewise Bounds for Estimating Bernoulli- Logistic Latent Gaussian Models Mohammad Emtiyaz Khan

Piecewise Bounds for Estimating Bernoulli- Logistic Latent Gaussian Models Mohammad Emtiyaz Khan

Piecewise Bounds for Estimating Bernoulli- Logistic Latent Gaussian Models Mohammad Emtiyaz Khan Joint work with Benjamin Marlin, and Kevin Murphy University of British Columbia June 29, 2011 Piecewise Bounds for Binary Latent Gaussian Models

694 views • 34 slides
E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes

E ffi cient Modeling of Latent Information in Supervised Learning using Gaussian Processes Mauricio A. Zhenwen Dai Alvarez Neil D. Lawrence Gaussian Process Approximation Workshop, 2017 Motivation I Machine learning has been very

685 views • 21 slides
The Structure of the Worst Noise in Gaussian Vector Broadcast Channels Wei Yu University of

The Structure of the Worst Noise in Gaussian Vector Broadcast Channels Wei Yu University of

The Structure of the Worst Noise in Gaussian Vector Broadcast Channels Wei Yu University of Toronto March 19, 2003 DIMACS Workshop on Network Information Theory Outline Sum capacity of Gaussian vector broadcast channels. Complete

554 views • 38 slides
Joint Optimisation of Tandem Systems using Gaussian Mixture Density Neural Network Discriminative

Joint Optimisation of Tandem Systems using Gaussian Mixture Density Neural Network Discriminative

Joint Optimisation of Tandem Systems using Gaussian Mixture Density Neural Network Discriminative Sequence Training Chao Zhang and Phil Woodland March 8, 2017 Cambridge University Engineering Department Introduction Tandem Systems as Mixture

926 views • 17 slides
Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon T. Georgiou and Allen

Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon T. Georgiou and Allen

Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon T. Georgiou and Allen Tannenbaum Presented by: Zach Lucas Intro and Motivation A mixture model is a probabilistic model describing properties of populations with

357 views • 35 slides
Latent class analysis and finite mixture models with Stata Isabel Canette Principal

Latent class analysis and finite mixture models with Stata Isabel Canette Principal

Latent class analysis and finite mixture models with Stata Isabel Canette Principal Mathematician and Statistician StataCorp LLC 2017 Stata Users Group Meeting Madrid, October 19th, 2017 Introduction Latent class analysis (LCA)

658 views • 32 slides
Parsing with Compositional Vector Grammars BY RICHARD SOCHER, JOHN BAUER, CHRISTOPHER D. MANNING,

Parsing with Compositional Vector Grammars BY RICHARD SOCHER, JOHN BAUER, CHRISTOPHER D. MANNING,

Parsing with Compositional Vector Grammars BY RICHARD SOCHER, JOHN BAUER, CHRISTOPHER D. MANNING, ANDREW Y. NG PRESENT BY YUNCHENG WU Outline Introduction Related Works Compositional Vector Grammars (this paper) Experiments Takeaways

888 views • 34 slides
ELEN E6884 - Topics in Signal Processing Recap Topic: Speech Recognition Gaussian Mixture

ELEN E6884 - Topics in Signal Processing Recap Topic: Speech Recognition Gaussian Mixture

Outline of Todays Lecture ELEN E6884 - Topics in Signal Processing Recap Topic: Speech Recognition Gaussian Mixture Models - A Gaussian Mixture Models - B Lecture 3

339 views • 13 slides
Vine copula mixture models and clustering for non-Gaussian data Statistical Methods in Machine

Vine copula mixture models and clustering for non-Gaussian data Statistical Methods in Machine

Vine copula mixture models and clustering for non-Gaussian data Statistical Methods in Machine Learning Prof. Claudia Czado Ozge Sahin <ozge.sahin@tum.de> Bernoulli-IMS One World Symposium August 2020 Finite mixture models k

467 views • 14 slides
Expectation Maximization Greg Mori - CMPT 419/726 Bishop PRML Ch. 9 K-Means Gaussian Mixture

Expectation Maximization Greg Mori - CMPT 419/726 Bishop PRML Ch. 9 K-Means Gaussian Mixture

K-Means Gaussian Mixture Models Expectation-Maximization Expectation Maximization Greg Mori - CMPT 419/726 Bishop PRML Ch. 9 K-Means Gaussian Mixture Models Expectation-Maximization Learning Parameters to Probability Distributions We

1.37k views • 91 slides
Lecture 20 Lecture 20 Nov 12 th 2008 Clustering with Mixture of Gaussians Clustering with Mixture

Lecture 20 Lecture 20 Nov 12 th 2008 Clustering with Mixture of Gaussians Clustering with Mixture

Lecture 20 Lecture 20 Nov 12 th 2008 Clustering with Mixture of Gaussians Clustering with Mixture of Gaussians Underlying model for data: assumes a generative process: Each cluster follows a Gaussian distribution First choose which

302 views • 19 slides
-- Deep Neural Network or Gaussian Mixture Model? Dong Yu Microsoft Research Thanks to my

-- Deep Neural Network or Gaussian Mixture Model? Dong Yu Microsoft Research Thanks to my

Who Can Understand Your Speech Better -- Deep Neural Network or Gaussian Mixture Model? Dong Yu Microsoft Research Thanks to my collaborators: Li Deng, Frank Seide, Gang Li, Mike Seltzer, Jinyu Li, Jui-Ting Huang, Kaisheng Yao, Adam

967 views • 53 slides
EM Algorithm 09-09-2019 For Mixture Gaussian Models Instructor - Sriram Ganapathy

EM Algorithm 09-09-2019 For Mixture Gaussian Models Instructor - Sriram Ganapathy

E9 205 Machine Learning for Signal Processing EM Algorithm 09-09-2019 For Mixture Gaussian Models Instructor - Sriram Ganapathy (sriramg@iisc.ac.in) Teaching Assistant - Prachi Singh (prachisingh@iisc.ac.in). Basics of Information Theory

606 views • 23 slides
Outline Latent Variable Generative Models Cooperative Vector Quantizer Model Model

Outline Latent Variable Generative Models Cooperative Vector Quantizer Model Model

2/4/2020 CS 3750 Advanced Machine Learning Latent Variable Generative Models II Ahmad Diab AHD23@cs.pitt.edu Feb 4, 2020 Based on slides of Professor Milos Hauskrecht Outline Latent Variable Generative Models Cooperative Vector

435 views • 20 slides
Convergence of latent mixing measures in finite and infinite mixture models Long Nguyen

Convergence of latent mixing measures in finite and infinite mixture models Long Nguyen

Convergence of latent mixing measures in finite and infinite mixture models Long Nguyen Department of Statistics University of Michigan BNP Workshop, ICERM 2012 Nguyen@BNP (ICERM12) 1 / 29 Outline Identifiability and consistency in

509 views • 36 slides

More recommend