optimal transport for gaussian mixture models
play

Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon - PowerPoint PPT Presentation

Feb 01, 2024 •7 likes •357 views

Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon T. Georgiou and Allen Tannenbaum Presented by: Zach Lucas Intro and Motivation A mixture model is a probabilistic model describing properties of populations with

  1. Optimal transport for Gaussian mixture models Yongxin Chen, Tryphon T. Georgiou and Allen Tannenbaum Presented by: Zach Lucas

  2. Intro and Motivation A mixture model is a probabilistic model describing properties of populations with subpopulations. To study OMT on certain submanifolds of probability densities. To retain the nice properties of OMT, herein, an explicit OMT framework on Gaussian mixture models is used. Data is sparsely distributed among subgroups. The difference between data within a subgroup is way less significant than that between subgroups.

  3. Gaussian Mixture Model (GMM) Learning Unsupervised clustering based on naive Bayes

  4. GMM: Expectation - Maximization (EM)

  5. GMM: Expectation

  6. GMM: Maximization

  7. GMM: 2D example https://www.youtube.com/watch?v=B36fzChfyGU

  8. OMT Background

  9. OMT Background: Kantorovich Coupling The unique optimal transport T is the gradient of a convex function

  10. OMT Background: Kantorovich The optimal coupling based on the transport map T in (2), where Id is the identity map. The square root of the minimum of the cost defines a Riemannian metric on , known as the Wasserstein metric . On this Riemannian-type manifold, the geodesic curve is given by Displacement Interpolation

  11. Gaussian marginal distributions Denote the mean and covariance of Let X, Y be two Gaussian random vectors associated with respectively. Our new cost from (1) becomes

  12. Gaussian marginal distributions The constraint is semidefinite constraint, so the (6) is a semidefinite programming (SDP). It turns out that the minimum is achieved by the unique minimizer in closed-form: With minimum value

  13. Gaussian marginal distributions Displacement Interpolation as a Gaussian: Wasserstein Distance can be extended to singular Gaussian distributions

  14. OMT for GMM Space of distributions: We view it as a discrete distribution on the Wasserstein space of Gaussian distributions:

  15. OMT for GMM The discrete OMT problem:

  18. Geodesic

  20. Notes This is due to the fact that the restriction to the submanifold induces suboptimality in the transport plan. d is a very good approximation of W2 if the variances of the Gaussian components are small compared with the differences between the means. Only (9) must be solved to compute a new distance, which is extremely efficient with small distributions

  21. Barycenter of GMM

  22. Barycenter of GMM Solve with fixed point iteration: Remark: unrealistic to solve (14) for more than 3 dimensions for both general and gaussian distributions

  23. Barycenter of GMM Modified problem: Let as a discrete measure on

  24. Barycenter of GMM The optimal v is gaussian. Denote the set of all such minimerzers For some probability vector The number of element N is bounded above by

  25. Barycenter of GMM Barycenter with

  26. Numerical Examples

  28. Geodesic

  34. Barycenter

Recommend

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M.

Gaussian Mixture Models & EM CE-717: Machine Learning Sharif University of Technology M. Soleymani Fall 2016 Mixture Models: definition Mixture models: Linear supper-position of mixtures or components | =

1.04k views • 39 slides
MLE 04-09-2019 For Gaussian and Mixture Gaussian Models Instructor - Sriram Ganapathy

MLE 04-09-2019 For Gaussian and Mixture Gaussian Models Instructor - Sriram Ganapathy

E9 205 Machine Learning for Signal Processing MLE 04-09-2019 For Gaussian and Mixture Gaussian Models Instructor - Sriram Ganapathy (sriramg@iisc.ac.in) Teaching Assistant - Prachi Singh (prachisingh@iisc.ac.in). Finding the parameters of the

390 views • 14 slides
Deep Gaussian Mixture Models Cinzia Viroli (University of Bologna, Italy) joint with Geoff

Deep Gaussian Mixture Models Cinzia Viroli (University of Bologna, Italy) joint with Geoff

Deep Gaussian Mixture Models Cinzia Viroli (University of Bologna, Italy) joint with Geoff McLachlan (University of Queensland, Australia) JOCLAD 2018, Lisbona, April 5th, 2018 Outline Deep Learning Mixture Models Deep Gaussian Mixture

915 views • 66 slides
Using Gaussian Mixture Models to Detect Figurative Language in Context Linlin Li and Caroline

Using Gaussian Mixture Models to Detect Figurative Language in Context Linlin Li and Caroline

Introduction Using Gaussian Mixture Model to Detect Figurative Language Evaluating the GMM Approach Conclusion Using Gaussian Mixture Models to Detect Figurative Language in Context Linlin Li and Caroline Sporleder Cluster of Excellence, MMCI

411 views • 21 slides
Clustering: Models and Algorithms Shikui Tu 2019-03-07 1 Outline Gaussian Mixture Models

Clustering: Models and Algorithms Shikui Tu 2019-03-07 1 Outline Gaussian Mixture Models

Clustering: Models and Algorithms Shikui Tu 2019-03-07 1 Outline Gaussian Mixture Models (GMM) Expectation-Maximization (EM) for maximum likelihood Model selection, Bayesian learning 2 From distance to probability distance

383 views • 37 slides
Vine copula mixture models and clustering for non-Gaussian data Statistical Methods in Machine

Vine copula mixture models and clustering for non-Gaussian data Statistical Methods in Machine

Vine copula mixture models and clustering for non-Gaussian data Statistical Methods in Machine Learning Prof. Claudia Czado Ozge Sahin <ozge.sahin@tum.de> Bernoulli-IMS One World Symposium August 2020 Finite mixture models k

467 views • 14 slides
z A single Gaussian might be a poor fit . . . . . Simplest form is 2 layer . . . ... .

z A single Gaussian might be a poor fit . . . . . Simplest form is 2 layer . . . ... .

Overview Hidden Variable Models 1: Mixture Models Hidden variable models Mixture models Chris Williams Mixtures of Gaussians Aside: Kullback-Leibler divergence School of Informatics, University of Edinburgh The EM algorithm October 2008

395 views • 6 slides
EM Algorithm 09-09-2019 For Mixture Gaussian Models Instructor - Sriram Ganapathy

EM Algorithm 09-09-2019 For Mixture Gaussian Models Instructor - Sriram Ganapathy

E9 205 Machine Learning for Signal Processing EM Algorithm 09-09-2019 For Mixture Gaussian Models Instructor - Sriram Ganapathy (sriramg@iisc.ac.in) Teaching Assistant - Prachi Singh (prachisingh@iisc.ac.in). Basics of Information Theory

606 views • 23 slides
Models for Probability Distributions and Density Functions 1 General Concepts Parametric:

Models for Probability Distributions and Density Functions 1 General Concepts Parametric:

Models for Probability Distributions and Density Functions 1 General Concepts Parametric: E.g., Gaussian, Gamma, Binomial Non-Parametric: E.g., kernel estimates Intermediate models: Mixture Models 2 Gaussian Mixture Model

382 views • 22 slides
Probabilistic & Unsupervised Learning Beyond linear-Gaussian and Mixture models Maneesh

Probabilistic & Unsupervised Learning Beyond linear-Gaussian and Mixture models Maneesh

Probabilistic & Unsupervised Learning Beyond linear-Gaussian and Mixture models Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1,

1.41k views • 124 slides
Probabilistic & Unsupervised Learning Beyond linear-Gaussian and Mixture models Maneesh

Probabilistic & Unsupervised Learning Beyond linear-Gaussian and Mixture models Maneesh

Probabilistic & Unsupervised Learning Beyond linear-Gaussian and Mixture models Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1,

1.46k views • 120 slides
ELEN E6884 - Topics in Signal Processing Recap Topic: Speech Recognition Gaussian Mixture

ELEN E6884 - Topics in Signal Processing Recap Topic: Speech Recognition Gaussian Mixture

Outline of Todays Lecture ELEN E6884 - Topics in Signal Processing Recap Topic: Speech Recognition Gaussian Mixture Models - A Gaussian Mixture Models - B Lecture 3

339 views • 13 slides
EM Algorithm 11-09-2019 For Mixture Gaussian Models Instructor - Sriram Ganapathy

EM Algorithm 11-09-2019 For Mixture Gaussian Models Instructor - Sriram Ganapathy

E9 205 Machine Learning for Signal Processing EM Algorithm 11-09-2019 For Mixture Gaussian Models Instructor - Sriram Ganapathy (sriramg@iisc.ac.in) Teaching Assistant - Prachi Singh (prachisingh@iisc.ac.in). EM Algorithm For GMMs EM

368 views • 18 slides
Lecture 3 Gaussian Mixture Models and Introduction to HMMs Michael Picheny, Bhuvana

Lecture 3 Gaussian Mixture Models and Introduction to HMMs Michael Picheny, Bhuvana

Lecture 3 Gaussian Mixture Models and Introduction to HMMs Michael Picheny, Bhuvana Ramabhadran, Stanley F . Chen, Markus Nussbaum-Thom Watson Group IBM T.J. Watson Research Center Yorktown Heights, New York, USA

1.31k views • 106 slides
Lecture 3 Gaussian Mixture Models and Introduction to HMMs Michael Picheny, Bhuvana

Lecture 3 Gaussian Mixture Models and Introduction to HMMs Michael Picheny, Bhuvana

Lecture 3 Gaussian Mixture Models and Introduction to HMMs Michael Picheny, Bhuvana Ramabhadran, Stanley F . Chen IBM T.J. Watson Research Center Yorktown Heights, New York, USA {picheny,bhuvana,stanchen}@us.ibm.com 24 September 2012

1.45k views • 113 slides
Expectation Maximization Greg Mori - CMPT 419/726 Bishop PRML Ch. 9 K-Means Gaussian Mixture

Expectation Maximization Greg Mori - CMPT 419/726 Bishop PRML Ch. 9 K-Means Gaussian Mixture

K-Means Gaussian Mixture Models Expectation-Maximization Expectation Maximization Greg Mori - CMPT 419/726 Bishop PRML Ch. 9 K-Means Gaussian Mixture Models Expectation-Maximization Learning Parameters to Probability Distributions We

1.37k views • 91 slides
Bernoulli Mixture Models Victor Medina Researcher at SBIF DataCamp Mixture Models in R The

Bernoulli Mixture Models Victor Medina Researcher at SBIF DataCamp Mixture Models in R The

DataCamp Mixture Models in R MIXTURE MODELS IN R Bernoulli Mixture Models Victor Medina Researcher at SBIF DataCamp Mixture Models in R The handwritten digits dataset DataCamp Mixture Models in R Continuous versus discrete variables

440 views • 41 slides
Structure of mixture models Victor Medina Researcher at SBIF DataCamp Mixture Models in R

Structure of mixture models Victor Medina Researcher at SBIF DataCamp Mixture Models in R

DataCamp Mixture Models in R MIXTURE MODELS IN R Structure of mixture models Victor Medina Researcher at SBIF DataCamp Mixture Models in R Description of mixture models 1. Which is the suitable probability distribution? Get familiar with

1.07k views • 36 slides
Direct Fitting of Gaussian Mixture Models Leonid Keselman , Martial Hebert Robotics Institute

Direct Fitting of Gaussian Mixture Models Leonid Keselman , Martial Hebert Robotics Institute

Direct Fitting of Gaussian Mixture Models Leonid Keselman , Martial Hebert Robotics Institute Carnegie Mellon University May 29, 2019 https://github.com/leonidk/direct_gmm 1 Representations of 3D data Point Cloud Point Cloud Triangular +

875 views • 49 slides
How to use Gaussian mixture models on patches for solving image inverse problems Workshop

How to use Gaussian mixture models on patches for solving image inverse problems Workshop

How to use Gaussian mixture models on patches for solving image inverse problems Workshop MixStatSeq Antoine Houdard LTCI, Tlcom ParisTech MAP5, Universit Paris Descartes antoine.houdard@telecom-paristech.fr houdard.wp.imt.fr Joint

799 views • 45 slides
Language, Dialect, and Speaker Recognition Using Gaussian Mixture Models on the Cell Processor

Language, Dialect, and Speaker Recognition Using Gaussian Mixture Models on the Cell Processor

Language, Dialect, and Speaker Recognition Using Gaussian Mixture Models on the Cell Processor Nicolas Malyska, Sanjeev Mohindra, Karen Lauro, Douglas Reynolds, and Jeremy Kepner {nmalyska, smohindra, karen.lauro, reynolds, kepner}@ll.mit.edu

720 views • 60 slides
Gaussian Mixture Models Instructor: John Thickstun Discussion Board: Available on Canvas Zoom

Gaussian Mixture Models Instructor: John Thickstun Discussion Board: Available on Canvas Zoom

Gaussian Mixture Models Instructor: John Thickstun Discussion Board: Available on Canvas Zoom Link: Available on Canvas Instructor Contact: thickstn@cs.washington.edu Course Webpage: https://courses.cs.washington.edu/courses/cse599i/20au/ CSE

380 views • 17 slides
Mixture Models Simulation-based Estimation Michel Bierlaire Transport and Mobility Laboratory

Mixture Models Simulation-based Estimation Michel Bierlaire Transport and Mobility Laboratory

Mixture Models Simulation-based Estimation Michel Bierlaire Transport and Mobility Laboratory School of Architecture, Civil and Environmental Engineering Ecole Polytechnique F ed erale de Lausanne M. Bierlaire (TRANSP-OR ENAC EPFL)

1.11k views • 78 slides
Estimating Gaussian Mixture Models from Data with Missing Features by Daniel McMichael CSSIP

Estimating Gaussian Mixture Models from Data with Missing Features by Daniel McMichael CSSIP

Estimating Gaussian Mixture Models from Data with Missing Features by Daniel McMichael CSSIP Missing Data In classification we frequently see to classify objects using vectors of measured features. Sometimes these features are missing: [ 0

478 views • 10 slides

More recommend