learning interpretable continuous time models of latent
play

Learning interpretable continuous-time models of latent stochastic - PowerPoint PPT Presentation

Aug 22, 2022 •337 likes •671 views

Learning interpretable continuous-time models of latent stochastic dynamical systems Lea Duncker, Gerg o Bohner, Julien Boussard, Maneesh Sahani Gatsby Computational Neuroscience Unit University College London ICML June 12, 2019 nonlinear

  1. Learning interpretable continuous-time models of latent stochastic dynamical systems Lea Duncker, Gerg˝ o Bohner, Julien Boussard, Maneesh Sahani Gatsby Computational Neuroscience Unit University College London ICML June 12, 2019

  2. nonlinear stochastic dynamical system 1 x 2 0 0 1 x 1

  3. nonlinear stochastic dynamical system 1 1 x 2 x 2 0 0 0 0 1 1 x 1 x 1 √ dx x = f x f f ( x x x ) dt + Σ dW W W

  4. nonlinear stochastic dynamical system 1 x 2 0 0 1 x 1 √ dx x x = f f f ( x x x ) dt + Σ dW W W

  5. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 0 0 0 0 1 1 x 1 x 1 √ dx x = f x f f ( x x x ) dt + Σ dW W W

  6. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 f f ( x x x ) = f f f ( s s s ) + ∇ x f f f ( x ) | x = s ( x x x − s s s ) + ... f 0 0 0 0 1 1 x 1 x 1 √ dx x = f x f f ( x x x ) dt + Σ dW W W

  7. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 f f ( x x x ) = f f ( s f s ) + ∇ x f s f f ( x ) | x = s ( x x x − s s s ) + ... f ≈ J J J ( x x x − s s s ) Jacobian matrix 0 0 0 0 1 1 x 1 x 1 √ dx x = f x f f ( x x x ) dt + Σ dW W W

  8. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 f f ( x x x ) = f f ( s f s ) + ∇ x f s f ( x ) | x = s ( x f x x − s s s ) + ... f ≈ J J J ( x x x − s s s ) Jacobian matrix 0 0 0 0 1 1 x 1 x 1 √ dx x x = f f f ( x x x ) dt + Σ dW W W

  9. nonlinear stochastic dynamical system 1 1 0 f f f ( s s s ) = 0 0 fixed point x 2 x 2 f f ( x x x ) = f f ( s f s s ) + ∇ x f f f ( x ) | x = s ( x x − s x s s ) + ... f ≈ J J ( x J x − s x s s ) Jacobian matrix 0 0 interpretability: 0 0 1 1 x 1 x 1 ◮ stability analysis √ ◮ locally linearised dynamics dx x = f x f f ( x x x ) dt + Σ dW W W ◮ . . .

  10. unevenly sampled high-d observations y 1 . . . y N time

  11. unevenly sampled high-d observations y 1 . . . y N time

  12. unevenly sampled high-d observations y 1 . . . y N time � y y ( t i ) � = g ( Cx y x x ( t i ) + d d d )

  13. latent low-d unevenly sampled stochastic process high-d observations y 1 . . x ( t ) . y N time time √ � y y ( t i ) � = g ( Cx y x ( t i ) + d x d d ) dx x x = f f f ( x x ) dt + x Σ dW W W

  14. latent low-d unevenly sampled stochastic process high-d observations y 1 . . x ( t ) . y N time time √ � y y ( t i ) � = g ( Cx y x ( t i ) + d x d d ) dx x x = f f f ( x x ) dt + x Σ dW W W

  15. GP conditioned on latent low-d unevenly sampled interpretable features stochastic process high-d observations y 1 2 . . f ( x ) x ( t ) . 0 − 2 y N − 2 0 2 x time time √ � y y ( t i ) � = g ( Cx f k ∼ GP ( µ θ ( x x x ) , k θ ( x x x , x x ′ )) x y x x ( t i ) + d d d ) dx x = f x f f ( x x x ) dt + Σ dW W W

  16. Variational q ( x x f f ) = q x ( x x x ) q f ( f f f ) x , f Bayes Archambeau (2007), Titsias (2009)

  17. Variational q ( x x f f ) = q x ( x x x ) q f ( f f f ) x , f Bayes Archambeau (2007), Titsias (2009)

  18. Variational q ( x x f f ) = q x ( x x x ) q f ( f f ) f x , f Bayes Gaussian Process Dynamics � � � � = P � u u u ,θ θ θ q u ( u u u ) du u u � N ( u u u | m m m u , S S S u ) sparse approx. with inducing variables Archambeau (2007), Titsias (2009)

  19. Variational q ( x x f f ) = q x ( x x x ) q f ( f f f ) x , f Bayes Latent SDE path Gaussian Process Dynamics √ � x = ( − A ( t ) x x x + b x b ( t )) dt + b Σ dW W � � dx W � = P � u u u ,θ θ θ q u ( u u ) du u u u � q ( x x x ( t )) = N ( x x ( t ) | m x m m x ( t ) , S S S x ( t )) m x = − A ( t ) m ˙ m m m m x + b b ( t ) b S x A ( t ) T + Σ ˙ S x = − A ( t ) S S x − S S S S S N ( u u u | m m u , S m S S u ) sparse approx. Gaussian approx. with inducing variables with Markov structure Archambeau (2007), Titsias (2009)

  20. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1

  21. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 x 2 0 − 2 − 2 0 2 x 1 true dynamics

  22. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 2 x 2 x 2 0 0 − 2 − 2 − 2 − 2 0 0 2 2 x 1 x 1 true dynamics

  23. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 2 0 5 x 2 x 2 0 0 0 − 5 19 0 1 samples of high-d output − 2 − 2 on 20 trials − 2 − 2 0 0 2 2 x 1 x 1 true dynamics

  24. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 x 2 0 − 2 − 2 0 2 x 1 true dynamics

  25. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 2 x 2 x 2 0 0 − 2 − 2 − 2 0 2 − 2 0 2 x 1 x 1 true dynamics learnt dynamics

  26. Example: Van der Pol’s Oscillator x ) = 2 τ ( x 1 − 1 3 x 3 dynamics: f 1 ( x 1 − x 2 ) x f 2 ( x x x ) = τ 2 x 1 2 2 2 2 x 2 x 2 x 2 x 2 0 0 0 0 − 2 − 2 − 2 − 2 − 2 − 2 0 0 2 2 − 2 − 2 0 0 2 2 x 1 x 1 x 1 x 1 true dynamics learnt dynamics

  27. limit cycles 2 2 x 2 x 2 0 0 − 2 − 2 − 2 0 2 − 2 0 2 x 1 x 1 true dynamics learnt dynamics

  28. limit cycles double-well dynamics trial 1 2 2 true 1 . 5 learnt 1 x ( t ) f ( x ) 0 0 . 0 x 2 x 2 0 0 − 1 − 1 . 5 − 1 . 5 0 . 0 1 . 5 − 2 − 2 x 0 1 2 3 − 2 0 2 − 2 0 2 t x 1 x 1 true dynamics learnt dynamics

  29. limit cycles double-well dynamics trial 1 2 2 true 1 . 5 learnt 1 x ( t ) f ( x ) 0 0 . 0 x 2 x 2 0 0 − 1 − 1 . 5 − 1 . 5 0 . 0 1 . 5 − 2 − 2 x 0 1 2 3 − 2 0 2 − 2 0 2 t x 1 x 1 true dynamics learnt dynamics multivariate point process 1 40 neuron number x 2 20 0 0 . 0 0 . 5 1 . 0 t 0 1 x 1

  30. limit cycles double-well dynamics trial 1 2 2 true 1 . 5 learnt 1 x ( t ) f ( x ) 0 0 . 0 x 2 x 2 0 0 − 1 − 1 . 5 − 1 . 5 0 . 0 1 . 5 − 2 − 2 x 0 1 2 3 − 2 0 2 − 2 0 2 t x 1 x 1 true dynamics learnt dynamics multivariate point process chemical reaction dynamics 1 0 . 8 40 neuron number 180 0 . 6 10 x 2 x 2 0 . 4 20 5 235 0 . 2 0 0 0 0.5 0 . 0 0 . 0 0 . 5 1 . 0 t 0 . 0 0 . 2 0 . 4 0 . 6 0 . 8 0 1 x 1 x 1

  31. limit cycles double-well dynamics trial 1 2 2 true 1 . 5 learnt 1 x ( t ) f ( x ) 0 0 . 0 x 2 x 2 0 0 − 1 − 1 . 5 − 1 . 5 0 . 0 1 . 5 − 2 − 2 x 0 1 2 3 − 2 0 2 − 2 0 2 t x 1 x 1 true dynamics learnt dynamics Tonight @ Pacific Ballroom Poster #229 multivariate point process chemical reaction dynamics 1 0 . 8 40 neuron number 180 0 . 6 10 x 2 x 2 0 . 4 20 5 235 0 . 2 0 0 0 0.5 0 . 0 0 . 0 0 . 5 1 . 0 t 0 . 0 0 . 2 0 . 4 0 . 6 0 . 8 0 1 x 1 x 1

  32. Gerg˝ o Bohner Julien Boussard Maneesh Sahani

Recommend

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada,

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada,

DeepMDP Learning Latent Space Continuous Models for Representation Learning Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, Marc G. Bellemare Simple Representations for RL 2 12 DeepMDP Latent Space Model: Neural networks MDP:

573 views • 17 slides
STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in

STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in

STA 4273 / CSC 2547 Spring 2018 Learning Discrete Latent Structure What recently became easy in machine learning? Training continuous latent- variable models (VAEs, GANs) to produce large images Training large supervised models with

1.26k views • 61 slides
Latent Classification Models Classification in continuous domains Helge Langseth and Thomas D.

Latent Classification Models Classification in continuous domains Helge Langseth and Thomas D.

Latent Classification Models Classification in continuous domains Helge Langseth and Thomas D. Nielsen LCM p.1/ ?? Outline Probabilistic classifiers Nave Bayes classifiers Relaxing the assumptions Latent classification

654 views • 29 slides
Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1, Autumn

2.08k views • 171 slides
Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models for Time Series Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1, Autumn

1.9k views • 168 slides
Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges

Learning Latent Variable Models through Tensor Methods Anima Anandkumar U.C. Irvine Challenges in Unsupervised Learning Learn a latent variable model without labeled examples. E.g. topic models, hidden Markov models, Gaussian mixtures,

763 views • 61 slides
1 Latent variable models In the next section we will discuss latent variable models for

1 Latent variable models In the next section we will discuss latent variable models for

Statistical Modeling and Analysis of Neural Data (NEU 560) Princeton University, Spring 2018 Jonathan Pillow Lecture 16 notes: Latent variable models and EM Tues, 4.10 1 Latent variable models In the next section we will discuss latent

271 views • 4 slides
Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Anima Anandkumar UC Irvine Joint work with Majid Janzamin Rong Ge UC Irvine Microsoft Research Latent Variable Probabilistic Models Latent (hidden) variable h R k ,

769 views • 47 slides
Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead Cynthia Rudin, Duke University Presenters : Sreya Dutta Roy, Ziqian Lin 1 Overview Introduction Explainable ML Vs

1.04k views • 39 slides
Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding (NNSE)

Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding (NNSE)

1 Learning Effective and Interpretable Semantic Models using Non-Negative Sparse Embedding (NNSE) Brian Murphy, Partha Talukdar, Tom Mitchell Machine Learning Department Carnegie Mellon University 2 Distributional Semantic Modeling 2

1.03k views • 86 slides
A Nonparametric N-Gram Topic Model with Interpretable Latent Topics Shoaib Jameel and Wai Lam

A Nonparametric N-Gram Topic Model with Interpretable Latent Topics Shoaib Jameel and Wai Lam

A Nonparametric N-Gram Topic Model with Interpretable Latent Topics Shoaib Jameel and Wai Lam The Chinese University of Hong Kong . One line summary of the work . We will see how maintaining the order of words in a document helps improve

620 views • 9 slides
Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of

Guaranteed Learning of Latent Variable Models through Tensor Methods Furong Huang University of Maryland furongh@cs.umd.edu ACM SIGMETRICS Tutorial 2018 1 / 75 Tutorial Topic Learning algorithms for latent variable models based on

1.96k views • 180 slides
From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box Models Huge volumes of data Solve task Computing power Deep Neural Network Information (implicit) ICIP18 Tutorial on Interpretable Deep

540 views • 33 slides
Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine

Learning Overcomplete Latent Variable Models through Tensor Methods Majid Janzamin UC Irvine Joint work with Anima Anandkumar Rong Ge UC Irvine Microsoft Research Latent Variable Modeling Goal: Discover hidden effects from observed

1.27k views • 115 slides
Learning Interpretable Models Expressed in Linear Temporal Logic Alberto Camacho 1 , 2 and Sheila

Learning Interpretable Models Expressed in Linear Temporal Logic Alberto Camacho 1 , 2 and Sheila

Learning Interpretable Models Expressed in Linear Temporal Logic Alberto Camacho 1 , 2 and Sheila McIlraith 1 , 2 1 Department of Computer Science, University of Toronto 2 Vector Institute { acamacho, sheila } @cs.toronto.edu ICAPS 2019 July 13,

457 views • 19 slides
Outline Latent Variable Generative Models Cooperative Vector Quantizer Model Model

Outline Latent Variable Generative Models Cooperative Vector Quantizer Model Model

2/4/2020 CS 3750 Advanced Machine Learning Latent Variable Generative Models II Ahmad Diab AHD23@cs.pitt.edu Feb 4, 2020 Based on slides of Professor Milos Hauskrecht Outline Latent Variable Generative Models Cooperative Vector

435 views • 20 slides
Latent Variable Models CS3750 Xiaoting Li 1 Out utli line Latent Variable Models

Latent Variable Models CS3750 Xiaoting Li 1 Out utli line Latent Variable Models

2/4/2020 Latent Variable Models CS3750 Xiaoting Li 1 Out utli line Latent Variable Models Expectation Maximization Algorithm (EM) Factor Analysis Probabilistic Principal Component Analysis Model Formulation Maximum

654 views • 18 slides
Part III: Latent Tree Models Le Song ICML 2012 Tutorial on Spectral Algorithms for Latent

Part III: Latent Tree Models Le Song ICML 2012 Tutorial on Spectral Algorithms for Latent

Spectral Algorithms for Latent Variable Models Part III: Latent Tree Models Le Song ICML 2012 Tutorial on Spectral Algorithms for Latent Variable Models, Edinburgh, UK Joint work with Mariya Ishteva, Ankur Parikh, Eric Xing, Byron Boots , Geoff

548 views • 40 slides
Latent Variable models for GWAs Oliver Stegle Machine Learning and Computational Biology Research

Latent Variable models for GWAs Oliver Stegle Machine Learning and Computational Biology Research

Latent Variable models for GWAs Oliver Stegle Machine Learning and Computational Biology Research Group Max-Planck-Institutes T ubingen, Germany September 2011 O. Stegle Latent variable models for GWAs T ubingen 1 Motivation Why

1.07k views • 84 slides
Latent Variables and Real-Time Forecasting in DSGE Models with Occasionally Binding

Latent Variables and Real-Time Forecasting in DSGE Models with Occasionally Binding

Latent Variables and Real-Time Forecasting in DSGE Models with Occasionally Binding Constraints. Can Non-Linearity Improve Our Understanding of the Great Recession? Massimo Giovannini and Marco Ratto JRC, European Commission Outline I

746 views • 30 slides
Empirical Study of the Benefits of Overparameterization in Learning Latent Variable Models

Empirical Study of the Benefits of Overparameterization in Learning Latent Variable Models

Empirical Study of the Benefits of Overparameterization in Learning Latent Variable Models Rares-Darius Buhai 1 , Yoni Halpern 2 , Yoon Kim 3 , Andrej Risteski 4 , David Sontag 1 1 MIT, 2 Google, 3 Harvard, 4 CMU Overparameterization = training a

395 views • 21 slides
Advanced Tools in Macroeconomics Continuous time models (and methods) Pontus Rendahl August 24,

Advanced Tools in Macroeconomics Continuous time models (and methods) Pontus Rendahl August 24,

Advanced Tools in Macroeconomics Continuous time models (and methods) Pontus Rendahl August 24, 2016 Introduction In this lecture we will take a look at models in continuous, as opposed to discrete, time. There are some advantages and

1.27k views • 74 slides
IMLI: An Incremental Framework for MaxSAT-Based Learning of Interpretable Classification Rules

IMLI: An Incremental Framework for MaxSAT-Based Learning of Interpretable Classification Rules

IMLI: An Incremental Framework for MaxSAT-Based Learning of Interpretable Classification Rules Bishwamittra Ghosh Joint work with Kuldeep S. Meel 1 Applications of Machine Learning 2 Example Dataset 3 Representation of an interpretable

1.24k views • 55 slides
Probabilistic & Unsupervised Learning Latent Variable Models Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models Maneesh Sahani

Probabilistic & Unsupervised Learning Latent Variable Models Maneesh Sahani maneesh@gatsby.ucl.ac.uk Gatsby Computational Neuroscience Unit, and MSc ML/CSML, Dept Computer Science University College London Term 1, Autumn 2018 Exponential

1.03k views • 101 slides

More recommend