part 3 probabilistic inference in graphical models
play

Part 3: Probabilistic Inference in Graphical Models Sebastian - PowerPoint PPT Presentation

Jan 30, 2023 •254 likes •951 views

Belief Propagation Variational Inference Sampling Break Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert Colorado Springs, 25th June 2011 Sebastian Nowozin and Christoph H. Lampert Part 3:

  1. Belief Propagation Variational Inference Sampling Break Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert Colorado Springs, 25th June 2011 Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  2. Belief Propagation Variational Inference Sampling Break Belief Propagation Belief Propagation ◮ “Message-passing” algorithm ◮ Exact and optimal for tree-structured graphs ◮ Approximate for cyclic graphs Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  3. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � Z = exp( − E ( y )) y ∈Y � � � � = exp( − E ( y i , y j , y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  4. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � Z = exp( − E ( y )) y ∈Y � � � � = exp( − E ( y i , y j , y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  5. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � Z = exp( − E ( y )) y ∈Y � � � � = exp( − E ( y i , y j , y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  6. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � � � � Z = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  7. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � � � � Z = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  8. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains Y i Y j Y k Y l F G H � � � � Z = exp( − ( E F ( y i , y j ) + E G ( y j , y k ) + E H ( y k , y l ))) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  9. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains r H → Y k ∈ R Y k Y i Y j Y k Y l F G H � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) Z y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � �� � r H → Yk ( y k ) � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) r H → Y k ( y k ) y i ∈Y i y j ∈Y j y k ∈Y i Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  10. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains r H → Y k ∈ R Y k Y i Y j Y k Y l F G H � � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) exp( − E H ( y k , y l )) Z y i ∈Y i y j ∈Y j y k ∈Y k y l ∈Y l � �� � r H → Yk ( y k ) � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) r H → Y k ( y k ) y i ∈Y i y j ∈Y j y k ∈Y i Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  11. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains r G → Y j ∈ R Y j Y i Y j Y k Y l F G H � � � Z = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) r H → Y k ( y k ) y i ∈Y i y j ∈Y j y k ∈Y k � �� � r G → Yj ( y j ) � � = exp( − E F ( y i , y j )) r G → Y j ( y j ) y i ∈Y i y j ∈Y j Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  12. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Chains r G → Y j ∈ R Y j Y i Y j Y k Y l F G H � � � Z = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) r H → Y k ( y k ) y i ∈Y i y j ∈Y j y k ∈Y k � �� � r G → Yj ( y j ) � � = exp( − E F ( y i , y j )) r G → Y j ( y j ) y i ∈Y i y j ∈Y j Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  13. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees Y i Y j Y k Y l F G H I Y m � Z = exp( − E ( y )) y ∈Y � � � � � = exp( − ( E F ( y i , y j ) + · · · + E I ( y k , y m ))) y i ∈Y i y j ∈Y i y k ∈Y i y l ∈Y i y m ∈Y m Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  14. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees Y i Y j Y k Y l F G H I Y m � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) · Z y i ∈Y i y j ∈Y j y k ∈Y k            �  �   exp( − E H ( y k , y l )) exp( − E I ( y k , y m )) ·         y l ∈Y l y m ∈Y m   � �� � � �� � r H → Yk ( y k ) r I → Yk ( y k ) Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  15. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees r H → Y k ( y k ) Y i Y j Y k Y l F G H r I → Y k ( y k ) I Y m � � � = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) · Z y i ∈Y i y j ∈Y j y k ∈Y k ( r H → Y k ( y k ) · r I → Y k ( y k )) Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  16. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees q Y k → G ( y k ) r H → Y k ( y k ) Y i Y j Y k Y l F G H r I → Y k ( y k ) I Y m � � � Z = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) · y i ∈Y i y j ∈Y j y k ∈Y k ( r H → Y k ( y k ) · r I → Y k ( y k )) � �� � q Yk → G ( y k ) Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  17. Belief Propagation Variational Inference Sampling Break Belief Propagation Example: Inference on Trees q Y k → G ( y k ) r H → Y k ( y k ) Y i Y j Y k Y l F G H r I → Y k ( y k ) I Y m � � � Z = exp( − E F ( y i , y j )) exp( − E G ( y j , y k )) q Y k → G ( y k ) y i ∈Y i y j ∈Y j y k ∈Y k Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

  18. Belief Propagation Variational Inference Sampling Break Belief Propagation Factor Graph Sum-Product Algorithm ◮ “Message”: pair of vectors at each . . . factor graph edge ( i , F ) ∈ E r F → Y i . . . 1. r F → Y i ∈ R Y i : factor-to-variable Y i . . . message F 2. q Y i → F ∈ R Y i : variable-to-factor . . . q Y i → F message ◮ Algorithm iteratively update messages ◮ After convergence: Z and µ F can be obtained from the messages Sebastian Nowozin and Christoph H. Lampert Part 3: Probabilistic Inference in Graphical Models

Recommend

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh

Probabilistic Graphical Models Probabilistic Graphical Models MAP inference Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives MAP inference and its complexity exact & approximate MAP inference max-product and max-sum

957 views • 74 slides
Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak Ravanbakhsh Fall 2019 Learning objective Learning objective an intuition for inference in graphical models why is it difficult? exact inference by

1.49k views • 62 slides
Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications

Graphical models Why All probabilistic inference and learning amount at repeated applications of the sum and product rules Probabilistic graphical models are graphical representations of the qualitative aspects of probability distribu-

551 views • 23 slides
Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family &

Probabilistic Graphical Models Probabilistic Graphical Models Exponential family & Variational Inference I Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives entropy exponential family distribution duality in

1.38k views • 85 slides
Approximate Inference: Mean Field Methods Probabilistic Graphical Models (10- Probabilistic

Approximate Inference: Mean Field Methods Probabilistic Graphical Models (10- Probabilistic

School of Computer Science Approximate Inference: Mean Field Methods Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 17, Nov 12, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X 2

672 views • 18 slides
Probabilistic Graphical Models Inference & Learning in DL Zhiting Hu Lecture 19, March 29,

Probabilistic Graphical Models Inference & Learning in DL Zhiting Hu Lecture 19, March 29,

Probabilistic Graphical Models Inference & Learning in DL Zhiting Hu Lecture 19, March 29, 2017 Reading: 1 Deep Generative Models l Explicit probabilistic models Provide an explicit parametric specification of the distribution of l

514 views • 50 slides
Probabilistic Graphical Models Probabilistic Graphical Models Markov Chain Monte Carlo Inference

Probabilistic Graphical Models Probabilistic Graphical Models Markov Chain Monte Carlo Inference

Probabilistic Graphical Models Probabilistic Graphical Models Markov Chain Monte Carlo Inference Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov chains the idea behind Markov Chain Monte Carlo (MCMC) two

1.03k views • 71 slides
Probabilistic Graphical Models Lecture 15 Inference as Optimization CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 15 Inference as Optimization CS/CNS/EE 155 Andreas

Probabilistic Graphical Models Lecture 15 Inference as Optimization CS/CNS/EE 155 Andreas Krause Announcements Homework 3 due next Monday (Nov 23) Project poster session on Friday December 4 (tentative) Final writeup (8 pages NIPS

805 views • 36 slides
Probabilistic Graphical Models Guest Lecture by Narges Razavian Machine Learning Class April 14

Probabilistic Graphical Models Guest Lecture by Narges Razavian Machine Learning Class April 14

Probabilistic Graphical Models Guest Lecture by Narges Razavian Machine Learning Class April 14 2017 Today What is probabilistic graphical model and why it is useful? Bayesian Networks Basic Inference Generative Models Fancy Inference

785 views • 51 slides
Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of

Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of

Approximate inference: Sampling methods Probabilistic Graphical Models Sharif University of Technology Spring 2018 Soleymani Approximate inference Approximate inference techniques Deterministic approximation Variational algorithms

1.96k views • 97 slides
Monte Carlo methods Draw random samples from the desired distribution Yield a stochastic

Monte Carlo methods Draw random samples from the desired distribution Yield a stochastic

School of Computer Science Approximate Inference: Monte Carlo Inference Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 18, Nov 19, 2007 Receptor A Receptor A X 1 X 1 X 1 Receptor B Receptor B X

503 views • 24 slides
Variational inference Probabilistic Graphical Models Sharif University of Technology Soleymani

Variational inference Probabilistic Graphical Models Sharif University of Technology Soleymani

Variational inference Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Some slides are adapted from Xing s slides Exact methods for inference Variable elimination Message Passing: shared terms

1.2k views • 84 slides
15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this

15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this

15-780 Graduate Artificial Intelligence: Probabilistic inference J. Zico Kolter (this lecture) and Nihar Shah Carnegie Mellon University Spring 2020 1 Outline Probabilistic graphical models Probabilistic inference Exact inference

665 views • 39 slides
Slides Set 11 (part a): Sampling Techniques for Probabilistic and Deterministic Graphical models

Slides Set 11 (part a): Sampling Techniques for Probabilistic and Deterministic Graphical models

Algorithms for Reasoning with graphical models Slides Set 11 (part a): Sampling Techniques for Probabilistic and Deterministic Graphical models Rina Dechter (Reading Darwiche chapter 15, related papers) slides11a 828X 2019 Sampling

1.05k views • 78 slides
Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov networks: how it represents a prob. dist. independence assumptions

941 views • 57 slides
Slides Set 9(part b): Sampling Techniques for Probabilistic and Deterministic Graphical models

Slides Set 9(part b): Sampling Techniques for Probabilistic and Deterministic Graphical models

Reasoning with graphical models Slides Set 9(part b): Sampling Techniques for Probabilistic and Deterministic Graphical models Rina Dechter (Reading Darwiche chapter 15, cutset-sampling paper posted) slides 9b 276 2020 Overview 1.

1.02k views • 63 slides
Variable Elimination Probabilistic Graphical Models Sharif University of Technology Spring 2018

Variable Elimination Probabilistic Graphical Models Sharif University of Technology Spring 2018

Exact Inference: Variable Elimination Probabilistic Graphical Models Sharif University of Technology Spring 2018 Soleymani Probabilistic Inference and Learning We now have compact representations of probability distributions

697 views • 31 slides
Computer Science Let me be provocative Probabilistic graphical models is how we do probabilistic

Computer Science Let me be provocative Probabilistic graphical models is how we do probabilistic

Computer Science Let me be provocative Probabilistic graphical models is how we do probabilistic AI! Graphical models of variable-level (in)dependence are a broken abstraction. [VdB KRR15] Let me be provocative Probabilistic graphical models

782 views • 63 slides
Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian

Probabilistic Graphical Models Probabilistic Graphical Models Parameter learning in Bayesian networks Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives likelihood function and MLE role of the sufficient statistics MLE for

644 views • 60 slides
& Exact inference for Gaussian networks Probabilistic Graphical Models Sharif University of

& Exact inference for Gaussian networks Probabilistic Graphical Models Sharif University of

Factor analysis & Exact inference for Gaussian networks Probabilistic Graphical Models Sharif University of Technology Spring 2016 Soleymani Multivariate Gaussian distribution 2 /2 1/2 exp{ 1 1 2

1.13k views • 43 slides
Factor analysis & Exact inference for Gaussian networks Probabilistic Graphical Models

Factor analysis & Exact inference for Gaussian networks Probabilistic Graphical Models

Factor analysis & Exact inference for Gaussian networks Probabilistic Graphical Models Sharif University of Technology Spring 2017 Soleymani Multivariate Gaussian distribution 2 /2 1/2 exp{ 1 1 2

808 views • 43 slides
The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models

The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models

School of Computer Science The Elimination Algorithm Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 4, Sep 26, 2007 Receptor A Receptor A Receptor B Receptor B X 1 X 1 X 1 X 2 X 2 X 2 Eric Xing

457 views • 17 slides
A road map to more complex dynamic models discrete discrete continuous Y Y Y discrete

A road map to more complex dynamic models discrete discrete continuous Y Y Y discrete

School of Computer Science State Space Models Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 13, part II Nov 5th, 2007 Receptor A Receptor A Receptor B Receptor B X 1 X 1 X 1 X 2 X 2 X 2 Eric

360 views • 21 slides
Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that

Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that

Probabilistic Graphical Models CMSC 678 UMBC Probabilistic Graphical Models A graph G that represents a probability distribution over random variables 1 , , Probabilistic Graphical Models A graph G that represents a

1.74k views • 129 slides

More recommend