part 2 introduction to graphical models
play

Part 2: Introduction to Graphical Models Sebastian Nowozin and - PowerPoint PPT Presentation

Feb 24, 2024 •233 likes •809 views

Graphical Models Factor Graphs Test-time Inference Training Software Part 2: Introduction to Graphical Models Sebastian Nowozin and Christoph H. Lampert Providence, 21st June 2012 Sebastian Nowozin and Christoph H. Lampert Part 2:

  1. Graphical Models Factor Graphs Test-time Inference Training Software Part 2: Introduction to Graphical Models Sebastian Nowozin and Christoph H. Lampert Providence, 21st June 2012 Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  2. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Introduction ◮ Model: relating observations x to quantities of interest y f ◮ Example 1: given RGB image x , infer depth y for each pixel X Y ◮ Example 2: given RGB image x , infer presence and positions y of all objects f : X → Y shown Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  3. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Introduction ◮ Model: relating observations x to quantities of interest y f ◮ Example 1: given RGB image x , infer depth y for each pixel X Y ◮ Example 2: given RGB image x , infer presence and positions y of all objects f : X → Y shown X : image, Y : object annotations Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  4. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Introduction ◮ General case: mapping x ∈ X to y ∈ Y ◮ Graphical models are a concise x language to define this mapping f ( x ) ◮ Mapping can be ambiguous : X Y measurement noise, lack of well-posedness (e.g. occlusions) f : X → Y ◮ Probabilistic graphical models: define form p ( y | x ) or p ( x , y ) for all y ∈ Y Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  5. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Introduction ◮ General case: mapping x ∈ X to y ∈ Y ? ◮ Graphical models are a concise language to define this mapping x ◮ Mapping can be ambiguous : ? X Y measurement noise, lack of well-posedness (e.g. occlusions) p ( Y | X = x ) ◮ Probabilistic graphical models: define form p ( y | x ) or p ( x , y ) for all y ∈ Y Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  6. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Graphical Models A graphical model defines ◮ a family of probability distributions over a set of random variables, ◮ by means of a graph, ◮ so that the random variables satisfy conditional independence assumptions encoded in the graph. Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  7. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Graphical Models A graphical model defines ◮ a family of probability distributions over a set of random variables, ◮ by means of a graph, ◮ so that the random variables satisfy conditional independence assumptions encoded in the graph. Popular classes of graphical models, ◮ Undirected graphical models (Markov random fields), ◮ Directed graphical models (Bayesian networks), ◮ Factor graphs, ◮ Others: chain graphs, influence diagrams, etc. Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  8. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Bayesian Networks ◮ Graph: G = ( V , E ), E ⊂ V × V Y i Y j ◮ directed ◮ acyclic ◮ Variable domains Y i Y k ◮ Factorization � p ( Y = y ) = p ( y i | y pa G ( i ) ) Y l i ∈ V A simple Bayes net over distributions, by conditioning on parent nodes. ◮ Example p ( Y = y ) = p ( Y l = y l | Y k = y k ) p ( Y k = y k | Y i = y i , Y j = y j ) p ( Y i = y i ) p ( Y j = y j ) . ◮ Family of distributions Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  9. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Bayesian Networks ◮ Graph: G = ( V , E ), E ⊂ V × V Y i Y j ◮ directed ◮ acyclic ◮ Variable domains Y i Y k ◮ Factorization � p ( Y = y ) = p ( y i | y pa G ( i ) ) Y l i ∈ V A simple Bayes net over distributions, by conditioning on parent nodes. ◮ Example p ( Y = y ) = p ( Y l = y l | Y k = y k ) p ( Y k = y k | Y i = y i , Y j = y j ) p ( Y i = y i ) p ( Y j = y j ) . ◮ Family of distributions Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  10. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Undirected Graphical Models Y i Y j Y k ◮ = Markov random field (MRF) = Markov network A simple MRF ◮ Graph: G = ( V , E ), E ⊂ V × V ◮ undirected, no self-edges ◮ Variable domains Y i ◮ Factorization over potentials ψ at cliques , p ( y ) = 1 � ψ C ( y C ) Z C ∈C ( G ) ◮ Constant Z = � � C ∈C ( G ) ψ C ( y C ) y ∈Y ◮ Example p ( y ) = 1 Z ψ i ( y i ) ψ j ( y j ) ψ l ( y l ) ψ i , j ( y i , y j ) Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  11. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Undirected Graphical Models Y i Y j Y k ◮ = Markov random field (MRF) = Markov network A simple MRF ◮ Graph: G = ( V , E ), E ⊂ V × V ◮ undirected, no self-edges ◮ Variable domains Y i ◮ Factorization over potentials ψ at cliques , p ( y ) = 1 � ψ C ( y C ) Z C ∈C ( G ) ◮ Constant Z = � � C ∈C ( G ) ψ C ( y C ) y ∈Y ◮ Example p ( y ) = 1 Z ψ i ( y i ) ψ j ( y j ) ψ l ( y l ) ψ i , j ( y i , y j ) Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  12. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Example 1 Y i Y j Y k ◮ Cliques C ( G ): set of vertex sets V ′ with V ′ ⊆ V , E ∩ ( V ′ × V ′ ) = V ′ × V ′ ◮ Here C ( G ) = {{ i } , { i , j } , { j } , { j , k } , { k }} ◮ p ( y ) = 1 Z ψ i ( y i ) ψ j ( y j ) ψ l ( y l ) ψ i , j ( y i , y j ) Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  13. Graphical Models Factor Graphs Test-time Inference Training Software Graphical Models Example 2 Y i Y j Y k Y l ◮ Here C ( G ) = 2 V : all subsets of V are cliques ◮ p ( y ) = 1 � ψ A ( y A ) . Z A ∈ 2 { i , j , k , l } Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  14. Graphical Models Factor Graphs Test-time Inference Training Software Factor Graphs Factor Graphs ◮ Graph: G = ( V , F , E ), E ⊆ V × F Y i Y j ◮ variable nodes V , ◮ factor nodes F , ◮ edges E between variable and factor nodes. ◮ scope of a factor, N ( F ) = { i ∈ V : ( i , F ) ∈ E} Y k Y l ◮ Variable domains Y i ◮ Factorization over potentials ψ at factors , Factor graph p ( y ) = 1 � ψ F ( y N ( F ) ) Z F ∈F ◮ Constant Z = � � F ∈F ψ F ( y N ( F ) ) y ∈Y Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  15. Graphical Models Factor Graphs Test-time Inference Training Software Factor Graphs Factor Graphs ◮ Graph: G = ( V , F , E ), E ⊆ V × F Y i Y j ◮ variable nodes V , ◮ factor nodes F , ◮ edges E between variable and factor nodes. ◮ scope of a factor, N ( F ) = { i ∈ V : ( i , F ) ∈ E} Y k Y l ◮ Variable domains Y i ◮ Factorization over potentials ψ at factors , Factor graph p ( y ) = 1 � ψ F ( y N ( F ) ) Z F ∈F ◮ Constant Z = � � F ∈F ψ F ( y N ( F ) ) y ∈Y Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  16. Graphical Models Factor Graphs Test-time Inference Training Software Factor Graphs Why factor graphs? Y i Y j Y i Y j Y i Y j Y k Y l Y k Y l Y k Y l ◮ Factor graphs are explicit about the factorization ◮ Hence, easier to work with ◮ Universal (just like MRFs and Bayesian networks) Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  17. Graphical Models Factor Graphs Test-time Inference Training Software Factor Graphs Capacity Y i Y j Y i Y j Y k Y l Y k Y l ◮ Factor graph defines family of distributions ◮ Some families are larger than others Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  18. Graphical Models Factor Graphs Test-time Inference Training Software Factor Graphs Four remaining pieces 1. Conditional distributions (CRFs) 2. Parameterization 3. Test-time inference 4. Learning the model from training data Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  19. Graphical Models Factor Graphs Test-time Inference Training Software Factor Graphs Four remaining pieces 1. Conditional distributions (CRFs) 2. Parameterization 3. Test-time inference 4. Learning the model from training data Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

  20. Graphical Models Factor Graphs Test-time Inference Training Software Factor Graphs Conditional Distributions X i X j ◮ We have discussed p ( y ), ◮ How do we define p ( y | x )? ◮ Potentials become a function of x N ( F ) ◮ Partition function depends on x Y i Y j ◮ Conditional random fields (CRFs) ◮ x is not part of the probability model, i.e. not conditional treated as random variable distribution Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction to Graphical Models

Recommend

Part 2: Introduction to Graphical Models Sebastian Nowozin and Christoph H. Lampert Colorado

Part 2: Introduction to Graphical Models Sebastian Nowozin and Christoph H. Lampert Colorado

Graphical Models Factor Graphs Test-time Inference Training Part 2: Introduction to Graphical Models Sebastian Nowozin and Christoph H. Lampert Colorado Springs, 25th June 2011 Sebastian Nowozin and Christoph H. Lampert Part 2: Introduction

820 views • 54 slides
Slides Set 11 (part a): Sampling Techniques for Probabilistic and Deterministic Graphical models

Slides Set 11 (part a): Sampling Techniques for Probabilistic and Deterministic Graphical models

Algorithms for Reasoning with graphical models Slides Set 11 (part a): Sampling Techniques for Probabilistic and Deterministic Graphical models Rina Dechter (Reading Darwiche chapter 15, related papers) slides11a 828X 2019 Sampling

1.05k views • 78 slides
Graphical models for Neuroscience Part I Giuseppe Vinci Department of Statistics Rice

Graphical models for Neuroscience Part I Giuseppe Vinci Department of Statistics Rice

Graphical models for Neuroscience Part I Giuseppe Vinci Department of Statistics Rice University 1/1 Outline 1. Neural connectivity 2. Graphical models 3. Gaussian Graphical models 4. Functional connectivity in Macaque V4 2/1 Neural

454 views • 17 slides
Slides Set 9(part b): Sampling Techniques for Probabilistic and Deterministic Graphical models

Slides Set 9(part b): Sampling Techniques for Probabilistic and Deterministic Graphical models

Reasoning with graphical models Slides Set 9(part b): Sampling Techniques for Probabilistic and Deterministic Graphical models Rina Dechter (Reading Darwiche chapter 15, cutset-sampling paper posted) slides 9b 276 2020 Overview 1.

1.02k views • 63 slides
Lecture 1: Introduction Statistical and Computational Methods for Learning through Graphical

Lecture 1: Introduction Statistical and Computational Methods for Learning through Graphical

Lecture 1: Introduction Statistical and Computational Methods for Learning through Graphical Models (aka Probabilistic Graphical Models) BIOSTAT 830 September 6 th , 2016 Zhenke Wu Some materials adapted from Eric Xings CMU Graphical Model

478 views • 18 slides
Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak

Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak

Probabilistic Graphical Models Probabilistic Graphical Models introduction to learning Siamak Ravanbakhsh Fall 2019 Learning objectives Learning objectives different goals of learning a graphical model effect of goals on the learning setup

273 views • 23 slides
Transforming Graphical System Models to Graphical Attack Models ! Joint work with Marieta

Transforming Graphical System Models to Graphical Attack Models ! Joint work with Marieta

From Graphical System Models... ... to Graphical Attack Models ... ... to Risk Assessment Transforming Graphical System Models to Graphical Attack Models ! Joint work with Marieta Georgieva Ivanova, ! ! Ren e Rydhof Hansen, and Florian

650 views • 27 slides
Contents Introduction 1 Graphical Models and the PC Algorithm Conditional Independence

Contents Introduction 1 Graphical Models and the PC Algorithm Conditional Independence

Contents Introduction 1 Graphical Models and the PC Algorithm Conditional Independence Graphical Models Directed Acyclic Graphs Ewan Donnachie Estimating DAG Structures 2 General Approach 14 July 2006 The PC Algorithm Example Ewan

350 views • 5 slides
A road map to more complex dynamic models discrete discrete continuous Y Y Y discrete

A road map to more complex dynamic models discrete discrete continuous Y Y Y discrete

School of Computer Science State Space Models Probabilistic Graphical Models (10- Probabilistic Graphical Models (10 -708) 708) Lecture 13, part II Nov 5th, 2007 Receptor A Receptor A Receptor B Receptor B X 1 X 1 X 1 X 2 X 2 X 2 Eric

360 views • 21 slides
Graphical Models Graphical Models Bayesian Networks Siamak Ravanbakhsh Fall 2019 Previously on

Graphical Models Graphical Models Bayesian Networks Siamak Ravanbakhsh Fall 2019 Previously on

Graphical Models Graphical Models Bayesian Networks Siamak Ravanbakhsh Fall 2019 Previously on Previously on Probabilistic Graphical Models Probabilistic Graphical Models Probability distribution and density functions Random variable

1.39k views • 104 slides
Undirected Graphical Models Aaron Courville, Universit de Montral 2 (UNDIRECTED) GRAPHICAL

Undirected Graphical Models Aaron Courville, Universit de Montral 2 (UNDIRECTED) GRAPHICAL

Undirected Graphical Models Aaron Courville, Universit de Montral 2 (UNDIRECTED) GRAPHICAL MODELS Overview : Directed versus undirected graphical models Conditional independence Energy function formalism Maximum likelihood

1.06k views • 61 slides
Probabilistic Graphical Models Part II: Undirected Graphical Models Selim Aksoy Department of

Probabilistic Graphical Models Part II: Undirected Graphical Models Selim Aksoy Department of

Probabilistic Graphical Models Part II: Undirected Graphical Models Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Spring 2019 CS 551, Spring 2019 2019, Selim Aksoy (Bilkent University)

599 views • 26 slides
Probabilistic Graphical Models Part II: Undirected Graphical Models Selim Aksoy Department of

Probabilistic Graphical Models Part II: Undirected Graphical Models Selim Aksoy Department of

Probabilistic Graphical Models Part II: Undirected Graphical Models Selim Aksoy Department of Computer Engineering Bilkent University saksoy@cs.bilkent.edu.tr CS 551, Fall 2018 CS 551, Fall 2018 2018, Selim Aksoy (Bilkent University) c

486 views • 27 slides
Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019

Probabilistic Graphical Models Probabilistic Graphical Models Undirected Models Fall 2019 Siamak Ravanbakhsh Learning objectives Learning objectives Markov networks: how it represents a prob. dist. independence assumptions

941 views • 57 slides
Graphical Models L eon Bottou COS 424 4/15/2010 Introduction People like drawings better

Graphical Models L eon Bottou COS 424 4/15/2010 Introduction People like drawings better

Graphical Models L eon Bottou COS 424 4/15/2010 Introduction People like drawings better than equations A graphical model is a diagram representing certain aspects of the algebraic structure of a probabilistic model. Purposes

647 views • 37 slides
Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of

Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of

Directed Graphical Models: Bayesian Networks Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Basics Multivariate distributions with large number of variables Independency assumptions are useful

503 views • 36 slides
An Introduction to An Introduction to Variational Variational Methods for Graphical Models

An Introduction to An Introduction to Variational Variational Methods for Graphical Models

An Introduction to An Introduction to Variational Variational Methods for Graphical Models Methods for Graphical Models By Jordan, M., Ghahramani, Z., Jaakkola, T.S., Saul, L.K. Basics of Basics of Variational Variational Methodology

549 views • 40 slides
Introduction To Graphical Models Peter V. Gehler Max Planck Institute for Intelligent Systems,

Introduction To Graphical Models Peter V. Gehler Max Planck Institute for Intelligent Systems,

Peter Gehler Introduction to Graphical Models Introduction To Graphical Models Peter V. Gehler Max Planck Institute for Intelligent Systems, T ubingen, Germany ENS/INRIA Summer School, Paris, July 2013 1 / 6 Peter Gehler

1.44k views • 109 slides
Graphical Models Independence & Factorization Including structure Complexity of

Graphical Models Independence & Factorization Including structure Complexity of

Graphical Models slides adopted from Sandro Schnborn 1 Graphical Models Independence & Factorization Including structure Complexity of multivariate problems Independence assumptions Graphical Models Graphs to depict

385 views • 23 slides
Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak

Probabilistic Graphical Models Probabilistic Graphical Models Variable elimination Siamak Ravanbakhsh Fall 2019 Learning objective Learning objective an intuition for inference in graphical models why is it difficult? exact inference by

1.49k views • 62 slides
Graphical Models (Lecture 1 - Introduction) Tibrio Caetano tiberiocaetano.com Statistical

Graphical Models (Lecture 1 - Introduction) Tibrio Caetano tiberiocaetano.com Statistical

Graphical Models (Lecture 1 - Introduction) Tibrio Caetano tiberiocaetano.com Statistical Machine Learning Group NICTA Canberra LLSS, Canberra, 2009 nicta-logo Tibrio Caetano: Graphical Models (Lecture 1 - Introduction) 1 / 17 Material

1.78k views • 135 slides
Probabilistic Graphical Models CMSC 691 UMBC Two Problems for Graphical Models 1 ,

Probabilistic Graphical Models CMSC 691 UMBC Two Problems for Graphical Models 1 ,

Probabilistic Graphical Models CMSC 691 UMBC Two Problems for Graphical Models 1 , 2 , 3 , , = 1 Finding the normalizer Computing the marginals Two Problems for Graphical

778 views • 61 slides
Undirected Graphical Models: Markov Random Fields Probabilistic Graphical Models Sharif

Undirected Graphical Models: Markov Random Fields Probabilistic Graphical Models Sharif

Undirected Graphical Models: Markov Random Fields Probabilistic Graphical Models Sharif University of Technology Soleymani Spring 2018 Markov Network Structure: undirected graph Undirected edges show correlations (non-causal

870 views • 53 slides
Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert

Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert

Belief Propagation Variational Inference Sampling Break Part 3: Probabilistic Inference in Graphical Models Sebastian Nowozin and Christoph H. Lampert Colorado Springs, 25th June 2011 Sebastian Nowozin and Christoph H. Lampert Part 3:

951 views • 69 slides

More recommend