Introduction Bayes Nets Independence MRF Example P Graphical Models Henrik I. Christensen Robotics & Intelligent Machines @ GT Georgia Institute of Technology, Atlanta, GA 30332-0280 hic@cc.gatech.edu Henrik I. Christensen (RIM@GT) Graphical Models 1 / 55
Introduction Bayes Nets Independence MRF Example P Outline Introduction 1 Bayesian Networks 2 Conditional Independence 3 Markov Random Fields 4 Small Example 5 Summary 6 Henrik I. Christensen (RIM@GT) Graphical Models 2 / 55
Introduction Bayes Nets Independence MRF Example P Introduction Basically we can describe Bayesian inference through repeated use of the sum and product rules Using a graphical / diagrammatical representation is often useful A way to visualize structure and consider relations 1 Provides insights into a model and possible independence 2 Allow us to leverage of the many graphical algorithms available 3 Will consider both directed and undirected models This is a very rich areas with numerous good references Henrik I. Christensen (RIM@GT) Graphical Models 3 / 55
Introduction Bayes Nets Independence MRF Example P Outline Introduction 1 Bayesian Networks 2 Conditional Independence 3 Markov Random Fields 4 Small Example 5 Summary 6 Henrik I. Christensen (RIM@GT) Graphical Models 4 / 55
Introduction Bayes Nets Independence MRF Example P Bayesian Networks Consider the joint probability p ( a , b , c ) Using product rule we can rewrite it as p ( a , b , c ) = p ( c | a , b ) p ( a , b ) Which again can be changed to p ( a , b , c ) = p ( c | a , b ) p ( b | a ) p ( a ) We can illustrate this as a b c Henrik I. Christensen (RIM@GT) Graphical Models 5 / 55
Introduction Bayes Nets Independence MRF Example P Bayesian Networks Nodes represent variables Arcs/links represent conditional dependence We can use the decomposition for any joint distribution. The direct / brute-force application generates fully connected graphs We can represent much more general relations Henrik I. Christensen (RIM@GT) Graphical Models 6 / 55
Introduction Bayes Nets Independence MRF Example P Example with ”sparse” connections We can represent relations such as p ( x 1 ) p ( x 2 ) p ( x 3 ) p ( x 4 | x 1 , x 2 , x 3 ) p ( x 5 | x 1 , x 3 ) p ( x 6 | x 4 ) p ( x 7 | x 4 , x 5 ) Which is shown below x 1 x 2 x 3 x 4 x 5 x 6 x 7 Henrik I. Christensen (RIM@GT) Graphical Models 7 / 55
Introduction Bayes Nets Independence MRF Example P The general case We can think of this as coding the factors p ( x k | pa k ) where pa k is the set of parents to a variable x k The inference is then � p ( x ) = p ( x k | pa k ) k we will refer to this as factorization There can be no directed cycles in the graph The general form is termed a directed acyclic graph - DAG Henrik I. Christensen (RIM@GT) Graphical Models 8 / 55
Introduction Bayes Nets Independence MRF Example P Basic example We have seen the polynomial regression before � p ( t n | w ) p ( t , w ) = p ( w ) n Which can be visualized as w t 1 t N Henrik I. Christensen (RIM@GT) Graphical Models 9 / 55
Introduction Bayes Nets Independence MRF Example P Bayesian Regression We can make the parameters and variables explicit p ( t , w | x , α, σ 2 ) = p ( w | α ) � p ( t n | w , x n , σ 2 ) n as shown here x n α w σ 2 t n N Henrik I. Christensen (RIM@GT) Graphical Models 10 / 55
Introduction Bayes Nets Independence MRF Example P Bayesian Regression - Learning When entering data we can condition inference on it � p ( w | t ) ∝ p ( w ) p ( t n | w ) n x n α w σ 2 t n N Henrik I. Christensen (RIM@GT) Graphical Models 11 / 55
Introduction Bayes Nets Independence MRF Example P Generative Models - Example Image Synthesis Object Position Orientation Image Henrik I. Christensen (RIM@GT) Graphical Models 12 / 55
Introduction Bayes Nets Independence MRF Example P Discrete Variables - 1 General joint distribution has K 2 − 1 parameters (for K possible outcomes) x 1 x 2 K K µ x 1 i x 2 j � � p ( x 1 , x 2 | µ ) = ij i =1 j =1 Independent joint distributions have 2( K − 1) parameters x 1 x 2 K K µ x 2 j � � µ x 1 i p ( x 1 , x 2 | µ ) = 1 i 2 j i =1 j =1 Henrik I. Christensen (RIM@GT) Graphical Models 13 / 55
Introduction Bayes Nets Independence MRF Example P Discrete Variables - 2 General joint distribution over M variables will have K M − 1 parameters A Markov chain with M nodes will have K − 1 + ( M − 1) K ( K − 1) parameters x 1 x 2 x M Henrik I. Christensen (RIM@GT) Graphical Models 14 / 55
Introduction Bayes Nets Independence MRF Example P Discrete Variables - Bayesian Parms µ 1 µ 2 µ M x 1 x 2 x M The parameters can be modelled explicitly M � p ( { x m , µ m } ) = p ( x 1 | µ 1 ) p ( µ 1 ) p ( x m | x m − 1 , µ m ) p ( µ m ) m =2 It is assumed that p ( µ m ) is a Dirachlet Henrik I. Christensen (RIM@GT) Graphical Models 15 / 55
Introduction Bayes Nets Independence MRF Example P Discrete Variables - Bayesian Parms (2) µ 1 µ x 1 x 2 x M For shared paraemeters the situation is simpler M � p ( { x m } , µ 1 , µ ) = p ( x 1 | µ 1 ) p ( µ 1 ) p ( x m | x m − 1 , µ ) p ( µ ) m =2 Henrik I. Christensen (RIM@GT) Graphical Models 16 / 55
Introduction Bayes Nets Independence MRF Example P Extension to Linear Gaussian Models The model can be extended to have each node as a Gaussian process/variable that is a linear function of its parents � � p ( x i | pa i ) = N x i w ij x j + b i , v i � � j ∈ pa i Henrik I. Christensen (RIM@GT) Graphical Models 17 / 55
Introduction Bayes Nets Independence MRF Example P Outline Introduction 1 Bayesian Networks 2 Conditional Independence 3 Markov Random Fields 4 Small Example 5 Summary 6 Henrik I. Christensen (RIM@GT) Graphical Models 18 / 55
Introduction Bayes Nets Independence MRF Example P Conditional Independence Considerations of independence is important as part of the analysis and setup of a system As an example a is independent of b given c p ( a | b , c ) = p ( a | c ) Or equivalently p ( a , b | c ) = p ( a | b , c ) p ( b | c ) p ( a | c ) p ( b | c ) = Frequent notation in statistics a ⊥ ⊥ b | c Henrik I. Christensen (RIM@GT) Graphical Models 19 / 55
Introduction Bayes Nets Independence MRF Example P Conditional Independence - Case 1 c p ( a | c ) p ( b | c ) p ( c ) p ( a , b , c ) = � p ( a , b ) = p ( a | c ) p ( b | c ) p ( c ) a b c a / ⊥ ⊥ b | ∅ Henrik I. Christensen (RIM@GT) Graphical Models 20 / 55
Introduction Bayes Nets Independence MRF Example P Conditional Independence - Case 1 c p ( a , b , c ) p ( a , b | c ) = p ( c ) p ( a | c ) p ( b | c ) = a b a ⊥ ⊥ b | c Henrik I. Christensen (RIM@GT) Graphical Models 21 / 55
Introduction Bayes Nets Independence MRF Example P Conditional Independence - Case 2 a c b p ( a , b , c ) = p ( a ) p ( c | a ) p ( b | c ) � p ( a , b ) = p ( a ) p ( c | a ) p ( b | c ) = p ( a ) p ( b | a ) c a / ⊥ ⊥ b |∅ Henrik I. Christensen (RIM@GT) Graphical Models 22 / 55
Introduction Bayes Nets Independence MRF Example P Conditional Independence - Case 2 a c b p ( a , b , c ) p ( a , b | c ) = p ( c ) p ( a ) p ( c | a ) p ( b | c ) = p ( c ) = p ( a | c ) p ( b | c ) a ⊥ ⊥ b | c Henrik I. Christensen (RIM@GT) Graphical Models 23 / 55
Introduction Bayes Nets Independence MRF Example P Conditional Independence - Case 3 a b p ( a ) p ( b ) p ( c | a , b ) p ( a , b , c ) = p ( a , b ) = p ( a ) p ( b ) a ⊥ ⊥ b | ∅ c This is the opposite of Case 1 - when c unobserved Henrik I. Christensen (RIM@GT) Graphical Models 24 / 55
Introduction Bayes Nets Independence MRF Example P Conditional Independence - Case 3 a b p ( a , b , c ) p ( a , b | c ) = p ( c ) p ( a ) p ( b ) p ( c | a , b ) = p ( c ) a / ⊥ ⊥ b | c c This is the opposite of Case 1 - when c observed Henrik I. Christensen (RIM@GT) Graphical Models 25 / 55
Introduction Bayes Nets Independence MRF Example P Diagnostics - Out of fuel? p ( G = 1 | B = 1 , F = 1) = 0 . 8 p ( G = 1 | B = 1 , F = 0) = 0 . 2 B F p ( G = 1 | B = 0 , F = 1) = 0 . 2 p ( G = 1 | B = 0 , F = 0) = 0 . 1 p ( B = 1) = 0 . 9 G B = Battery p ( F = 1) = 0 . 9 F = Fuel Tank ⇒ p ( F = 0) = 0 . 1 G = Fuel Gauge Henrik I. Christensen (RIM@GT) Graphical Models 26 / 55
Introduction Bayes Nets Independence MRF Example P Diagnostics - Out of fuel? B F G p ( G = 0 | F = 0) p ( F = 0) p ( F = 0 | G = 0) = p ( G = 0) ≈ 0 . 257 Observing G=0 increased the probability of an empty tank Henrik I. Christensen (RIM@GT) Graphical Models 27 / 55
Introduction Bayes Nets Independence MRF Example P Diagnostics - Out of fuel? B F G p ( G = 0 | B = 0 , F = 0) p ( F = 0) p ( F = 0 | G = 0 , B = 0) = � p ( G = 0 | B = 0 , F ) p ( F ) ≈ 0 . 111 Observing B=0 implies less likely empty tank Henrik I. Christensen (RIM@GT) Graphical Models 28 / 55
Recommend
More recommend
