Probabilistic Graphical Models Probabilistic Graphical Models - PowerPoint PPT Presentation

Probabilistic Graphical Models Probabilistic Graphical Models Relationship between the directed & undirected models Siamak Ravanbakhsh Fall 2019

Learning Objective Learning Objective understand the relationship between CIs in directed and undirected models. ⇒ Markov network Bayes-net convert ⇐ Markov network Bayes-net

1. From 1. From Bayesian Bayesian to to Markov Markov networks networks build an I-map for the following G G G 1 3 2

1. From Bayesian 1. From Bayesian to to Markov Markov networks networks build an I-map for the following G G G 1 3 2 moralized I ( M [ G ]) = I ( G ) I ( M [ G ]) ⊆ I ( G ) 1 1 3 3

1. From 1. From Bayesian Bayesian to to Markov Markov networks networks build an I-map for the following G G G 1 3 G 2 4 moralized I ( M [ G ]) = I ( G ) I ( M [ G ]) ⊆ I ( G ) I ( M [ G ]) = I ( G ) 1 1 3 3 3 3

1. From 1. From Bayesian Bayesian to to Markov Markov networks networks build an I-map for the following G G G 1 3 G 2 4 moralized I ( M [ G ]) = I ( G ) I ( M [ G ]) ⊆ I ( G ) I ( M [ G ]) = I ( G ) 1 1 3 3 3 3 Moralize :connect parents keep the skeleton G → M ( G )

From From Bayesian Bayesian to to Markov Markov networks networks moralize & keep the skeleton M [ G ] G for moral , we get a perfect map I ( M [ G ]) = I ( G ) G directed and undirected CI tests are equivalent

From From Bayesian Bayesian to to Markov Markov networks networks alternative approach in both directed and undirected models ⊥ every other var. ∣ MB ( X ) X i i connect each node to its Markov blanket children + parents + parents of children G

From From Bayesian Bayesian to to Markov Markov networks networks alternative approach in both directed and undirected models ⊥ every other var. ∣ MB ( X ) X i i connect each node to its Markov blanket children + parents + parents of children G M [ G ] gives the same moralized graph

2. From 2. From Markov Markov to to Bayesian Bayesian networks networks minimal examples 1. I ( G ) = I ( G ) = I ( H ) 1 2 G H G 2 1

2. From 2. From Markov Markov to to Bayesian Bayesian networks networks minimal examples 1. I ( G ) = I ( G ) = I ( H ) 1 2 G H G 2 1 minimal examples 2. I ( G ) = I ( H ) H G

From From Markov Markov to to Bayesian Bayesian networks networks minimal examples 3. D C B A I ( G ) ⊂ I ( H ) B ⊥ C ∣ A

From From Markov Markov to to Bayesian Bayesian networks networks minimal examples 3. D C B A I ( G ) ⊂ I ( H ) B ⊥ C ∣ A examples 4. I ( G ) ⊂ I ( H ) G H

From Markov From Markov to to Bayesian Bayesian networks networks minimal examples 3. D C B A I ( G ) ⊂ I ( H ) B ⊥ C ∣ A examples 4. how? I ( G ) ⊂ I ( H ) G H

From From Markov Markov to to Bayesian Bayesian networks networks examples 4. H G I ( G ) ⊂ I ( H )

From From Markov Markov to to Bayesian Bayesian networks networks examples 4. build a minimal I-map from CIs in : H pick an ordering - e.g., A,B,C,D,E,F select a minimal parent set s.t. local CI (CI from non-descendents given parents) H G I ( G ) ⊂ I ( H )

From Markov From Markov to to Bayesian Bayesian networks networks examples 4. build a minimal I-map from CIs in : H pick an ordering - e.g., A,B,C,D,E,F select a minimal parent set s.t. local CI (CI from non-descendents given parents) H G I ( G ) ⊂ I ( H ) any non-triangulated loop > 3 has immorality have to triangulate the loops

From Markov From Markov to to Bayesian Bayesian networks networks examples 4. build a minimal I-map from CIs in : H pick an ordering - e.g., A,B,C,D,E,F select a minimal parent set s.t. local CI (CI from non-descendents given parents) H G I ( G ) ⊂ I ( H ) any non-triangulated loop > 3 has immorality chordal G loops of size >3 have chords have to triangulate the loops

 ∩ Chordal = Chordal = Markov Markov Bayesian Bayesian networks networks is not chordal, then for every I ( G ) = I ( H ) H G no perfect MAP in the form of Bayes-net

 ∩ Chordal = Markov Chordal = Markov Bayesian Bayesian networks networks is not chordal, then for every I ( G ) = I ( H ) H G no perfect MAP in the form of Bayes-net is chordal, then for some I ( G ) = I ( H ) H G has a Bayes-net perfect map

 ∩ Chordal = Chordal = Markov Markov Bayesian Bayesian networks networks is not chordal, then for every I ( G ) = I ( H ) H G no perfect MAP in the form of Bayes-net is chordal, then for some I ( G ) = I ( H ) H G has a Bayes-net perfect map need clique-trees to build these

directed directed undirected undirected parameter-estimation is easy simpler CI semantics can represent causal relations less interpretable form for local factors better for encoding expert less restrictive in structural form (loops) domain knowledge

Summary Summary directed to undirected: moralize undirected to directed: triangulate ∩ Chordal graphs = Markov Bayesian networks p-maps in both directions