Bayes Nets 10-701 recitation 04-02-2013 Bayes Nets Represent - PowerPoint PPT Presentation

Bayes Nets 10-701 recitation 04-02-2013

Bayes Nets • Represent dependencies between variables • Compact representation of probability distribution Allergy Flu • Encodes causal relationships Sinus Nose Headache

Conditional independence • P(X,Y|Z) = P(X|Z) x P(Y|Z) F not ⊥ N Nose Flu N not ⊥ H F ⊥ N | S Sinus Sinus N ⊥ H | S Headache Nose

Conditional independence • Explaining away: Flu – P(F = t | S = t) is high – But P(F = t | S = t , A = t) F ⊥ A is lower Sinus F not ⊥ A | S Allergy

Joint probability distribution • Chain rule of probability: P(X 1 , X 2, …, X n ) = P( X 1 ) P( X 2 |X 1 ) … P(X n |X 1 ,X 2 …X n- 1 )

Joint probability distribution • Chain rule of probability: Allergy Flu P(F,A,S,H,N) = P(F) P(A|F) P(S|A,F) P(H|F,A,S) Sinus P(N|F,A,S,H) Nose Headache Table with 2 5 entries!

Joint probability distribution • Local markov assumption: P(F) P(A) A variable X is independent Allergy Flu of it’s non-descendants given it’s parents P(S|F,A) Sinus P(F,A,S,H,N) = P(F) P(A) P(S|A,F) P(H|S) Nose Headache P(N|S) P(H|S) P(N|S) F = t, A = t F = t, A = f F = f, A = t F = f, A = f S = t 0.9 0.8 0.7 0.1 S = f 0.1 0.2 0.3 0.9

Queries, Inference • P(F = t | N = t) ? P(F) P(A) Allergy Flu • P(F=t|N=t) = P(F=t,N=t)/P(N=t) P(S|F,A) Sinus P(F,N =t) = Σ A,S,H P(F,A,S,H,N=t) = Σ A,S,H P(F) P(A) Nose=t Headache P(S|A,F) P(H|S) P(H|S) P(N|S) P(N=t|S)

Moralizing the graph • Eliminating A will create a factor with F and S Allergy Flu • To assess complexity we can Sinus moralize the graph: connect parents Nose=t Headache

Chose an optimal order If we start with H: P(F,N=t) = Σ A,S P(F) P(A) Allergy Flu P(S|A,F) P(N=t |S) Σ H P(H|S) =1 Sinus = Σ A,S P(F) P(A) P(S|A,F) P(N=t |S) Nose=t

Removing S P(F,N=t)= Σ A,S P(F) P(A) P(S|A,F) Allergy Flu P(N=t |S) = Σ A P(F) P(A) Sinus Σ s P(S|A,F) P(N=t |S) = Σ A P(F) P(A) g 1 (F,A) Nose=t

Removing A P(F,N=t)= P(F) Σ A P(A) g 1 (F,A) Allergy Flu = P(F) g 2 (F) =P(N=t|F) Nose=t P(F=t|N=t) = P(F=t,N=t)/P(N=t) P(N=t) = Σ F P(F,N=t)

Independencies and active trails Is A ⊥ H? When is it not? A is not ⊥ H when given C and F or F’ or F’’ and not {B,D,E,G}

Independencies and active trails • Active trail between variables X 1 ,X 2 …X n-1 when: – X i-1 -> X i -> X i+1 and X i not observed – X i-1 <- X i <- X i+1 and X i not observed – X i-1 <- X i -> X i+1 and X i not observed – X i-1 -> X i <- X i+1 and X i or one of its descendants is observed

Independencies and active trails A ⊥ B ?

Independencies and active trails B ⊥ G |E ?

Independencies and active trails I ⊥ J |K ?

Independencies and active trails E ⊥ F |K ?

Independencies and active trails F ⊥ K |I ?

Independencies and active trails E ⊥ F |I,K ?

Independencies and active trails F ⊥ G |H ?

Independencies and active trails F ⊥ G |H ,A ?