1 Bayes Nets: Assumptions Independence in a BN Assumptions we are - PDF document

1 2 Recap: Bayes’ Nets CS 473: Artificial Intelligence Bayes’ Nets: Independence  A Bayes’ net is an efficient encoding of a probabilistic model of a domain  Questions we can ask:  Inference: given a fixed BN, what is P(X | e)?  Representation: given a BN graph, what kinds of distributions can it encode?  Modeling: what BN is most appropriate for a given domain? Steve Tanimoto [These slides were created by Dan Klein and Pieter Abbeel for CS188 Intro to AI at UC Berkeley. All CS188 materials are available at http://ai.berkeley.edu.] Bayes’ Nets Conditional Independence  X and Y are independent if  Representation  Conditional Independences  X and Y are conditionally independent given Z  Probabilistic Inference  Learning Bayes’ Nets from Data  (Conditional) independence is a property of a distribution  Example: 1

Bayes Nets: Assumptions Independence in a BN  Assumptions we are required to make to define the  Important question about a BN: Bayes net when given the graph:  Are two nodes independent given certain evidence?  If yes, can prove using algebra (tedious in general)  If no, can prove with a counter example  Beyond above “chain rule  Bayes net” conditional  Example: independence assumptions  Often additional conditional independences X Y Z  They can be read off the graph  Question: are X and Z necessarily independent?  Important for modeling: understand assumptions made  Answer: no. Example: low pressure causes rain, which causes traffic.  X can influence Z, Z can influence X (via Y) when choosing a Bayes net graph  Addendum: they could be independent: how? D-separation: Outline D-separation: Outline  Study independence properties for triples  Analyze complex cases in terms of member triples  D-separation: a condition / algorithm for answering such queries Causal Chains Causal Chains  This configuration is a “ causal chain ”  This configuration is a “ causal chain ”  Guaranteed X independent of Z ? No!  Guaranteed X independent of Z given Y?  One example set of CPTs for which X is not independent of Z is sufficient to show this independence is not guaranteed.  Example:  Low pressure causes rain causes traffic, high pressure causes no rain causes no traffic X: Low pressure Y: Rain Z: Traffic X: Low pressure Y: Rain Z: Traffic  In numbers: Yes! P( +y | +x ) = 1, P( -y | - x ) = 1,  Evidence along the chain “ blocks ” the P( +z | +y ) = 1, P( -z | -y ) = 1 influence 2

Common Cause Common Cause  This configuration is a “ common cause ”  Guaranteed X independent of Z ? No!  This configuration is a “ common cause ”  Guaranteed X and Z independent given Y?  One example set of CPTs for which X is not Y: Project Y: Project independent of Z is sufficient to show this due due independence is not guaranteed.  Example:  Project due causes both forums busy and lab full  In numbers: X: Forums X: Forums Z: Lab full Z: Lab full P( +x | +y ) = 1, P( -x | -y ) = 1, busy busy Yes! P( +z | +y ) = 1, P( -z | -y ) = 1  Observing the cause blocks influence between effects. Common Effect The General Case  Last configuration: two causes of one  Are X and Y independent? effect (v-structures)  Yes : the ballgame and the rain cause traffic, but they are not correlated X: Raining Y: Ballgame  Still need to prove they must be (try it!)  Are X and Y independent given Z?  No : seeing traffic puts the rain and the ballgame in competition as explanation.  This is backwards from the other cases  Observing an effect activates influence between Z: Traffic possible causes . The General Case Reachability L  Recipe: shade evidence nodes, look  General question: in a given BN, are two variables independent for paths in the resulting graph (given evidence)? R B  Attempt 1: if two nodes are connected by an undirected path not blocked by  Solution: analyze the graph a shaded node, then they are not conditionally independent D T  Any complex example can be broken  Almost works, but not quite into repetitions of the three canonical cases  Where does it break?  Answer: the v-structure at T doesn’t count as a link in a path unless “active” 3

Active / Inactive Paths D-Separation ?  Question: Are X and Y conditionally independent given Active Triples Inactive Triples  Query: evidence variables {Z}?  Yes, if X and Y “ d-separated ” by Z  Check all (undirected!) paths between and  Consider all (undirected) paths from X to Y  No active paths = independence!  If one or more active, then independence not guaranteed  A path is active if each triple is active:  Causal chain A  B  C where B is unobserved (either direction)  Common cause A  B  C where B is unobserved  Otherwise (i.e. if all paths are inactive),  Common effect (aka v-structure) A  B  C where B or one of its descendents is observed then independence is guaranteed  All it takes to block a path is a single inactive segment Example Example L R B Yes Yes R B Yes T D T T ’ Yes T ’ Example Structure Implications  Variables:  Given a Bayes net structure, can run d- separation algorithm to build a complete list of  R: Raining conditional independences that are necessarily R  T: Traffic true of the form  D: Roof drips T D  S: I’m sad  Questions: S  This list determines the set of probability distributions that can be represented Yes 4

Computing All Independences Topology Limits Distributions Y  Given some graph topology G, only certain joint Y Y X Z distributions can be encoded X Z X Z Y Y  The graph structure X Z guarantees certain X Z (conditional) independences Y  (There might be more X Z X Z independence)  Adding arcs increases the Y set of distributions, but has Y Y Y several costs Y X Z X Z X Z  Full conditioning can encode Y Y Y any distribution X Z X Z X Z X Z Bayes’ Nets Bayes Nets Representation Summary  Bayes nets compactly encode joint distributions  Representation  Conditional Independences  Guaranteed independencies of distributions can be deduced from BN graph structure  Probabilistic Inference  Enumeration (exact, exponential complexity)  D-separation gives precise conditional independence  Variable elimination (exact, worst-case guarantees from graph alone exponential complexity, often better)  Probabilistic inference is NP-complete  A Bayes ’ net ’ s joint distribution may have further  Sampling (approximate) (conditional) independence that is not detectable until you inspect its specific distribution  Learning Bayes’ Nets from Data 5