ECE 4524 Artificial Intelligence and Engineering Applications - PowerPoint PPT Presentation

ECE 4524 Artificial Intelligence and Engineering Applications Lecture 19: Bayesian Networks Reading: AIAMA 14.1-14.4 Today’s Schedule: ◮ Bayesian Networks ◮ Causal Reasoning ◮ Exact Inference

Bayesian Networks (BN) ◮ A BN is a directed acyclic graph whose nodes are random variables. ◮ The nodes are annotated with the conditional probability of that node given its parents in the network. ◮ The graph structure encodes the assumptions of the joint distribution/density and ◮ encodes the causal structure of the KB.

How do we guarantee a graph is a valid encoding of the joint probability? Recall the product rule n − 1 � P ( X 1 , X 2 , · · · , X n ) = P ( X n ) P ( X i | X i − 1 , · · · , X n ) i =1 ◮ This defines the graph structure ◮ Some ordering of the variables is better than others in the sense it leads to graphs of less complexity.

BN Topology The topology of the graph is related to the conditional independence of variables. Define the Markov blanket for node X i as the set M ( X i ) ≡ { Parents( X i ) , Children( X i ) , Parents(Children( X i )) } Then X i is conditionally independent of any variable not in M ( X i ) given the variables in M ( X i ).

The nodes are annotated with the Conditional Probability Table (CPT) There are 3 cases ◮ Discrete R.V.s ◮ Continuous R.V.s ◮ Hybrid Networks (mixtures of Discrete and Continuous R.V.s) The simplest case is for Bernoulli R.V.s with a ”noisy or”.

Doing Inference There are two approaches ◮ exact (today) ◮ approximate (next time) First some terminology. We partition the variables into 3 sets ◮ Evidence Variables, E ◮ Hidden Variables, Y ◮ Query Variables, X Inference is the procedure to answer the query P ( X | E )

For Discrete BN P ( X | E ) = P ( X , E ) 1 � = P ( X , E , Y ) P ( E ) P ( E ) y We marginalize over the hidden variables.

Complexity of Exact Inference ◮ for singly-connected graphs (polytrees) it is linear in the number of CPT entries. ◮ for general BNs it is in #P-Hard, strictly harder than NP-complete problems This is why approximation algorithms are important.

Next Actions ◮ Reading on Approximate Inference (AIAMA 14.5)