Belief net w orks Chapter 15.1{2 c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 1
Outline } Conditional indep end enc e } Ba y esian net w o rks: syntax and semantics } Exact inference } App ro ximate inferenc e c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 2
Indep endence Tw o random va riables A B a re (absolutely) indep end en t i� ( A j B ) = ( A ) P P ( A; ) = ( A j B ) P ( B ) = ( A ) P ( B ) o r P B P P e.g., A and B a re t w o coin tosses If n Bo olean va riables a re indep end en t, the full joint is � P ( X ) = P ( X ) ; : : : ; X 1 n i i hence can b e sp eci�ed b y just n numb ers Absolute indep end en ce is a very strong requiremen t, seldom met c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 3
Conditional indep endence Consider the dentist p roblem with three random va riables: T oothache , C av ity , C atch (steel p rob e catches in my to oth) 3 2 1 The full joint distribu tion has � = 7 indep en de nt entries If I have a cavit y , the p robabili t y that the p rob e catches in it do esn't dep end on whether I have a to othache: ( C ) = ( C ) (1) P atch j T oothache; C av ity P atch j C av ity i.e., C atch is conditiona ll y indep en den t of T oothache given C av ity The same indep end enc e holds if I haven't got a cavit y: ( C ) = ( C ) (2) P atch j T oothache; : C av ity P atch j: C av ity c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 4
Conditional indep endence con td. Equivalent statements to (1) ( T ) = ( T ) (1a) P oothache j C atch; C av ity P oothache j C av ity Why ?? ( T ) = ( T ) P ( C ) (1b) P oothache; C atch j C av ity P oothache j C av ity atch j C av ity Why?? F ull joint distributi on can no w b e written as P ( T ) = P ( T ) P ( C ) oothache; C atch; C av ity oothache; C atch j C av ity av ity ( T ) P ( C ) P ( C ) = oothache j C av ity atch j C av ity av ity P i.e., 2 + 2 + 1 = 5 indep en de nt numb ers (equations 1 and 2 remove 2) c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 5
Conditional indep endence con td. Equivalent statements to (1) ( T ) = ( T ) (1a) P oothache j C atch; C av ity P oothache j C av ity Why ?? ( T ) P oothache j C atch; C av ity = ( C ) P ( T ) =P ( C ) P atch j T oothache; C av ity oothache j C av ity atch j C av ity = ( C ) P ( T ) =P ( C ) (from 1) P atch j C av ity oothache j C av ity atch j C av ity = ( T ) P oothache j C av ity ( T ) = ( T ) P ( C ) (1b) P oothache; C atch j C av ity P oothache j C av ity atch j C av ity Why?? ( T ) P oothache; C atch j C av ity = ( T ) P ( C ) (pro duct rule) P oothache j C atch; C av ity atch j C av ity = ( T ) P ( C ) (from 1a) P oothache j C av ity atch j C av ity c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 6
Belief net w orks A simple, graphical notation fo r conditiona l indep end en ce assertions and hence fo r compact sp eci�cati on of full joint distribut ion s Syntax: a set of no des, one p er va riable � a directed, acyclic graph (link \directly in�uences") a conditional distributio n fo r each no de given its pa rents: P ( X ents ( X )) j P ar i i In the simplest case, conditiona l distributi on rep resented as a conditional p robabilit y table (CPT) c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 7
Example I'm at w o rk, neighb o r John calls to sa y my ala rm is ringing, but neighb o r Ma ry do esn't call. Sometimes it's set o� b y mino r ea rthquak es. Is there a burgla r? V a riables: B ur g l ar , E ar thq uak e , Al ar m , J ohnC al l s , M ar y C al l s Net w o rk top ology re�ects \causal" kno wledge: P(E) Burglary P(B) Earthquake .002 .001 B E P(A) T T .95 Alarm T F .94 F T .29 F F .001 A P(J) A P(M) JohnCalls T .90 MaryCalls T .70 F .05 F .01 ( d n ) ( d ) Note: � k pa rents ) O numb ers vs. O k n c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 8
Seman tics \Global" semantics de�nes the full joint distribution as the p ro duct of the lo cal conditional distribution s: � n ( X ) = ( X j P ents ( X )) P ; : : : ; X P ar 1 = 1 n i i i ( J ) e.g., P ^ M ^ A ^ : B ^ : E is given b y?? = c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 9
Seman tics \Global" semantics de�nes the full joint distribution as the p ro duct of the lo cal conditional distribution s: � n ( X ) = ( X j P ents ( X )) P ; : : : ; X P ar 1 = 1 n i i i ( J ) e.g., P ^ M ^ A ^ : B ^ : E is given b y?? ( : B ) P ( : E ) P ( A j: B ^ : E ) P ( J j A ) P ( M j A ) = P \Lo cal" semantics: each no de is conditiona ll y indep en den t of its nondescend an ts given its pa rents Theo rem: Lo cal semantics , global semantics c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 10
Mark o v blank et Each no de is conditional ly indep end ent of all others given its Ma rk ov blank et : pa rents + children + children's pa rents U 1 U m . . . X Z 1j Z nj Y Y n 1 . . . c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 11
Constructing b elief net w orks Need a metho d such that a series of lo cally testable assertions of conditional indep end enc e gua rantees the required global semantics 1. Cho ose an o rdering of va riables X ; : : : ; X 1 n 2. F o r i = 1 to n add X to the net w o rk i select pa rents from X ; : : : ; X such that 1 i � 1 P ( X j P ents ( X )) = P ( X j X ) ar ; : : : ; X 1 i � 1 i i i This choice of pa rents gua rantees the global semantics: � n P ( X ) = ( X ) ; : : : ; X P j X ; : : : ; X (chain rule) 1 1 i � 1 = 1 n i i � n P ( X j P ents ( X )) = ar b y construction = 1 i i i c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 12
Example Supp ose w e cho ose the o rdering M , J , A , B , E MaryCalls JohnCalls ( J ) = ( J ) ? P j M P c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 13
. Alarm . No ( A j J ) = ( A j J ) ? ( A j J ) = ( A ) ? P ; M P P ; M P c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 14
. Burglary . . No ( B ) = ( B j A ) ? P j A; J ; M P ( B ) = ( B ) ? P j A; J ; M P c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 15
. Earthquake . . . Y es . No ( E j B ) = ( E j A ) ? P ; A; J ; M P ( E j B ) = ( E j A; ) ? P ; A; J ; M P B c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 16
. . . . . . No . Y es c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 17
Example: Car diagnosis Initial evidence: engine w on't sta rt T estable va riables (thin ovals), diagnosis va riables (thick ovals) Hidden va riables (shaded) ensure spa rse structure, reduce pa rameters fanbelt alternator battery age broken broken battery no charging dead fuel line starter battery no oil no gas blocked broken flat engine won’t oil light gas gauge lights start c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 18
Example: Car insurance Predict claim costs (medical, liabilit y , p rop ert y) given data on applicati on fo rm (other unshaded no des) SocioEcon Age GoodStudent ExtraCar Mileage RiskAversion VehicleYear SeniorTrain MakeModel DrivingSkill DrivingHist Antilock DrivQuality AntiTheft HomeBase CarValue Airbag Accident Ruggedness Theft OwnDamage Cushioning OwnCost OtherCost MedicalCost LiabilityCost PropertyCost c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 19
Compact conditional distributions CPT gro ws exp onential ly with no. of pa rents CPT b ecomes in�nite with continuous-valu ed pa rent o r child Solution: canonical distribut ion s that a re de�ned compactly Deterministi c no des a re the simplest case: = ( P ents ( X )) X f ar fo r some function f E.g., Bo olean functions N or thAmer ican , C anadian _ U S _ M exican E.g., numerical relationsh ip s among continuous va riables @ Lev el = in�o w + p recipation - out�o w - evap o ration @ t c AIMA Slides � Stuart Russell and P eter Norvig, 1998 Chapter 15.1{2 20
Recommend
More recommend
