BAYES FORMULA a two-stage experiment Xingru Chen - PowerPoint PPT Presentation

BAYES’ FORMULA a two-stage experiment Xingru Chen xingru.chen.gr@dartmouth.edu XC 2020

Simplest Bayes’ Formula 𝑄 𝐶|𝐵 = ! " !($|") . B B !($) Bayes probabilities Bayes’ theorem links the B B degree of belief in a proposition be before and after accounting for ev eviden ence ce . XC 2020

Simplest Bayes’ Formula 𝑄 𝐶 𝐵 = !("∩$) !($) . = 𝑄 𝐶|𝐵 = ! " !($|") . B B !($) 𝑄 𝐵 𝐶 = !($∩") !(") . = XC 2020

Even Event-1: 1: rain We Weather Forecast 𝑄 rain = 0.6 Prior Pr probab ability lity of The pr prior pr probabili an event (often simply called th rior ) is its probability obtained from the pri some prior information. Event-2: Even windy & cl clou oudy 𝑄 windy & cloudy = 0.48 Evidence ce ce term in The ev eviden ence Bayes’ theorem refers to lity of the ov overall pr probabili this new piece of information. XC 2020

Even Event-1: 1: rain Prior probability 𝑄 rain = 0.6 Even Event-2: windy & cl clou oudy Evidence 𝑄 windy & cloudy = 0.48 𝑄 rain|windy & cloudy = ! ()*+ !(,*+-. & 0123-.|()*+) . B B !(,*+-. & 0123-.) XC 2020

Even Event-1: 1: rain Prior probability 𝑄 rain = 0.6 Even Event-2: windy & cl clou oudy Evidence 𝑄 windy & cloudy = 0.48 windy & cl clou oudy | | rain 𝑄(windy & cloudy ｜ rain) = 0.64 𝑄 rain|windy & cloudy = B B ! "#$% !('$%() & +,-.()|"#$%) Like kelihood ood . !('$%() & +,-.()) The like kelihood represents a conditional probability. It is the degree to which the first event is consistent with the second event. XC 2020

Even Event-1: 1: rain rain | | windy & cl clou oudy Prior probability Posterior probability 𝑄 rain|windy & cloudy = ⋯ 𝑄 rain = 0.6 Even Event-2: windy & cl clou oudy windy & cl clou oudy | | rain Likelihood Evidence 𝑄 windy & cloudy = 0.48 𝑄(windy & cloudy ｜ rain) = 0.64 rain | | windy & cl clou oudy Posterior probability 𝑄 rain|windy & cloudy = 𝑄 rain|windy & cloudy = 0.6×0.64 B B ! "#$% !('$%() & +,-.()|"#$%) . = 0.8 !('$%() & +,-.()) 0.48 XC 2020

Prior Pr probab ability Posterior Po probability ra rain rain | | windy & cl clou oudy lity of lity represents The pr prior pr probabili an event (often simply The po post ster erior pr probabili lity after called th the pri rior ) is its probability obtained from the up updated pr prior pr probabili taking into some prior information. account some new piece of information. Evidence ce Like kelihood ood windy & cl clou oudy windy & cl clou oudy | | rain The ev eviden ence ce term in Bayes’ theorem refers to The like kelihood represents a conditional the ov overall pr probabili lity of this new piece of probability. It is the degree to which the first information. event is consistent with the second event. Posterior probability = 4(*2( 5(26)6*1*7.×9*:;1*<22- . B B =>*-;+0; XC 2020

Bayes probabilities Bayes probabilities are particularly appropriate for medical diagnosis. XC 2020

Solving Inverse Problems Using Bayes Probabilities A doctor is trying to decide if a patient has one of three diseases 𝑒 1 , 𝑒 2 , or 𝑒 3 . Two § tests are to be carried out, each of which results in a positive (+) or a negative (−) outcome. There are four possible test patterns ++, +−, −+, and −−. National records have indicated that, for 10,000 people having one of these three § diseases, the distribution of diseases and test results are as in Table below. Nu Number having th this disease Number Nu having Disease Di this th disease + + + + + − − − + − − − 3215 2110 301 704 100 𝑒 ! 2125 396 132 1187 410 𝑒 " 4660 510 3568 73 509 𝑒 # Total 10000 3016 4001 1964 1019 XC 2020

Use Bayes’ formula to compute various posterior probabilities 𝒆 𝟐 𝒆 𝟑 𝒆 𝟒 + + + + + − − − + − − − XC 2020

Use Bayes’ formula to compute various posterior probabilities 𝒆 𝟐 𝒆 𝟑 𝒆 𝟒 + + + + + − − − + − − − Po Posterior pr probabili lity: 𝒆 𝒋 | | + + Posterior pr Po probabili lity: 𝒆 𝒋 | | + − |+ + = 𝑸 + +| 𝒆 𝒋 𝑸(𝒆 𝒋 ) |+ − = 𝑸 + −| 𝒆 𝒋 𝑸(𝒆 𝒋 ) 𝐐 𝒆 𝒋 |+ 𝐐 𝒆 𝒋 |+ 𝑸( + + ) 𝑸( + − ) Po Posterior pr probabili lity: 𝒆 𝒋 | − − + Po Posterior pr probabili lity: 𝒆 𝒋 | − − − 𝐐 𝒆 𝒋 |− + = 𝑸 − +| 𝒆 𝒋 𝑸(𝒆 𝒋 ) 𝐐 𝒆 𝒋 |− − = 𝑸 − −| 𝒆 𝒋 𝑸(𝒆 𝒋 ) 𝑸( − + ) 𝑸( − − ) XC 2020

Prior Probability Nu Number having th this disease Number Nu having Disease Di this th disease + + + + + − − + − − − − 3215 2110 301 704 100 𝑒 ! 2125 396 132 1187 410 𝑒 " 4660 510 3568 73 509 𝑒 # Total 10000 3016 4001 1964 1019 Prio Pr ior probabil ilit ity: dis isease 1 Pr Prio ior probabil ilit ity: dis isease 2 Pr Prio ior probabil ilit ity: dis isease 3 𝑄 𝑒 # = 4660 𝑄 𝑒 ! = 3215 𝑄 𝑒 " = 2125 10000 = 0.466 10000 = 0.3215 10000 = 0.2125 XC 2020

Evidence Number Nu having th this disease Nu Number having Di Disease this th disease + + + + + − − − + − − − 3215 2110 301 704 100 𝑒 ! 2125 396 132 1187 410 𝑒 " 4660 510 3568 73 509 𝑒 # Total 10000 3016 4001 1964 1019 Ev Evide dence: + + Evide Ev dence: + − 𝑄 + + = 3016 𝑄 + − = 4001 10000 = 0.3016 10000 = 0.4001 Ev Evide dence: − + Ev Evide dence − − 𝑄 − + = 1964 𝑄 − − = 1019 10000 = 0.1964 10000 = 0.1019 XC 2020

Likelihood Number Nu having th this disease Nu Number having Disease Di this th disease + + + + + − − − + − − − 3215 2110 301 704 100 𝑒 ! 2125 396 132 1187 410 𝑒 " 4660 510 3568 73 509 𝑒 # Total 10000 3016 4001 1964 1019 Like kelihood: d: + + | 𝒆 𝟐 Like kelihood: d: + − | 𝒆 𝟐 𝑄 + +| 𝑒 ! = 2110 𝑄 + −| 𝑒 ! = 301 3215 3215 Like kelihood: d: − + | 𝒆 𝟐 Like kelihood: d: − − | 𝒆 𝟐 𝑄 − + = 704 𝑄 − − = 100 3215 3215 XC 2020

Posterior Probability Nu Number having th this disease Nu Number having Disease Di this th disease + + + + + − − − + − − − 3215 2110 301 704 100 𝑒 ! 2125 396 132 1187 410 𝑒 " 4660 510 3568 73 509 𝑒 # Total 10000 3016 4001 1964 1019 Pr Prio ior probabil ilit ity: dis isease 1 Posterior pr Po probabili lity: 𝒆 𝟐 | | + + 𝑄 𝑒 ! = 3215 Like kelihood: d: + + | 𝒆 𝟐 |+ + = 𝑸 + +| 𝒆 𝟐 𝑸(𝒆 𝟐 ) 10000 = 0.3215 𝐐 𝒆 𝟐 |+ 𝑸( + + ) 𝑄 + +| 𝑒 ! = 2110 2110 3215 Evide Ev dence: + + = 2110 3215 3215 10000 = 3016 = 𝟏. 𝟖𝟏𝟏 3016 𝑄 + + = 3016 10000 10000 = 0.3016 XC 2020

Posterior Probability 𝒆 𝟐 𝒆 𝟑 𝒆 𝟒 + + + 0.700 0.131 0.169 + + − 0.075 0.033 0.892 − − + 0.358 0.604 0.038 − − − 0.098 0.403 0.499 Po Posterior pr probabili lity: 𝒆 𝒋 | | + + Po Posterior pr probabili lity: 𝒆 𝒋 | | + − |+ + = 𝑸 + +| 𝒆 𝒋 𝑸(𝒆 𝒋 ) |+ − = 𝑸 + −| 𝒆 𝒋 𝑸(𝒆 𝒋 ) 𝐐 𝒆 𝒋 |+ 𝐐 𝒆 𝒋 |+ 𝑸( + + ) 𝑸( + − ) Posterior pr Po probabili lity: 𝒆 𝒋 | − − + Po Posterior pr probabili lity: 𝒆 𝒋 | − − − 𝐐 𝒆 𝒋 |− + = 𝑸 − +| 𝒆 𝒋 𝑸(𝒆 𝒋 ) 𝐐 𝒆 𝒋 |− − = 𝑸 − −| 𝒆 𝒋 𝑸(𝒆 𝒋 ) 𝑸( − + ) 𝑸( − − ) XC 2020

HOW TO GET THE EVIDENCE? Sometimes the evidence is not directly given to us… XC 2020

Bayes’ Formula § Ba Bayes es p probabil ilit ities ies : given the outcome of the second stage of a two-stage experiment, the probability for an outcome at the fi rst stage. § Suppose we have a set of hypotheses 𝐼 1 , 𝐼 2 , ⋯ , 𝐼 5 , which are pairwise disjoint and such that Ω = 𝐼 1 ∪ 𝐼 2 ∪ ⋯ ∪ 𝐼 5 . We have a set of prior probabilities 𝑄 𝐼 1 , 𝑄 𝐼 2 , ⋯ , 𝑄(𝐼 5 ) for the hypotheses. 𝐼 % 𝐼 ! 𝐼 & 𝐼 # 𝐼 " XC 2020