Probability and Statistics for Computer Science A major use of - PowerPoint PPT Presentation

Probability and Statistics ì for Computer Science “A major use of probability in sta4s4cal inference is the upda4ng of probabili4es when certain events are observed” – Prof. M.H. DeGroot Credit: wikipedia Hongye Liu, Teaching Assistant Prof, CS361, UIUC, 9.10.2020

Last time Probability More probability calculation • Conditional probability * Bayes rule ✓ * Independence

PC AIB ) = PCANBSP Objectives ( B ) � Condi4onal Probability joint prob . * Product rule of * Bayes rule * Independence

Counting: how many ways? chats hats 7 are to put indistinguishable ) 7 of lo on # dourly ? A € -1 :)

Warm up: which is larger? PC Al B ) PC An B ) or A ) PC An B ) B ) PC Al B ) c) unsure

Conditional Probability � The probability of A given B P ( A | B ) = P ( A ∩ B ) P ( B ) P ( B ) � = 0 The line-crossed area is the new sample space for condi4onal P(A| B)

Joint Probability Calculation ⇒ P ( A ∩ B ) = P ( A | B ) P ( B ) Pc Anb ) pl Al B ) . 8 × 0.5 o I ] - PCB ) - - ' disjoint - - II P ( soup ∩ meat ) = P ( meat | soup ) P ( soup ) = 0 . 5 × 0 . 8 = 0 . 4

Bayes rule � Given the defini4on of condi4onal probability and the symmetry of joint probability, we have: P ( A | B ) P ( B ) = P ( A ∩ B ) = P ( B ∩ A ) = P ( B | A ) P ( A ) And it leads to the famous Bayes rule: Plants P ( A | B ) = P ( B | A ) P ( A ) PCB ) P ( B )

Total probability Pc B) =p ( Bn A) t Pc Bn Ac ) C B IA ) PLA ) t p CB IAC ) PCA ' ) =p i. e . A = It , r A 1 A 3 B A 2

Total probability general form PCBN Aj ) PL B) = I j 5g PCB l Aj > PCAJ ) = Ain B A 1 tiny A 3 B , * A 2

Total probability: I soup , Pc soup n neat parent p ( meat ) = ? + pain : un meat o - C meat I soup ) p c soup , P p ( soup ' - + pc meat I juice ) P l juice ) disjoint p c soup I meat ) - mile ) P " = p e meat n som ' - E-

Bayes rule using total prob. PCBI Aj ) PCAJ ) = # PCAJIB ) = PlBlAj)pcAj# p" > pcttj ) PCBI Aj pub ) I j → disjoint = Of Ajntti it iej

Bayes rule: rare disease test There is a blood test for a rare disease. The frequency of the disease is 1/100,000. If one has it, the test confirms it with probability 0.95. If one doesn't have, the test gives false posi4ve with probability 0.001. What is , the probability P ( D | T ) of having disease given a posi4ve test result? P ( D | T ) = P ( T | D ) P ( D ) Using total prob. P ( T ) P ( T | D ) P ( D ) = P ( T | D ) P ( D ) + P ( T | D c ) P ( D c )

Bayes rule: rare disease test There is a blood test for a rare disease. The frequency of the disease is 1/100,000 . If one has it, the test confirms it with probability 0.95. If one o doesn't have, the test gives false posi4ve with probability 0.001 . What is , the probability P ( D | T ) of having disease given a posi4ve test result? ① P ( T | D ) P ( D ) P ( D | T ) = P ( T | D ) P ( D ) + P ( T | D c ) P ( D c ) = o÷÷xF o .gfX ' hoon = , ,g .

could test ? about what . - tf Cour 'd =/ freq et Suppose . - 95% accuracy = test = T ool positive false - - TIPI . .gs P' " = ' -2% P PIPIT ) = ? ' )=o . - ol pctlp Z T - pipa ) PCTLD ) PCD ) - 21% = # I = , - PCTID > PCDHPCTIDC ) pep ) 99 . -

Independence � One defini4on: P ( A | B ) = P ( A ) or - - P ( B | A ) = P ( B ) Whether A happened doesn’t change the probability of B and vice versa

Independence: example � Suppose that we have a fair coin and it is tossed twice. let A be the event “the first toss is a head” and B the event “the two outcomes are the same.” A 1 € ' " . r:÷÷÷÷÷=÷¥ ' i' IIE ' I ✓ � These two events are independent!

Independence � Alterna4ve defini4on P ( A | B ) = P ( A ) LHS by defini4on ⇒ P ( A ∩ B ) = P ( A ) P ( B ) 12HnB)=ppcBgyg#** ⇒ P ( A ∩ B ) = P ( A ) P ( B )

( Testing Independence: � Suppose you draw one card from a standard deck of cards. E 1 is the event that the card is a King, Queen or Jack. E 2 - is the event the card is a Heart. Are E 1 #nM=pcH ) PIB ) and E 2 independent? pi Ein Er ) = # - Pp "i÷'¥÷=÷s¥¥= 's

⇒ Independence vs Disjoint � Q. Two disjoint events that have probability> 0 are certainly dependent to each other. ol pc Anny A. True E) = o B. False pimp o > a pas ) p can b) f p Cbs PCB ) o = #

Independence of empty event � Q. Any event is independent of empty event B. D A. True a- ol B. False p ( 0/1=0 Ion H C B NH ) P = plot ) in = o p

Pairwise independence is not mutual independence in larger context (÷ AV.AZ#3UAy=zggpcAnB)=pCAspcBjv A 1 A 2 P( A 1 ) = P( A 2 ) = P( A 3 ) = P( A 4 ) = 1/4 A 3 A 4 - ① - =L .cl?-pcb3pccgv.C--tu.C%cann4--PUtpY..--pFmp.adEP A = A 1 ∪ A 2 ; P ( A ) = 1 PCB ? 2 B = A 1 ∪ A 3 ; P ( B ) = 1 2 C = A 1 ∪ A 4 ; P ( C ) = 1 2 * P ( ABC ) is the shorthand for P ( A ∩ B ∩ C )

⇒ ④ ⇒ it Mutual independence � Mutual independence of a collec4on of events is : A 1 , A 2 , A 3 ...A n - Ap ) =p ( Ai ) |pcAilAjn Ann / - - P can Bnc ) BCA ) p ( A1 Bnc ) = - =p LA , j, k, ...p � = i penny I - � It’s very strong independence! ⇒ PLANE .IT#pco3n4pcA1pcBjpiy

Probability using the property of Independence: Airline overbooking (1) � An airline has a flight with 6 seats. They always sell 7 4ckets for this flight. If 4cket we holders show up independently with probability p , what is the probability that the flight is overbooked ? (7) = 1 . 7 y . P - Ail " C Ai - p ( Az ) p - - =p CA . ) pc Au ) - - =p_p.r--p#

Probability using the property of Independence: Airline overbooking (1) � An airline has a flight with 6 seats. They always sell 7 4ckets for this flight. If 4cket holders show up independently with probability p , what is the probability that the flight is overbooked ? P( 7 passengers showed up)

Probability using the property of Independence: Airline overbooking (2) � An airline has a flight with 6 seats. They always sell 8 4ckets for this flight. If 4cket - o holders show up independently with probability p , what is the probability that exactly 6 people showed up? Cl - p > Ctp ) - p6 ftp.IP.P-rp.p.p P(6 people showed up) = = ( 8)

Probability using the property of Independence: Airline overbooking (3) � An airline has a flight with 6 seats. They always sell 8 4ckets for this flight. If 4cket holders show up independently with probability p , what is the probability that the flight is overbooked ? P( overbooked) =

Probability using the property of Independence: Airline overbooking (4) � An airline has a flight with s seats. They O always sell t ( t > s ) 4ckets for this flight. If O 4cket holders show up independently with probability p , what is the probability th that exactly u people showed up? for P( exactly u people showed up) E- a " (E) e i - p , p .

Probability using the property of Independence: Airline overbooking (5) � An airline has a flight with s seats. They G- always sell t ( t > s ) 4ckets for this flight. If 4cket holders show up independently with probability p , what is the probability that the flight is overbooked ? FT . . ?¥* ¥ s P( overbooked) lil r ; a - pit - u - ¥4 . u

Condition may affect Independence � Assume event A and B are pairwise independent Given C , A and B are r not independent C A any more because they become B disjoint

Conditional Independence � Event A and B are condi4onal independent given event C if the following is true. P ( A ∩ B | C ) = P ( A | C ) P ( B | C ) See an example in Degroot et al. Example 2.2.10

Assignments � HW3 - � Finish Chapter 3 of the textbook � Next 4me: Random variable

Additional References � Charles M. Grinstead and J. Laurie Snell "Introduc4on to Probability” � Morris H. Degroot and Mark J. Schervish "Probability and Sta4s4cs”

Another counting problem � There are several (>10) freshmen, sophomores, juniors and seniors in a dormitory. In how many ways can a team of 10 students be chosen to represent the dorm? There are no dis4nc4on to make between each individual student other than their year in school.

See you next time See You!