Conditional Probabilities Anders Ringgaard Kristensen Department of - PowerPoint PPT Presentation

Department of Veterinary and Animal Sciences Conditional Probabilities Anders Ringgaard Kristensen

Department of Veterinary and Animal Sciences Outline Probabilities Conditional probabilities Bayes’ theorem Slide 2

Department of Veterinary and Animal Sciences Probabilities: Basic concepts The probability concept is used in daily language. What do we mean when we say: • The probability of the outcome ”5” when rolling a dice is 1/6? • The probability that cow no. 543 is pregnant is 0.40? • The probability that USA will attack North Korea within 5 years is 0.05? Slide 3

Department of Veterinary and Animal Sciences Interpretations of probabilities At least 3 different interpretations are observed: • A “frequentist” interpretation: • The probability expresses how frequent we will observe a given outcome if exactly the same experiment is repeated a “large” number of times. The value is rather objective. • An objective belief interpretation: • The probability expresses our belief in a certain (unobservable) state or event. The belief may be based on an underlying frequentist interpretation of similar cases and thus be rather objective. • A subjective belief interpretation: • The probability expresses our belief in a certain unobservable (or not yet observed) event. Slide 4

Department of Veterinary and Animal Sciences ”Experiments” An experiment may be anything creating an outcome we can observe. The sample space, S, is the set of all possible outcomes. An event, A, is a subset of S, i.e. A ⊆ S Two events A 1 and A 2 are called disjoint , if they have no common outcomes, i.e. if A 1 ∩ A 2 = ∅ Slide 5

Department of Veterinary and Animal Sciences Example of experiment Rolling a dice: • The sample space is S = {1, 2, 3, 4, 5, 6} • Examples of events: • A 1 = {1} • A 2 = {1, 5} • A 3 = {4, 5, 6} • Since A 1 ∩ A 3 = ∅ , A 1 and A 3 are disjoint. • A 1 and A 2 are not disjoint, because A 1 ∩ A 2 = {1} Slide 6

Department of Veterinary and Animal Sciences A simplified definition Let S be the sample space of an experiment. A probability distribution P on S is a function, so that • P( S ) = 1. • For any event A ⊆ S, 0 ≤ P( A ) ≤ 1 • For any two disjoint events A 1 and A 2 , • P(A 1 ∪ A 2 ) = P(A 1 ) + P(A 2 ) Slide 7

Department of Veterinary and Animal Sciences Example: Rolling a dice Like before: S = {1, 2, 3, 4, 5, 6} A valid probability function on S is, for A ⊆ S : • P( A ) = | A |/6 where | A | is the size of A (i.e. the number of elements it contains) • P({1}) = P({2}) = P({3}) = P({4}) = P({5}) = P({6}) = 1/6 • P({1, 5}) = 2/6 = 1/3 • P({1, 2, 3}) = 3/6 = 1/2 Notice, that many other valid probability functions could be defined (even though the one above is the only one that makes sense from a frequentist point of view). Slide 8

Department of Veterinary and Animal Sciences Independence If two events A and B are independent, then • P( A ∩ B ) = P( A )P( B ). Example: Rolling two dices • S = {(1, 1), (1, 2),…, (1, 6),…, (6, 6)} • For any A ⊆ S : P( A ) = | A |/36 • A = {(6, 1), (6, 2), …, (6, 6)} ⇒ P( A ) = 6/36 = 1/6 • B = {(1, 6), (2, 6), …, (6, 6)} ⇒ P( B ) = 6/36 = 1/6 • A ∩ B = {(6, 6)} and P( A ∩ B) = (1/6)(1/6) = 1/36 Slide 9

Department of Veterinary and Animal Sciences Conditional probabilities Let A and B be two events, where P( B ) > 0 The conditional probability of A given B is written as P( A | B ), and it is by definition Slide 10

Department of Veterinary and Animal Sciences Example: Rolling a dice Again, let S = {1, 2, 3, 4, 5, 6}, and P( A ) = | A |/6. Define B = {1, 2, 3}, and A = {2}. Then A ∩ B = {2}, and The logical result: If you know the outcome is 1, 2 or 3, it is reasonable to assume that all 3 values are equally probable. Slide 11

Department of Veterinary and Animal Sciences Conditional sum rule Let A 1 , A 2 , … A n be pair wise disjoint events so that Let B be an event so that P( B ) > 0. Then Slide 12

Department of Veterinary and Animal Sciences Sum rule: Dice example Define the 3 disjoint events A 1 = {1, 2}, A 2 = {3, 4}, A 3 = {5, 6} Thus A 1 ∪ A 2 ∪ A 3 = S Define B = {1, 3, 5} (we know that P( B ) = ½) P( B | A 1 ) = P( B ∩ A 1 )/P( A 1 ) = (1/6)/(1/3) = ½ P( B | A 2 ) = P( B ∩ A 2 )/P( A 2 ) = (1/6)/(1/3) = ½ P( B | A 3 ) = P( B ∩ A 3 )/P( A 3 ) = (1/6)/(1/3) = ½ Thus Slide 13

Department of Veterinary and Animal Sciences Bayes’ theorem Let A 1 , A 2 , … A n be pair wise disjoint events so that Let B be an event so that P( B ) > 0. Then Bayes’ theorem is extremely important in all kinds of reasoning under uncertainty. Updating of belief. Slide 14

Department of Veterinary and Animal Sciences Updating of belief, I In a dairy herd, the conception rate is known to be 0.40. Define M as the event ”mating” for a cow. Define Π + as the event ”pregnant” for the same cow, and Π - as the event ”not pregnant”. Thus P( Π + | M ) = 0.40 is a conditional probability. Given that the cow has been mated, the probability of pregnancy is 0.40. Accordingly, P( Π - | M ) = 0.60 After 3 weeks the farmer observes the cow for heat. The farmer’s heat detection rate is 0.55. Define H + as the event that the farmer detects heat. Thus, P( H + | Π - ) = 0.55, and P( H - | Π - ) = 0.45 There is a slight risk that the farmer erroneously observes a pregnant cow to be in heat. We assume, that P( H + | Π + ) = 0.01 Notice, that all probabilities are figures that makes sense and are estimated on a routine basis (except P( H + | Π + ) which is a guess) Slide 15

Department of Veterinary and Animal Sciences Updating of belief, II Now, let us assume that the farmer observes the cow, and concludes, that it is not in heat. Thus, we have observed the event H - and we would like to know the probability, that the cow is pregnant, i.e. we wish to calculate P( Π + | H - ) We apply Bayes’ theorem: We know all probabilities in the formula, and get In other words, our belief in the event ”pregnant” increases from 0.40 to 0.59 based on a negative heat observation result Slide 16

Department of Veterinary and Animal Sciences Summary of probabilities Probabilities may be interpreted • As frequencies • As objective or subjective beliefs in certain events The belief interpretation enables us to represent uncertain knowledge in a concise way. Bayes’ theorem lets us update our belief (knowledge) as new observations are done. Slide 17