CIC Experiments Experiments and events and events ❑ Tossing a Coin Until an H is Obtained . If we toss a coin until an H is obtained, we cannot say in advance how many tosses will be required, and so the natural sample space is S = {H, TH, TTH, TTTH, . . . }, an infinite set. We can use, of course, many other sample spaces as well, for instance, we may be interested only in whether we had to toss the coin more than twice or not, in which case S = {1 or 2, more than 2} is adequate. ❑ Selecting a Number from an Interval . Sometimes, we need an uncountable set for a sample space. For instance, if the experiment consists of choosing a random number between 0 and 1, we may use S = {x : 0 < x < 1}. 30
CIC The The probability probability law law ❑ Specifies the “likelihood” of any outcome, or of any set of possible outcomes. ❑ Assigns to every event A, a number P(A), called the probability of A. 31
CIC Pr Probabil obabilit ity y Space Space [Schay Schay 2007] 2007] ❑ Given a sample space S and a certain collection ℱ of its subsets, called events, an assignment P of a number P(A) to each event A in ℱ is called a probability measure, and P(A) the probability of A, if P has the following properties: 1. P(A) ≥ 0 for every A, 2. P(S) = 1, and 3. P(A1 ∪ A2 ∪ · · · ) = P(A1)+ P(A2) + ·· · for any finite or countably infinite set of mutually exclusive events A1, A2, … Then, the sample space S together with ℱ and P is called a probability space . 32
CIC Probabilit Probability y Axioms Axioms [Ber Berts tsekas ekas and and Tsit Tsitsikli siklis, , 2008] 2008] S P (S) = 1. 33
CIC Pr Probabil obabilit ity y Space Space [Blitzstein [Blit zstein and Hwang, 2015] and Hwang, 2015] ❑ Definition 1.6.1 ( General definition of probability ). A probability space consists of a sample space S and a probability function P which takes an event A S as input and returns P(A), a real number between 0 and 1, as output. The function P must satisfy the following axioms: 1. P( ) = 0, P(S) = 1. 2. If A 1 , A 2 , . . . are disjoint events, then: (Saying that these events are disjoint means that they are mutually exclusive: A i ∩ A j = for i ≠ j.) 34
CIC Proper Properties ties of probabilit of probabilities ies ❑ The Probability of the Empty Set Is 0 . In any probability space , P( ∅ ) = 0. ❑ Proof: 1 = P(S) = P(S ∪ ) = P(S) + P( ) = 1 + P( ) 35
CIC Proper Properties ties of probabilit of probabilities ies ❑ The Probability of the Union of Two Events . For any two events A and B, P(A ∪ B) = P(A) + P(B) − P(A ∩ B ) Proof: 36
CIC Proper Properties ties of probabilit of probabilities ies ❑ Probability of Complements. For any event A, P(A c ) = 1 − P(A ) Proof: A c ∩ A = ∅ and A c ∪ A = S by the definition of A c . Thus, by Axiom 3, P(S) = P(A c ∪ A) = P(A c ) + P(A). Now, Axiom 2 says that P(S) = 1, and so, comparing these two values of P(S), we obtain P(A c ) + P(A) = 1. 37
CIC Proper Properties ties of probabilit of probabilities ies ❑ Probability of Subsets . If A ⊂ B, then P(A) ≤ P(B). Proof: If A B, then we can write B as the union of A and B ∩ A c , where B ∩ A c is the part of B not also in A. Since A and B ∩ A c are disjoint, we can apply the second axiom: P(B) = P(A ∪ (B ∩ A c )) = P(A) + P(B ∩ A c ) Probability is nonnegative, so P(B ∩ A c ) ≥ 0, proving that P(B) ≥ P(A). 38
CIC Proper Properties ties of probabilit of probabilities ies ❑ Inclusion-exclusion . For any events A1, . . . ,An, 39
CIC Proper Properties ties of probabilit of probabilities ies ❑ Example: 40
CIC Proper Properties ties of Pr of Probability obability Laws Laws 41
CIC Discret Discrete e Probabilit Probability y Law Law 42
CIC Discret Discrete e Uniform Uniform Probabilit Probability y Law Law ❑ In the special case where the probabilities P( s 1 ), …, P( s n ) are all the same, by necessity equal to 1/ n , in view of the normalization axiom, we obtain: 43
CIC Discret Discrete e Uniform Uniform Probabilit Probability y Law Law 44
CIC Discret Discrete e Uniform Uniform Probabilit Probability y Law Law 45
CIC Counting Counting ❑ The calculation of probabilities often involves counting the number of outcomes in various events. ➢ When the sample space S has finite number of equally likely outcomes, so that the discrete uniform probability law applies. Then, the probability of any event A is given by: number of elements of 𝐵 𝑙 𝑄 𝐵 = number of elements of 𝑇 = 𝑜 ➢ When we want to calculate the probability of an event A with a finite number of equally likely outcomes, each of which has an already known probability p . Then the probability of A is given by: 𝑄 𝐵 = 𝑞 ∙ (number of elements of 𝐵) 46
CIC Basic Counting Principle ❑ In how many ways you can dress today if you find: ➢ 4 shirts ➢ 3 ties ➢ 2 jackets in your closet? 47
CIC The The Multiplicat Multiplication ion Principle Principle ❑ Consider a process that consists of r stages. Suppose that: a) There are n 1 possible results at the firs stage. b) For every possible result at the first stage, there are n 2 possible results at the second stage. c) More generally, for any sequence of possible results at the first i ˗ 1 stage, there are n i possible results at the i th stage. Then, the total number of possible results of the r -stage process is: 𝑜 1 𝑜 2 ⋯ 𝑜 𝑠 48
CIC The The Multiplicat Multiplication ion Principle Principle 49
CIC The The Multiplicat Multiplication ion Principle Principle ❑ Example 1. The number of telephone numbers. A local telephone company number is a 7-digit sequence, but the first digit has to be different from 0 or 1. How many distinct telephone numbers are there? 50
CIC The The Multiplicat Multiplication ion Principle Principle ❑ Example 2. The number of subsets of an n - element set. Consider an n -element set {s 1 , s 2 ,…, s n }. ❑ How many subsets it have, including itself and the empty set? ❑ Example, in the set {1,2,3}? 51
CIC The The Multiplicat Multiplication ion Principle Principle ❑ This is a sequential process where we take in turn each of the n elements and decide whether to include it in the desired subset or not. ❑ Thus we have n steps, and in each step two choices, namely yes or no to the question of whether the element belongs to the desired subset. Therefore the number of subsets is: ❑ for n = 1? 52
CIC Number Number of subset of subsets ❑ Example 3. Drawing three cards. What is the number of ways three cards can be drawn one after the other from a regular 52 cards deck without replacement? ❑ What is this number if we replace each card before the next one is drawn? 53
CIC Number Number of subset of subsets ❑ Example 3. Drawing three cards. What is the number of ways three cards can be drawn one after the other from a regular 52 cards deck without replacement? 52 51 50 n 1 = 52, n 2 = 51, n 3 = 50 ❑ What is this number if we replace each card before the next one is drawn? n 1 = n 2 = n 3 = 52 52 3 54
CIC Permut Permutation ation and Combination and Combination ❑ Involve the selection of k objects out of a collection of n objects. ❑ If the order of selection matters, the selection is called a permutation . ❑ If the order of selection does not matter, the selection is called a combination . 55
CIC Permut Permutation ation k permutations ❑ Assume there are n distinct objects, and let k be some positive integer with k n . ❑ We want to count the number of different ways that we can pick k out of these n objects and arrange them in a sequence, e.g. the number of distinct k -object sequences. 56
CIC Permut Permutation ation ➢ In place 1 we can put n objects, which we can write as n −1+1 ; ➢ In place 2 we can put n −1 = n − 2+1 objects; and so on. ❑ Thus the k th factor will be n − k + 1, and so, for any 2 positive integers n and k ≤ n : n ( n − 1)( n − 2) · · · ( n − k + 1) = P n , k ❑ In the special case where k = n : n ( n − 1)( n − 2) · · · 3 · 2 · 1 = n ! ❑ The number of possible sequences is simple called permutations 57
CIC Permut Permutation ation ❑ From the definitions of n !, ( n − k )! and P n , k we can obtain the following relation: n ! = [ n ( n − 1)( n − 2 ) · · · ( n − k + 1)][( n − k )( n − k − 1 ) · · · 2 · 1] = P n , k · ( n − k )! ❑ and so: 𝑜! 𝑄 𝑜,𝑙 = 𝑜 − 𝑙 ! with 0! = 1. 58
CIC Probabilit Probability y calculation calculation ❑ Example 4. Six rolls of a die. Find the probability that: ➢ Six rolls of a (six sided) die all give different numbers ➢ Assume all outcomes are equally likely P(all six rolls give different numbers) = ? number of elements of 𝐵 𝑙 𝑄 𝐵 = number of elements of 𝑇 = 𝑜 𝑄 𝐵 = 𝑞 ∙ (number of elements of 𝐵) p = probability of each equally likely outcome in A 59
CIC Probabilit Probability y calculation calculation ❑ Example 4. Six rolls of a die. Find the probability that: ➢ Six rolls of a (six sided) die all give different numbers ➢ Assume all outcomes are equally likely P(all six rolls give different numbers) = ? 𝑄 6,6 number of elements of 𝐵 𝑙 6! 𝑄 𝐵 = number of elements of 𝑇 = 𝑜 = # 𝑓𝑚𝑓𝑛𝑓𝑜𝑢𝑡 𝑗𝑜 𝑇 = 6 6 1 𝑄 𝐵 = 𝑞 ∙ number of elements of 𝐵 = 6 6 6! p = probability of each equally likely outcome in A 60
CIC Permut Permutation ation ❑ Example 5. Dealing Three Cards. In how many ways can three cards be dealt from a regular deck of 52 cards? 61
CIC Permut Permutation ation ❑ Example 5. Dealing Three Cards. In how many ways can three cards be dealt from a regular deck of 52 cards? 𝑜! P 52,3 = 𝑄 𝑜,𝑙 = 𝑜−𝑙 ! = 52·51·50 = 132, 600. 62
CIC Permut Permutation ation ❑ Example 6. Birthday problem. There are k people in a room. Assume each person’s birthday is equally likely to be any of the 365 days of the year (we exclude February 29), and that people’s birthdays are independent (we assume there are no twins in the room). What is the probability that two or more people in the group have the same birthday? 63
CIC Permut Permutation ation ❑ This amounts to sampling the 365 days of the year without replacement, so: 365 · 364 · 363 · · · (365− k +1) for k 365 Therefore the probability of no birthday matches in a group of k people is: and the probability of at least one birthday match is: 64
CIC Permut Permutation ation Probability that in a room of k people, at least two were born on the same day. This probability first exceeds 0.5 when k = 23. 65
CIC Combinations Combinations ❑ The number of possible unordered selections of k different things out of n different ones is denoted by C n,k , and each such selection is called a combination of the given things. ❑ If we select k things out of n without regard to order, then, this can be done in C n,k ways. ❑ In each case we have k things which can be ordered k ! ways. ❑ Thus, by the multiplication principle, the number of ordered selections is C n,k · k ! ❑ On the other hand, this number is, by definition, P n,k . Therefore C n,k · k ! = P n,k , and so: 𝐷 𝑜,𝑙 = 𝑄 𝑜,𝑙 𝑜! 𝑙! = 𝑙! 𝑜 − 𝑙 ! 66
CIC Combinations Combinations ❑ The quantity on the right-hand side is usually abbreviated as 𝑜 𝑙 , and is called a binomial coefficient. ❑ Thus, for any positive integer n and k = 1, 2, . . . , n : 𝑙 = 𝑜(𝑜 − 1)(𝑜 − 2) ⋯ (𝑜 − 𝑙 + 1) 𝐷 𝑜,𝑙 = 𝑜 𝑙! 𝑜! = 𝑙! 𝑜 − 𝑙 ! n ! = [ n ( n − 1)( n − 2) · · · ( n − k + 1)][( n − k )( n − k − 1) · · · 2 · 1] 67
CIC Combinations Combinations 68
CIC Binomial Binomial probabiliti probabilities es ❑ Binomial coefficient 𝑜 𝑙 → Binomial probabilities ❑ n 1 independent coin tosses: P(H) = p ; P( k heads) = ? ❑ Example: P(HTTTHH) = ? ❑ P(particular sequence) = ? ❑ P(particular k -head sequence) = ? 69
CIC Binomial Binomial probabiliti probabilities es ❑ Binomial coefficient 𝑜 𝑙 → Binomial probabilities ❑ n 1 independent coin tosses: P(H) = p ; P( k heads) = ? ❑ Example: P(HTTTHH) = p (1-p)(1-p)(1-p) p p ❑ P(particular sequence) = p #H (1-p) #T ❑ P(particular k -head sequence) = p k (1-p) n-k ❑ P(k heads) = p k (1-p) n-k (# k-head sequence) = 𝑜 𝑙 p k (1-p) n-k 70
CIC Part Partitions itions ❑ A combination can be seen as a partition of the set in two: one part contains k elements and the other contains the remaining n ˗ k elements. ❑ Given an n -element set and nonnegative integers n 1 , n 2 , …, n r , whose sum is equal to n ; consider partitions of the set into r disjoint subsets, with the i th subset containing exactly n i elements. ❑ In how many ways this can be done? 71
CIC Part Partitions itions ❑ There are 𝑜 𝑜 1 ways of forming the first subset. ❑ Having formed the first subset, there are left n – n 1 elements. We need to choose n 2 of them in order to form the second subset, and have 𝑜 − 𝑜 1 choices, and so on. 𝑜 2 ❑ Thus, using the Counting Principle: 72
CIC Part Partitions itions ❑ As several terms cancel, it results: ❑ This is called the multinomial coefficient and is usually denoted by: 73
CIC Part Partitions itions 74
CIC Part Partitions itions ❑ Example 7. Each person gets an ace. There is a 52- card deck, dealt (fairly) to four players. What is the probability of each player getting an ace? 75
CIC Part Partitions itions ❑ Example 7. Each person gets an ace. There is a 52- card deck, dealt (fairly) to four players. What is the probability of each player getting an ace? 52! ➢ The size of the sample space is: 13!13!13!13! ➢ Constructing an outcome with one ace for each person: o # of different ways of distributing the 4 aces to 4 players: 4! 48! o Distribution of the remaining 48 cards: 12!12!12!12! 76
CIC Summary Summary of Counting Results of Counting Results 77
CIC Conditional Conditional Probabilit Probability ❑ Conditional probability provides us with a way to reason about the outcome of an experiment, based on partial information. ❑ Examples: ❑ A) In an experiment involving two successive rolls of a die, you are told that the sum of the two rolls is 9. How likely is that the first roll was 6? ❑ B) In a word guessing game, the first letter of the word is a “t”. What is the likelihood that the second letter is an “h”? 78
CIC Conditional Conditional Probabilit Probability ❑ C) How likely is it that a person has certain disease given that a medical test was negative? ❑ D) A spot shows up on a radar screen. How likely is it to correspond to an aircraft? 79
CIC Conditional Conditional Probabilit Probability ❑ Given: ➢ An experiment ➢ A corresponding sample space ➢ A probability law ➢ We know that the outcome is within some given event B. ❑ Quantify the likelihood that the outcome also belongs to some other given event A. 80
CIC Conditional Conditional Probabilit Probability y ❑ Construct a new probability law that takes into account the available knowledge. ➢ A probability law that for any event A, specifies the conditional probability of A given B, P(A|B). ❑ The conditional probabilities P(A|B) of different events A should satisfy the probability axioms. 81
CIC Conditional Conditional Probabilit Probability y ❑ Example: ➢ Suppose that all six possible outcomes of a fair die roll are equally likely. ➢ If the outcome is even, then there are only three possible outcomes: 2, 4 and 6. ➢ What is the probability of the outcome being 6 given that the outcome is even? 82
CIC Conditional Conditional Probabilit Probability y ❑ If all possible outcomes are equally likely: ❑ Conditional probability definition : ➢ With P (B) > 0. ❑ The total probability of the elements of B, P(A|B) is the fraction that is assigned to possible outcomes that also belong to A. 83
CIC Conditional Conditional Probabilit Probability y ❑ Probability law of conditional probabilities satisfy the three axioms: P(A|B ) ≥ 0 for every event A , 1. P(S|B) = 1, 2. P(A 1 ∪ A 2 ∪ ·· · |B) = P(A 1 |B)+ P(A 2 |B) + ·· · for 3. any finite or countably infinite number of mutually exclusive events A 1 , A 2 , . . . . 84
CIC Conditional Conditional Probabilit Probability y ❑ Proofs: 1. In the definition of P(A|B) the numerator is nonnegative by Axiom 1, and the denominator is positive by assumption. Thus, the fraction is nonnegative. 2. Taking A = S in the definition of P(A|B), we get: 85
CIC Conditional Conditional Probabilit Probability y 3. 86
CIC Conditional Conditional Probabilit Probability y Knowledge that event B has occurred implies that the outcome of the experiment is in the set B . In computing P(A|B) we can therefore view the experiment as now having the reduced sample space B. The event A occurs in the reduced sample space if and only if the outcome ζ is in A ∩ B. The equation simply renormalizes the probability of events that occur jointly with B. 87
CIC Conditional Conditional Probabilit Probability y Suppose that we learn that B occurred. Upon obtaining this information, we get rid of all the pebbles in B c because they are incompatible with the knowledge that B has occurred. Then P(A∩B) is the total mass of the pebbles remaining in A. Finally, we renormalize, that is, divide all the masses by a constant so that the new total mass of the remaining pebbles is 1. This is achieved by dividing by P(B), the total mass of the pebbles in B. The updated mass of the outcomes corresponding to event A is the conditional probability P(A|B) = P(A∩B )/P(B). 88
CIC Conditional Conditional Probabilit Probability y ❑ If we interpret probability as relative frequency: ➢ P(A|B) should be the relative frequency of the event P(A∩B) in experiments where B occurred . ➢ Suppose that the experiment is performed n times, and suppose that event B occurs n B times, and that event A ∩ B occurs n A ∩ B times. The relative frequency of interest is then: ➢ where we have implicitly assumed that P(B) > 0. 89
CIC Conditional Conditional Probabilit Probability y ❑ Example 1. Given the figure below, obtain P(A|B) 90
CIC Conditional Conditional Probabilit Probability y ❑ Example 2. A ball is selected from an urn containing two black balls, numbered 1 and 2, and two white balls, numbered 3 and 4. The number and color of the ball is noted, so the sample space is {(1, b ),(2, b ), (3, w ), (4, w )}. Assuming that the four outcomes are equally likely, find P(A|B) and P(A|C), where A, B, and C are the following events: 91
CIC Conditional Conditional Probabilit Probability y ❑ Example 3. From all families with three children, we select one family at random. What is the probability that the children are all boys, if we know that a) the first one is a boy, and b) at least one is a boy? (Assume that each child is a boy or a girl with probability 1/2, independently of each other.) 92
CIC Conditional Conditional Probabilit Probability y ❑ Example 4. A card is drawn at random from a deck of 52 cards. What is the probability that it is a King or a 2, given that it is a face card (J, Q, K) ? 93
CIC Total Probability Total Probability Theorem and Theorem and Bayes’ Rule ❑ If we multiply both sides of the definition of P(A|B) by P(B) we obtain: P(A ∩ B) = P(A|B) P(B) ❑ Similarly, if we multiply both sides of the definition of P(B|A) by P(A) we obtain: P(B ∩ A) = P(B|A) P(A) 94
CIC Total Probability Total Probability Theorem and Theorem and Bayes’ Rule ❑ Joint Probability of Two Events. For any events A and B with positive probabilities: P(A ∩ B) = P(B ) P(A|B) = P(A) P(B|A) ❑ Joint Probability of Three Events P(A∩B∩C) = P(A) P(B|A) P(C|A∩B) P(A 1 ∩A 2 ∩A 3 ) = P(A 1 ) P(A 2 |A 1 ) P(A 3 |A 1 ∩A 2 ) 95
CIC Total Probability Total Probability Theorem and Theorem and Bayes’ Rule ❑ Applying repeatedly, we can generalise to the intersection of n events. 96
CIC Total Probability Total Probability Theorem and Theorem and Bayes’ Rule 97
CIC Total Probability Total Probability Theorem Theorem ❑ Total Probability Theorem: 98
CIC Total Probability Total Probability Theorem Theorem ❑ P(B) = P(A 1 ) P(B|A 1 ) + · · · + P(A n ) P(B|A n ) The probability that B occurs is a weighted average of its conditional probability under each scenario, where each scenario is weighted according to its (unconditional) probability. ➢ The A i partition the sample space; P(B) is equal to: 99
CIC Total Probability Total Probability Theorem Theorem 100
Recommend
More recommend
