Quick Review of Probability Geometric Distribution Coupon - PowerPoint PPT Presentation

Quick Review of Probability Anil Maheshwari Sample Space & Events Random Variable Quick Review of Probability Geometric Distribution Coupon Collector Problem Anil Maheshwari School of Computer Science Carleton University Canada

Outline Quick Review of Probability Anil Maheshwari Sample Space & Events Random Variable Sample Space & Events Geometric 1 Distribution Coupon Collector Problem Random Variable 2 Geometric Distribution 3 Coupon Collector Problem 4

Basic Definition Quick Review of Probability Anil Maheshwari Sample Space & Events Random Variable Definitions Geometric Sample Space S = Set of Outcomes. Distribution Coupon Collector Events E = Subsets of S . Problem Probability is a function from subsets A ⊆ S to positive real numbers between [0 , 1] such that: Pr ( S ) = 1 1 For all A, B ⊆ S if A ∩ B = ∅ , 2 Pr ( A ∪ B ) = Pr ( A ) + Pr ( B ) . If A ⊂ B ⊆ S , Pr ( A ) ≤ Pr ( B ) . 3 Probability of complement of A , Pr ( ¯ A ) = 1 − Pr ( A ) . 4

Basic Definition Quick Review of Probability Anil Maheshwari Sample Space & Examples: Events Flipping a fair coin: 1 Random Variable S = { H, T } ; Geometric Distribution E = {∅ , { H } , { T } , S = { H, T }} Coupon Collector Problem Flipping fair coin twice: 2 S = { HH, HT, TH, TT } ; E = {∅ , { HH } , { HT } , { TH } , { TT } , { HH, TT } , { HH, TH } , { HH, HT } , { HT, TH } , { HT, TT } , { TH, TT } , { HH, HT, TH } , { HH, HT, TT } , { HH, TH, TT } , { HT, TH, TT } , S = { HH, HT, TH, TT }} Rolling fair die twice: 3 S = { ( i, j ) : 1 ≤ i, j ≤ 6 } ; E = {∅ , { 1 , 1 } , { 1 , 2 } , . . . , S }

Expectation Quick Review of Probability Anil Maheshwari Definition Sample Space & Events A random variable X is a function from sample space S Random Variable to Real numbers, X : S → ℜ . Geometric Distribution Expected value of a discrete random variable X is given Coupon Collector by E [ X ] = � s ∈ S X ( s ) ∗ Pr ( X = X ( s )) . Problem Note: Its a misnomer to say X is a random variable, it’s a function. Example: Flip a fair coin and define the random variable X : { H, T } → ℜ as � 1 Outcome is Heads X = 0 Outcome is Tails s ∈{ H,T } X ( s ) ∗ Pr ( X = X ( s )) = 1 ∗ 1 2 +0 ∗ 1 2 = 1 E [ X ] = � 2

Linearity of Expectation Quick Review of Probability Anil Maheshwari Sample Space & Events Definition Random Variable Consider two random variables X, Y such that Geometric X, Y : S → ℜ , then E [ X + Y ] = E [ X ] + E [ Y ] . Distribution Coupon Collector In general, consider n random variables X 1 , X 2 , . . . , X n Problem such that X i : S → ℜ , then E [ � n i =1 X i ] = � n i =1 E [ X i ] . Example: Flip a fair coin n times and define n random variable X 1 , . . . , X n as � 1 Outcome is Heads X i = 0 Outcome is Tails E [ X 1 + · · · + X n ] = E [ X 1 ] + · · · + E [ X n ] = 1 2 + · · · + 1 2 = n 2 = Expected # of Heads in n tosses.

Geometric Distribuition Quick Review of Probability Anil Maheshwari Sample Space & Events Definition Random Variable Perform a sequence of independent trials till the first Geometric Distribution success. Each trial succeeds with probability p (and fails Coupon Collector with probability 1 − p ). Problem A Geometric Random Variable X with parameter p is defined to be equal to n ∈ N if the first n − 1 trials are failures and the n -th trial is success. Probability distribution function of X is Pr ( X = n ) = (1 − p ) n − 1 p . Let Z to be the r.v. that equals the # failures before the first success, i.e. Z = X − 1 . Problem: Evaluate E [ X ] and E [ Z ] . To show: E [ Z ] = 1 − p and E [ X ] = 1 + 1 − p = 1 p . p p

Computation of E [ Z ] Quick Review of Probability Anil Maheshwari Z = # failures before the first success. Set q = 1 − p . Sample Space & Events Pr ( Z = k ) = q k p Random Variable k =0 q k (for 0 < q < 1 ) 1 − q = � ∞ 1 Geometric Distribution (1 − q ) 2 = � ∞ 1 k =0 kq k − 1 Coupon Collector Problem ∞ � E [ Z ] = kPr ( Z = k ) k =0 ∞ � kq k p = k =0 ∞ � kq k − 1 = pq k =0 pq = (1 − q ) 2 1 − p = p

Examples Quick Review of Probability Anil Maheshwari Sample Space & Events Random Variable Examples: Geometric Distribution Flipping a fair coin till we get a Head: 1 Coupon Collector Problem p = 1 2 and E [ X ] = 1 p = 2 Roll a die till we see a 6 : 2 p = 1 6 and E [ X ] = 1 p = 6 Keep buying LottoMax tickets till we win (assuming 3 we have 1 in 33294800 chance). 33294800 and E [ X ] = 1 1 p = p = 33 , 294 , 800 .

Coupon’s Collector Problem Quick Review of Probability Anil Maheshwari Sample Space & Events Random Variable Problem Definition Geometric Distribution There are a total of n different types of coupons Coupon Collector (Pokemon cards). A cereal manufacturer has ensured Problem that each cereal box contains a coupon. Probability that a box contains any particular type of coupon is 1 n . What is the expected number of boxes we need to buy to collect all the n coupons? Define r.v. N 1 , N 2 , . . . , N n , where N i = # of boxes bought till the i -th coupon is collected. Each N i is a geometric random variable.

Coupon’s Collector Problem Contd. Quick Review of Probability Anil Maheshwari Sample Space & Events Random Variable Geometric Let N = � n j =1 N i ; Note N 1 = 1 Distribution Coupon Collector 1 1 E [ N j ] = Pr of success in finding the j th coupon = Problem n − j +1 n E [ N ] = � n n n − j +1 = nH n , where H n = n -th Harmonic j =1 Number. H n = � n 1 i and ln n ≤ H n ≤ ln n + 1 . i =1 Thus, E [ N ] = nH n ≈ n ln n ,

Is E [ N ] = nH n = n ln n a good estimate? Quick Review of Probability Anil Maheshwari Sample Space & Events What is the probability that E [ N ] exceeds 2 nH n ? Random Variable Applying Markov’s Inequality: Pr ( X > s ) ≤ E [ X ] Geometric s Distribution Pr ( N > 2 nH n ) < E [ N ] 2 nH n = nH n 2 nH n = 1 Coupon Collector 2 Problem Can we have a better bound? Next: We show Pr ( N > n ln n + cn ) < 1 e c Pr. of missing a coupon after n ln n + cn boxes have been n ) n ln n + nc ≤ e − 1 n ( n ln n + cn ) = bought = (1 − 1 1 ne c . Pr. of missing at least one coupon ≤ n ( 1 ne c ) = 1 e c .

References Quick Review of Probability Anil Maheshwari Sample Space & Events Random Variable Geometric Distribution Introduction to Probability by Blitzstein and Hwang, 1 Coupon Collector Problem CRC Press 2015. Courses Notes of COMP 2804 by Michiel Smid. 2 Probability and Computing by Mitzenmacher and 3 Upfal, Cambridge Univ. Press 2005. My Notes on Algorithm Design. 4