Balls & Bins Geometric Distribution Coupon Collector Problem - PowerPoint PPT Presentation

Balls & Bins Anil Maheshwari Basics Random Variable Balls & Bins Geometric Distribution Coupon Collector Problem Balls & Bins Anil Maheshwari Collisions Size of Bins School of Computer Science Carleton University Canada

Outline Balls & Bins Anil Maheshwari Basics Random Variable Basics 1 Geometric Distribution Coupon Collector Random Variable 2 Problem Balls & Bins Collisions Size of Bins Geometric Distribution 3 Coupon Collector Problem 4 Balls & Bins 5 Collisions Size of Bins

Basic Definition Balls & Bins Anil Maheshwari Basics Definitions Random Variable Sample Space S = Set of Outcomes. Geometric Distribution Events E = Subsets of S . Coupon Collector Probability is a function from subsets A ⊆ S to positive Problem real numbers between [0 , 1] such that: Balls & Bins Collisions Pr ( S ) = 1 Size of Bins 1 For all A, B ⊆ S if A ∩ B = ∅ , 2 Pr ( A ∪ B ) = Pr ( A ) + Pr ( B ) . If A ⊂ B ⊆ S , Pr ( A ) ≤ Pr ( B ) . 3 Probability of complement of A , Pr ( ¯ A ) = 1 − Pr ( A ) . 4

Examples Balls & Bins Anil Maheshwari Basics Flipping a fair coin: 1 Random Variable S = { H, T } ; Geometric E = {∅ , { H } , { T } , S = { H, T }} Distribution Flipping fair coin twice: Coupon Collector 2 Problem S = { HH, HT, TH, TT } ; Balls & Bins E = {∅ , { HH } , { HT } , { TH } , { TT } , Collisions Size of Bins { HH, TT } , { HH, TH } , { HH, HT } , { HT, TH } , { HT, TT } , { TH, TT } , { HH, HT, TH } , { HH, HT, TT } , { HH, TH, TT } , { HT, TH, TT } , S = { HH, HT, TH, TT }} Rolling fair die twice: 3 S = { ( i, j ) : 1 ≤ i, j ≤ 6 } ; E = {∅ , { 1 , 1 } , { 1 , 2 } , . . . , S }

Expectation Balls & Bins Anil Maheshwari Basics Definition Random Variable A random variable X is a function from sample space S Geometric Distribution to real numbers, X : S → ℜ . Coupon Collector Expected value of a discrete random variable X : Problem E [ X ] = � X ( s ) ∗ Pr ( X = X ( s )) . Balls & Bins Collisions s ∈ S Size of Bins Example: Flip a fair coin. Let r.v. X : { H, T } → ℜ be � 1 Outcome is Heads X = 0 Outcome is Tails X ( s ) ∗ Pr ( X = X ( s )) = 1 ∗ 1 2 + 0 ∗ 1 2 = 1 � E [ X ] = 2 s ∈{ H,T }

Linearity of Expectation Balls & Bins Anil Maheshwari Basics Consider two random variables X, Y : S → ℜ , then Random Variable E [ X + Y ] = E [ X ] + E [ Y ] . Geometric Distribution In general, consider n random variables X 1 , X 2 , . . . , X n n n Coupon Collector � � Problem such that X i : S → ℜ , then E [ X i ] = E [ X i ] . i =1 i =1 Balls & Bins Collisions Size of Bins Example: Flip a fair coin n times and define n random variable X 1 , . . . , X n as � 1 Outcome is Heads X i = 0 Outcome is Tails E [ X 1 + · · · + X n ] = E [ X 1 ] + · · · + E [ X n ] = 1 2 + · · · + 1 2 = n 2 (Expected # of Heads in n tosses)

Geometric Distribuition Balls & Bins Anil Maheshwari Basics Definition Random Variable Perform a sequence of independent trials till the first Geometric Distribution success. Each trial succeeds with probability p (and fails Coupon Collector with probability 1 − p ). Problem A geometric r.v. X with parameter p is defined to be Balls & Bins Collisions equal to n ∈ N if the first n − 1 trials are failures and the Size of Bins n -th trial is success. Probability distribution function of X is Pr ( X = n ) = (1 − p ) n − 1 p . Let Z to be the r.v. that equals the # failures before the first success, i.e. Z = X − 1 . Problem: Evaluate E [ X ] and E [ Z ] .

Computation of E [ Z ] Balls & Bins Anil Maheshwari Z = # failures before the first success. Basics To show: E [ Z ] = 1 − p and E [ X ] = 1 + E [ Z ] = 1 Random Variable p p Geometric Distribution Coupon Collector Problem Balls & Bins Collisions Size of Bins

Examples Balls & Bins Anil Maheshwari Examples: Basics Random Variable Flipping a fair coin till we get a Head: 1 Geometric p = 1 2 and E [ X ] = 1 p = 2 Distribution Coupon Collector Roll a die till we see a 6 : 2 Problem p = 1 6 and E [ X ] = 1 p = 6 Balls & Bins Collisions Size of Bins Keep buying LottoMax tickets till we win (assuming 3 we have 1 in 33294800 chance). 33294800 and E [ X ] = 1 1 p = p = 33 , 294 , 800 .

Coupon’s Collector Problem Balls & Bins Anil Maheshwari Basics Problem Definition Random Variable A cereal manufacturer has ensured that each cereal box Geometric Distribution contains a coupon among a possible n coupon types. Coupon Collector Probability that a box contains any particular type of Problem coupon is 1 n . Show that the expected number of boxes Balls & Bins Collisions that we need to buy to collect all the n coupons is n ln n . Size of Bins

Is E [ N ] = nH n = n ln n a good estimate? Balls & Bins Anil Maheshwari Basics Random Variable Geometric Distribution Coupon Collector Problem Balls & Bins Collisions Size of Bins

Balls & Bins Balls & Bins Anil Maheshwari Basics Model Random Variable We have m Balls and n Bins. We throw each ball in a bin Geometric Distribution uniformly at random. Coupon Collector Problem What is the probability of following events: Balls & Bins Collisions Balls i and j are in the same bin. 1 Size of Bins Bin # i receives (a) 0 balls, (b) k balls, and (c) ≥ k 2 balls. All bins have ≤ c ln n ln ln n balls. 3 Applications: Birthday Paradox, Load Balancing, Perfect Hashing

Probability[Balls i and j in the same bin] Balls & Bins Anil Maheshwari Number of Balls = m Basics Number of Bins = n . Random Variable Geometric Distribution Pr [ Balls i and j in same bin ] = 1 Coupon Collector n Problem Balls & Bins Collisions Size of Bins

Expected number of collisions Balls & Bins Anil Maheshwari Number of Balls = m Basics Number of Bins = n Random Variable � m Show that Expected number of collisions is 1 � Geometric n 2 Distribution Coupon Collector Problem Balls & Bins Collisions Size of Bins

Birthday Paradox Balls & Bins Anil Maheshwari Number of Balls = m = Number of Students Basics Number of Bins = n = Number of days in a Year. Random Variable Geometric For two students to have same Birthday: Distribution What value of m will result in E [ X ] = 1 � m � ≥ 1 Coupon Collector n 2 Problem � 28 1 � Balls & Bins Answer: m = 28 , since E [ X ] = = 1 . 04 > 1 365 2 Collisions Size of Bins

Birthday Paradox Contd. Balls & Bins Anil Maheshwari What is minimum value of m so that the probability that Basics two students share the same birthday is ≥ 1 2 ? Random Variable Geometric Distribution Coupon Collector Problem Balls & Bins Collisions Size of Bins

Number of Balls in Bin i Balls & Bins Anil Maheshwari Number of Balls = m ; Number of Bins = n . Basics Random Variable Problem I Geometric Distribution What is the probability that Bin i receives no balls? Coupon Collector Problem � m � Balls & Bins 1 − 1 ≤ e − m Collisions n Size of Bins n n ) n ≤ e − 1 = 0 . 37 . If n = m , (1 − 1 Problem II What is the probability that Bin i receives exactly k balls? � � 1 � m − k � k � � m 1 − 1 k n n

Number of Balls in Bin i contd. Balls & Bins Anil Maheshwari Number of Balls = m ; Number of Bins = n . Basics Random Variable Problem III Geometric Distribution What is the probability that Bin i receives ≥ k balls? Coupon Collector Problem � � 1 Balls & Bins � k � m Collisions Size of Bins ≤ k n � en � k ), � n � If n = m and using Stirling’s approximation ( ≤ k k � k ≤ � � 1 � e � k � n we have k n k

Expected Number of Balls in a Bin Balls & Bins Anil Maheshwari Number of Balls = m ; Number of Bins = n . Basics Random Variable Problem IV Geometric Distribution Show that the Expected # of Balls in a Bin is m n Coupon Collector Problem Balls & Bins Collisions Size of Bins

Expected Number of Empty Bins Balls & Bins Anil Maheshwari Number of Balls = m ; Number of Bins = n . Basics Random Variable Problem V Geometric Distribution What is Expected # of Empty Bins? Coupon Collector Problem Define a r.v. X i such that Balls & Bins Collisions Size of Bins � 1 if Bin i is empty X i = 0 Otherwise From Problem I, Pr ( X i = 1) ≤ e − m n and E [ X i ] ≤ e − m n n E [ X i ] ≤ ne − m Thus, E [ # of Empty Bins ] = � n i =1 When n = m , E [ # of Empty Bins ] ≤ n e

Max # Balls in Bins Balls & Bins Anil Maheshwari Number of Balls = Number of Bins = n . Basics Random Variable Max # of Balls in Bins Geometric Distribution With probability ≥ 1 − 1 n all bins receive fewer than 3 ln n ln ln n Coupon Collector balls. Problem Balls & Bins Collisions Size of Bins

References Balls & Bins Anil Maheshwari Basics Probability and Computing by Mitzenmacher and 1 Random Variable Upfal, Cambridge Univ. Press 2005. Geometric Introduction to Probability by Blitzstein and Hwang, Distribution 2 CRC Press 2015. Coupon Collector Problem Courses Notes of COMP 2804 by Michiel Smid. 3 Balls & Bins Collisions My Notes on Algorithm Design. Size of Bins 4 Introduction to Probability by Blitzstein and Hwang, 5 CRC Press 2015.