Advanced Algorithms (II) Shanghai Jiao Tong University Chihao Zhang March 9th, 2020
Random Variables
Random Variables Recall that a probability space is a tuple ( Ω , ℱ , Pr)
Random Variables Recall that a probability space is a tuple ( Ω , ℱ , Pr) In this course, we mainly focus on countable Ω
Random Variables Recall that a probability space is a tuple ( Ω , ℱ , Pr) In this course, we mainly focus on countable Ω A random variable is a function X : Ω → ℝ X
Random Variables Recall that a probability space is a tuple ( Ω , ℱ , Pr) In this course, we mainly focus on countable Ω A random variable is a function X : Ω → ℝ X ∑ The expectation E [ X ] = a ⋅ Pr[ X = a ] a ∈Ω :Pr[ X = a ]>0
Linearity of Expectations
Linearity of Expectations For any random variables X 1 , …, X n n E [ X i ] = n n ∑ ∑ E [ X i ] i =1 i =1
Linearity of Expectations For any random variables X 1 , …, X n n E [ X i ] = n n ∑ ∑ E [ X i ] i =1 i =1 E [ X 1 + X 2 ] = ∑ ( a + b ) ⋅ Pr[ X 1 = a , X 2 = b ] a , b = ∑ a ⋅ Pr[ X 1 = a , X 2 = b ] + ∑ b ⋅ Pr[ X 1 = a , X 2 = b ] a , b a , b = ∑ a ⋅ Pr[ X 1 = a ] + ∑ b ⋅ Pr[ X 2 = b ] = E [ X 1 ] + E [ X 2 ] a b
Coupon Collector
Coupon Collector There are coupons to collect… n
Coupon Collector There are coupons to collect… n Each time one coupon is drawn independently uniformly at random
Coupon Collector There are coupons to collect… n Each time one coupon is drawn independently uniformly at random How many times one needs to draw to collect all coupons?
Let be the number of draws between -th distinct X i i coupon to the -th distinct coupon i + 1
Let be the number of draws between -th distinct X i i coupon to the -th distinct coupon i + 1 n − 1 ∑ Number of draws = X := X i i =0
Let be the number of draws between -th distinct X i i coupon to the -th distinct coupon i + 1 n − 1 ∑ Number of draws = X := X i i =0 For any , follows geometric distribution with i X i n − i probability n
Geometric Distribution
Geometric Distribution Let be a random variable following geometric X distribution with probability . p
Geometric Distribution Let be a random variable following geometric X distribution with probability . p Namely, we toss a coin who comes to HEAD with probability , is the number of tosses to see the first p X HEAD.
Geometric Distribution Let be a random variable following geometric X distribution with probability . p Namely, we toss a coin who comes to HEAD with probability , is the number of tosses to see the first p X HEAD. It is not hard to see that E [ X ] = 1 p
Back to Coupon Collector…
Back to Coupon Collector… E [ X ] = E [ X i ] = n − 1 n − 1 ∑ ∑ E [ X i ] i =0 i =0 n − 1 n − i = n n n n − 2 + … + n n ∑ = n + n − 1 + 1 i =0 = n ⋅ H ( n ) → n log n + γ n
Back to Coupon Collector… E [ X ] = E [ X i ] = n − 1 n − 1 ∑ ∑ E [ X i ] i =0 i =0 n − 1 n − i = n n n n − 2 + … + n n ∑ = n + n − 1 + 1 i =0 = n ⋅ H ( n ) → n log n + γ n The constant is called Euler constant γ = 0.577...
Linearity may fail when…
Linearity may fail when… • n = ∞
Linearity may fail when… • n = ∞ St. Petersburg paradox Each stage of the game a fair coin is tossed and a gambler guesses the result. He wins the amount he bet if his guess is correct and lose the money if he is wrong. He bets at the first $1 stage. If he loses, he doubles the money and bets again. The game ends when the gambler wins.
What is the expected money he wins?
What is the expected money he wins? - In stage , he wins with , E [ X i ] = 0 i X i ∞ ∑ - so E [ X i ] = 0 i =1
What is the expected money he wins? - In stage , he wins with , E [ X i ] = 0 i X i ∞ ∑ - so E [ X i ] = 0 i =1 - On the other hand, he eventually wins , $1 E [ X i ] = 1 ≠ ∞ ∞ ∑ ∑ - so ! E [ X i ] i =1 i =1
Linearity may fail when…
Linearity may fail when… • is random n = N
Linearity may fail when… • is random n = N Suppose we draw a number and toss dices N N E [ X N ] N ∑ , what is ? X 1 , …, X N i =1
Each is uniform in , one might expect {1,…,6} X i E [ X i ] = E [ N ] ⋅ E [ X 1 ] = 3.5 × 3.5 = 12.25 N ∑ i =1
Each is uniform in , one might expect {1,…,6} X i E [ X i ] = E [ N ] ⋅ E [ X 1 ] = 3.5 × 3.5 = 12.25 N ∑ i =1 If itself is drawn by tossing a dice and let N X 1 = X 2 = … = X N = N
Each is uniform in , one might expect {1,…,6} X i E [ X i ] = E [ N ] ⋅ E [ X 1 ] = 3.5 × 3.5 = 12.25 N ∑ i =1 If itself is drawn by tossing a dice and let N X 1 = X 2 = … = X N = N Then E [ X i ] = E [ N ⋅ N ] = 15.166.. N ∑ i =1
Wald’s Equation
Wald’s Equation If the variables satisfy
Wald’s Equation If the variables satisfy - and all are independent and finite; N X i - All are identically distributed X i
Wald’s Equation If the variables satisfy - and all are independent and finite; N X i - All are identically distributed X i N E [ X i ] = E [ N ] ⋅ E [ X 1 ] ∑ i =1
Wald’s Equation If the variables satisfy - and all are independent and finite; N X i - All are identically distributed X i N E [ X i ] = E [ N ] ⋅ E [ X 1 ] ∑ i =1 More generally if is a stopping time N
Application: Quick Select
Application: Quick Select Find the -th largest number in an unsorted array k A
Application: Quick Select Find the -th largest number in an unsorted array k A Find ( ) A , k Randomly choose a pivot x ∈ A 1. Partition into such that A − { x } A 1 , A 2 , ∀ y ∈ A 1 , y < x ∀ z ∈ A 2 , z > x 2. If , return | A 1 | = k − 1 x 3. If , return Find ( ) | A 1 | ≥ k A 1 , k 4. return Find ( ) A 2 , k − | A 1 | − 1
The partition step takes time O ( | A | )
The partition step takes time O ( | A | ) What is the total time cost in expectation?
The partition step takes time O ( | A | ) What is the total time cost in expectation? - size of at -th round X i A i E [ X i +1 ∣ X i ] ≤ 3 and X 1 = n 4 X i ∞ ∑ The time cost is X i i =1
E [ X i +1 ∣ X i ] ≤ 3 4 X i 4 E [ X i ] ≤ ( i 4 ) ⟹ E [ X i +1 ] = E [ E [ X i +1 ∣ X i ]] ≤ 3 3 n
E [ X i +1 ∣ X i ] ≤ 3 4 X i 4 E [ X i ] ≤ ( i 4 ) ⟹ E [ X i +1 ] = E [ E [ X i +1 ∣ X i ]] ≤ 3 3 n E [ X i ] = E [ X i ] ∞ n ∑ ∑ i =1 i =1 i − 1 n n i =1 ( 4 ) 3 ∑ ∑ = E [ X i ] ≤ n i =1 = 4 n .
KUW inequality
KUW inequality While analyzing random algorithms, a common recursion is for random T ( n ) = 1 + T ( n − X n ) X n
KUW inequality While analyzing random algorithms, a common recursion is for random T ( n ) = 1 + T ( n − X n ) X n Theorem . (Karp-Upfal-Wigderson Inequality) Assume for every , is an integer for some such that n 0 ≤ X n ≤ n − a a . If for all , where is positive and T ( a ) = 0 E [ X n ] ≥ μ ( n ) n > a μ ( n ) increasing , then E [ T ( n )] ≤ ∫ n 1 μ ( t ) d t a
Application: Expectation of Geometric Variables
Application: Expectation of Geometric Variables T (1) = 1 + T (1 − X 1 ), where E [ X 1 ] = p
Application: Expectation of Geometric Variables T (1) = 1 + T (1 − X 1 ), where E [ X 1 ] = p E [ T (1)] ≤ ∫ 1 p d t = 1 1 Choosing gives μ ( n ) = p p . 0
Application: Rounds of Quick Select
Application: Rounds of Quick Select In our Find ( ) algorithm, we have A , k T ( n ) = 1 + max{ T ( m ), T ( n − m − 1)},
Application: Rounds of Quick Select In our Find ( ) algorithm, we have A , k T ( n ) = 1 + max{ T ( m ), T ( n − m − 1)}, where is in uniformly at random. {1,2,…, n − 1} m
Application: Rounds of Quick Select In our Find ( ) algorithm, we have A , k T ( n ) = 1 + max{ T ( m ), T ( n − m − 1)}, where is in uniformly at random. {1,2,…, n − 1} m μ ( n ) = n We can choose (Why?) 4
Application: Rounds of Quick Select In our Find ( ) algorithm, we have A , k T ( n ) = 1 + max{ T ( m ), T ( n − m − 1)}, where is in uniformly at random. {1,2,…, n − 1} m μ ( n ) = n We can choose (Why?) 4 KUW implies E [ T ( n )] ≤ ∫ n 4 t d t = 4 log n 1
Application: Coupon Collector
Application: Coupon Collector where T ( m ) = 1 + T ( n − X m ) X m ∼ Ber( m / n )
Application: Coupon Collector where T ( m ) = 1 + T ( n − X m ) X m ∼ Ber( m / n ) So we can choose μ ( m ) = ⌈ m ⌉ n
Application: Coupon Collector where T ( m ) = 1 + T ( n − X m ) X m ∼ Ber( m / n ) So we can choose μ ( m ) = ⌈ m ⌉ n KUW implies E [ T ( n )] ≤ ∫ n n ⌈ t ⌉ d t = n ⋅ H n 0
Proof of KUW inequality
Recommend
More recommend
