Randomness in Computing L ECTURE 7 Last time Randomized quicksort - PowerPoint PPT Presentation

Randomness in Computing L ECTURE 7 Last time • Randomized quicksort • Markov’s inequality • Variance • Today • Variance, covariance • Chebyshev’s inequality • Variance of Binomial and Geometric RVs 2/11/2020 Sofya Raskhodnikova;Randomness in Computing

Recall: variance • The variance of a random variable X with expectation 𝔽[𝑌] = 𝜈 is Var 𝑌 = 𝔽 𝑌 − 𝜈 2 . • Equivalently, Var 𝑌 = 𝔽 𝑌 2 −𝜈 2 . • The standard deviation of X is 𝜏 𝑌 = Var 𝑌 . 2/11/2020

Compute expectation and variance • Fair die. Let X be the number showing on a roll of a die. Var 𝑌 = 𝐹 𝑌 2 −𝜈 2 2 = 1 + 4 + 9 + 16 + 25 + 36 − 7 = 91 6 − 49 4 = 35 6 2 12 • Uniform distribution. X is uniformly distributed over [𝑜] . 𝑜(𝑜+1)(2𝑜+1) – The sum of the first 𝑜 squares is 6 2 Var 𝑌 = 1 1 𝑗 2 − 𝑜 𝑜 𝑗 𝑗∈[𝑜] 𝑗∈ 𝑜 = 𝑜 2 − 1 − 𝑜 + 1 2 = (𝑜 + 1)(2𝑜 + 1) 6 4 12 2/11/2020

Compute expectation and variance Number of fixed points of a permutation. Let X be the number of students that get their hats back when 𝑜 students randomly switch hats, so that every permutation of hats is equally likely. Solution: 𝑌 𝑗 = the indicator R.V. for person 𝑗 getting their hat back. 𝑌 = 𝑌 1 + ⋯ + 𝑌 𝑜 1 By linearity of expectation and symmetry, 𝔽 𝑌 = 𝑜 ⋅ 𝔽 𝑌 1 = 𝑜 ⋅ 𝑜 = 1 𝔽 𝑌 2 = 𝔽 𝑌 1 + ⋯ + 𝑌 𝑜 2 2 + 𝑜 𝑜 − 1 ⋅ 𝔽 𝑌 1 ⋅ 𝑌 2 = 𝑜 ⋅ 𝔽 𝑌 1 = 𝑜 ⋅ 1 1 𝑜 + 𝑜 𝑜 − 1 ⋅ 𝑜 𝑜 − 1 = 2 Var 𝑌 = 𝔽 𝑌 2 − 𝔽 𝑌 2 = 2 − 1 = 1 2/11/2020

Random variables: covariance • The covariance of two random variables X and Y with expectations 𝔽[𝑌] = 𝜈 𝑌 and 𝔽[𝑍] = 𝜈 𝑍 is C ov(𝑌, 𝑍) = 𝔽 (𝑌 − 𝜈 𝑌 )(𝑍 − 𝜈 𝑍 ) . • Theorem. For any two random variables 𝑌 and 𝑍, Var 𝑌 + 𝑍 = Var 𝑌 + Var 𝑍 + 2 Cov 𝑌, 𝑍 . • Proof: Var 𝑌 + 𝑍 2 = 𝔽 𝑌 + 𝑍 − 𝔽 𝑌 + 𝑍 = 𝔽 ( 𝑌 − 𝜈 𝑌 ) + (𝑍 − 𝜈 𝑍 ) 2 = 𝔽 𝑌 − 𝜈 𝑌 2 + 𝑍 − 𝜈 𝑍 2 + 2(𝑌 − 𝜈 𝑌 )(𝑍 − 𝜈 𝑍 ) = 𝔽 𝑌 − 𝜈 𝑌 2 ] + 𝔽[ 𝑍 − 𝜈 𝑍 2 ] + 2𝔽[(𝑌 − 𝜈 𝑌 )(𝑍 − 𝜈 𝑍 ) = Var 𝑌 + Var 𝑍 + 2 Cov 𝑌, 𝑍 2/11/2020

Independent RVs • Random variables X and Y on the same probability space are independent if for all values 𝑏 and 𝑐 , the events 𝑌 = 𝑏 and 𝑍 = 𝑐 are independent. Equivalently, for all 𝑏, 𝑐 , Pr 𝑌 = 𝑏 ∧ 𝑍 = 𝑐 = Pr 𝑌 = 𝑏 ⋅ Pr[𝑍 = 𝑐]. • Theorem. For independent random variables X and Y, – 𝔽 𝑌𝑍 = 𝔽 𝑌 ⋅ 𝔽[𝑍] . – Var 𝑌 + 𝑍 = Var 𝑌 + Var [𝑍] . – Cov(X,Y)=0. 2/11/2020

Example: 𝑜 coin tosses • Let X be the number of HEADS in 𝑜 tosses of a biased coin with HEADS probability 𝑞. • We know: X has binomial distribution Bin( 𝑜, 𝑞). • What is the variance of X? Answer: 𝑜𝑞 1 − 𝑞 . 2/11/2020

Example: Geometric RV • Let X be the # of coin tosses until the first HEADS of a biased coin with HEADS probability 𝑞. • We know: X has geometric distribution Geom( 𝑞). • What is the variance of X? 1−𝑞 𝑞 2 . Answer: Sofya Raskhodnikova; Randomness in Computing 2/11/2020

Variance: additional facts • Theorem. For 𝑏, 𝑐 ∈ ℝ and a random variable X, Var 𝑏𝑌 + 𝑐 = 𝑏 2 Var 𝑌 . • Theorem. If 𝑌 1 , … , 𝑌 𝑜 are pairwise independent random variables, then Var 𝑌 1 + ⋯ + 𝑌 𝑜 = Var 𝑌 1 + ⋯ + Var[𝑌 𝑜 ]. Sofya Raskhodnikova; Randomness in Computing 2/11/2020

Chebyshev’s Inequality • Theorem. For a random variable X and 𝑏 > 0, Pr |𝑌 − 𝔽 𝑌 | ≥ 𝑏 ≤ Var[𝑌] . 𝑏 2 2 ≥ 𝑏 2 • Proof: Pr |𝑌 − 𝔽 𝑌 | ≥ 𝑏 = Pr 𝑌 − 𝔽 𝑌 𝒁 ≥ 0 ≤ 𝔽 𝑍 (by Markov) 𝑏 2 2 = 𝔽 𝑌 − 𝔽 𝑌 𝑏 2 = Var 𝑌 𝑏 2 2/11/2020

Chebyshev’s Inequality • Theorem. For a random variable X and 𝑏 > 0, Pr |𝑌 − 𝔽 𝑌 | ≥ 𝑏 ≤ Var[𝑌] . 𝑏 2 • Alternatively: Then, for all 𝑢 > 1, Pr |𝑌 − 𝔽 𝑌 | ≥ 𝑢 ⋅ 𝜏[𝑌] ≤ 1 𝑢 2 . • Example 1: 𝑌 ∼ Bin( 𝑜 ,1/2). 3𝑜 Bound Pr 𝑌 > using Markov and Chebyshev. 4 • Example 2: Coupon Collector Problem. Bound Pr 𝑌 > 2𝑜𝐼 𝑜 using Markov and Chebyshev. 2/11/2020