Randomness in Computing L ECTURE 10 Last time Chernoff Bounds - PowerPoint PPT Presentation

Randomness in Computing L ECTURE 10 Last time • Chernoff Bounds Today • Hoeffding Bounds • Applications of Chernoff- Hoeffding Bounds • Estimating a Parameter • Set Balancing 2/25/2020 Sofya Raskhodnikova;Randomness in Computing

Sums of independent RVs Chernoff Bound (Upper Tail). Let X 1 , … , 𝑌 𝑜 be independent Bernoulli RVs . Let 𝑌 = 𝑌 1 + ⋯ + 𝑌 𝑜 and 𝜈 = 𝔽 𝑌 . Then • (stronger) for any 𝜀 > 0, 𝜈 𝑓 𝜀 Pr 𝑌 ≥ 1 + 𝜀 𝜈 ≤ . 1 + 𝜀 1+𝜀 • (easier to use) for any 𝜀 ∈ (0,1], Pr 𝑌 ≥ 1 + 𝜀 𝜈 ≤ 𝑓 −𝜈𝜀 2 /3 . Sofya Raskhodnikova; Randomness in Computing 2/25/2020

Sums of independent RVs Chernoff Bound (Lower Tail). Let X 1 , … , 𝑌 𝑜 be independent Bernoulli RVs. Let 𝑌 = 𝑌 1 + ⋯ + 𝑌 𝑜 and 𝜈 = 𝔽 𝑌 . Then • (stronger) for any 𝜀 ∈ (0,1), 𝜈 𝑓 −𝜀 Pr 𝑌 ≤ 1 − 𝜀 𝜈 ≤ . 1 − 𝜀 1−𝜀 • (easier to use) for any 𝜀 ∈ (0,1), Pr 𝑌 ≤ 1 − 𝜀 𝜈 ≤ 𝑓 −𝜈𝜀 2 /2 . Sofya Raskhodnikova; Randomness in Computing 2/25/2020

Sums of independent RVs Chernoff Bound (Both Tails). Let X 1 , … , 𝑌 𝑜 be independent Bernoulli RVs . Let 𝑌 = 𝑌 1 + ⋯ + 𝑌 𝑜 and 𝜈 = 𝔽 𝑌 . Then • for any 𝜀 ∈ 0,1 , Pr |𝑌 − 𝜈| ≥ 𝜀𝜈 ≤ 2𝑓 −𝜈𝜀 2 /3 . Sofya Raskhodnikova; Randomness in Computing 2/25/2020

Exercise 2 • The Halting Problem Team wins each hockey game they play with probability 1/3. Assuming outcomes of the games are independent, derive an upper bound on the probability that they have a winning season in 𝑜 games. • The Halting Problem Team hires a new coach, and critics revise their probability of winning each game to 3/4. Derive an upper bound on the probability they suffer a losing season. 2/25/2020

Exercise 3 We throw 𝑜 balls uniformly and independently into 𝑜 bins. Let 𝑍 1 be the number of balls that fell into bin 1. 1 Determine 𝑛 such that Pr 𝑍 1 > 𝑛 ≤ 𝑜 2 . Sofya Raskhodnikova; Randomness in Computing 2/25/2020

Sums of independent RVs Hoeffding Bound. Let X 1 , … , 𝑌 𝑜 be independent RVs with 𝔽 𝑌 𝑗 = 𝜈 0 and Pr 𝑏 ≤ 𝑌 𝑗 ≤ 𝑐 = 1. Let 𝑌 = 𝑌 1 + ⋯ + 𝑌 𝑜 . Then • (upper tail) Pr 𝑌 ≥ 𝜈 0 𝑜 + 𝜗𝑜 ≤ 𝑓 −2𝑜𝜗 2 / 𝑐−𝑏 2 . • (lower tail) Pr 𝑌 ≤ 𝜈 0 𝑜 − 𝜗𝑜 ≤ 𝑓 −2𝑜𝜗 2 / 𝑐−𝑏 2 Sofya Raskhodnikova; Randomness in Computing 2/25/2020

Application: Estimating a parameter • Unknown: probability 𝑞 that a feature occurs in the population. • Obtain an estimate by taking 𝑜 samples • 𝑌 ∼ Bin(𝑜, 𝑞) • Suppose 𝑌 = 𝑞𝑜 . • A 1 − 𝛿 confidence interval for parameter 𝑞 is [ 𝑞 − 𝜀, 𝑞 + 𝜀 ] such that Pr[𝑞 ∈ [ 𝑞 − 𝜀, 𝑞 + 𝜀 ] ] ≥ 1 − 𝛿. • Find a tradeoff between 𝛿, 𝜀 and 𝑜 . Sofya Raskhodnikova; Randomness in Computing 2/25/2020

Application: Estimating a parameter • A 1 − 𝛿 confidence interval for parameter 𝑞 is [ 𝑞 − 𝜀, 𝑞 + 𝜀 ] such that Pr[𝑞 ∈ [ 𝑞 − 𝜀, 𝑞 + 𝜀 ] ] ≥ 1 − 𝛿. • Find a tradeoff between 𝛿, 𝜀 and 𝑜 . Solution: 𝔽 𝑌 = 𝑜𝑞 • Suppose 𝑞 ∉ [ 𝑞 − 𝜀, 𝑞 + 𝜀 ] • Case 1: 𝑞 < 𝑞 − 𝜀 . Then 𝑞 > 𝑞 + 𝜀 • Case 2: 𝑞 > 𝑞 + 𝜀 . Then 𝑞 < 𝑞 − 𝜀 • 𝛿 = 2 ⋅ 𝑓 −2𝜀 2 𝑜 Sofya Raskhodnikova; Randomness in Computing 2/25/2020

Application: Set Balancing • Given: an 𝑜 × 𝑛 matrix 𝐵 with 0-1 entries • Definition: | (𝑦 1 , … , 𝑦 𝑜 ) | ∞ = max 𝑗∈[𝑜] |𝑦 𝑗 | • Find: 𝑐 ∈ −1,1 𝑛 minimizing 𝐵𝑐 ∞ 𝑏 11 𝑏 12 … 𝑏 1𝑛 𝑏 21 𝑏 22 … 𝑏 2𝑛 𝒐 features … 𝑏 𝑜1 𝑏 𝑜2 … 𝑏 𝑜𝑛 𝒏 subjects Partition subjects into two groups, so that each feature is balanced. Sofya Raskhodnikova; Randomness in Computing 2/25/2020

Application: Set Balancing • Algorithm: Choose each 𝑐 𝑗 u.a.r. from −1,1 . 𝐵𝑐 ∞ ≥ 4𝑛 ln 𝑜 ≤ 2/𝑜. • Theorem. Pr Sofya Raskhodnikova; Randomness in Computing 2/25/2020