Randomness in Computing L ECTURE 6 Last time Conditional - PowerPoint PPT Presentation

Randomness in Computing L ECTURE 6 Last time • Conditional expectation • Branching process • Geometric RVs • Coupon collector problem • Today • Randomized quicksort • Markov’s inequality • Variance 2/6/2020 Sofya Raskhodnikova;Randomness in Computing

Quicksort: divide and conquer • Find a pivot element • Divide: Find the correct position of the pivot by comparing it to all elements. pivot 𝒐 1 A: ≤ 𝑦 𝑦 ≥ 𝑦 • Conquer: Recursively sort the two parts, resulting from removing the pivot. ≤ 𝑦 𝑦 ≥ 𝑦 sort sort Sofya Raskhodnikova; based on notes by E. Demaine and C. Leiserson

Exercise How many comparisons does Quicksort perform on sorted array? Answer: 𝑜 − 1 + 𝑜 − 2 + ⋯ + 2 + 1 = 𝑜 𝑜 − 1 = Ω(𝑜 2 ) 2 How many comparisons does Quicksort perform if, in every iteration, the pivot splits the array into two halves? Answer: Let 𝐷 𝑜 be the number of comparisons performed on an array with 𝑜 elements. 𝐷 𝑜 = Θ(𝑜 log 𝑜) 2/6/2020

Randomized Quicksort BIG IDEA : Partition around a random element. • Analysis is similar when the input arrives in random order. • But randomness in the input is unreliable. • Rely instead on random number generator. 2/6/2020 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

Analysis of Randomized Quicksort Theorem. If Quicksort chooses each pivot uniformly and independently at random from all possibilities then, for any input, the expected number of comparisons is 2𝑜 ln 𝑜 + 𝑃(𝑜). Proof (with an assumption that all elements are distinct): • Let 𝑌 be the R.V. for the # of comparisons. • Let 𝑦 1 , 𝑦 2 , … , 𝑦 𝑜 be the input values. • Let 𝑧 1 , 𝑧 2 , … , 𝑧 𝑜 be the input values sorted in increasing order. • For 𝑗, 𝑘 ∈ 𝑜 , 𝑗 < 𝑘, let 𝑌 𝑗𝑘 be the indicator R.V. for the event that 𝑧 𝑗 and 𝑧 𝑘 are compared by the algorithm. 𝔽 𝑌 𝑗𝑘 𝑌 𝑗𝑘 and, by linearity of expextation, 𝔽[𝑌] = 𝑌 = 𝑗,𝑘 𝑜 :𝑗<𝑘 𝑗,𝑘 𝑜 :𝑗<𝑘 2/6/2020

Analysis of Randomized Quicksort Theorem. The expected number of comparisons is 2𝑜 ln 𝑜 + 𝑃(𝑜). Proof (continued): • Let 𝑧 1 , 𝑧 2 , … , 𝑧 𝑜 be the input values sorted in increasing order. • For 𝑗, 𝑘 ∈ 𝑜 , 𝑗 < 𝑘, let 𝑌 𝑗𝑘 be the indicator R.V. for the event that 𝑧 𝑗 and 𝑧 𝑘 are compared by the algorithm. • 𝔽 𝑌 𝑗𝑘 = Pr[X ij = 1] • 𝑧 𝑗 and 𝑧 𝑘 are compared iff either 𝑧 𝑗 or 𝑧 𝑘 is the first pivot chosen from 𝑍 𝑗𝑘 = {𝑧 𝑗 , … , 𝑧 𝑘 } • The first time a pivot is chose from 𝑍 𝑗𝑘 , it is equally likely to be any of 𝑘 − 𝑗 + 1 elements of 𝑍 𝑗𝑘 . 2/6/2020 Sofya Raskhodnikova; Randomness in Computing

Analysis of Randomized Quicksort Theorem. The expected number of comparisons is 2𝑜 ln 𝑜 + 𝑃(𝑜). Proof (continued): 2/6/2020 Sofya Raskhodnikova; Randomness in Computing

Markov’s Inequality Theorem. Let X be a RV taking only nonnegative values. Then, for all 𝑏 > 0, Pr 𝑌 ≥ 𝑏 ≤ 𝔽 [𝑌] . 𝑏 Proof: Let a > 0. 2/6/2020

Markov’s Inequality Theorem. Let X be a RV taking only nonnegative values. Then, for all 𝑏 > 0, Pr 𝑌 ≥ 𝑏 ≤ 𝔽 [𝑌] . 𝑏 Alternative proof: Let a > 0. 2/6/2020

Markov’s Inequality • Theorem. Let X be a RV taking only nonnegative values. Then, for all 𝑏 > 0, Pr 𝑌 ≥ 𝑏 ≤ 𝔽 [𝑌] . 𝑏 • Alternatively: Then, for all 𝑐 > 1, Pr 𝑌 ≥ 𝑐 ⋅ 𝔽[𝑌] ≤ 1 𝑐 . • Example: Show that Randomized Quicksort uses ≤ 10𝑜 ln 𝑜 + 𝑃(𝑜) comparisons with probability at least 4/5. 2/6/2020

Bernoulli random walk • Start at position 0. • At every step go up or down by 1 with probability 1/2 each. • Let X be the position after 𝑜 steps. What is the probability space? A. Uniform over 0,1 . B. Uniform over {-1,1}. C. 2 𝑜 . D. Position after 𝑜 steps. E. Uniform over 𝑡 1 , … , 𝑡 𝑜 𝑡 𝑗 ∈ −1,1 for 𝑗 = 1, … , 𝑜} . 2/6/2020

Bernoulli random walk • Start at position 0. • At every step go up or down by 1 with probability 1/2 each. • Let X be the position after 𝑜 steps. • What is E[X]? • How far from the origin should we expect X to be? A precise answer to this question is the expectation of |X|. However, it is easier to work with the expectation of 𝑌 2 . (It is not the same! But gives us an idea.) 2/6/2020

Random variables: variance • The variance of a random variable X with expectation 𝐹[𝑌] = 𝜈 is 𝑌 − 𝜈 2 . Var 𝑌 = 𝐹 • Equivalently, Var 𝑌 = 𝐹 𝑌 2 −𝜈 2 . • The standard deviation of X is 𝜏 𝑌 = Var 𝑌 . 2/6/2020

Variance as a measure of spread • X= −2 with probability 1/2 2 with probability 1/2 −10 with probability 0.001 0 with probability 0.998 • Y= 10 with probability 0.001 −5 with probability 1/3 0 with probability 1/3 • Z= 5 with probability 1/3 • Compute the variances and standard deviations of X,Y and Z. 2/6/2020 Example from Mathematics: a Discrete introduction by E. Scheinerman