Randomized algorithms Quick-sort Closest pair of points Inge Li - PowerPoint PPT Presentation

Randomized algorithms • Today • What are randomized algorithms? • Properties of randomized algorithms • Three examples: • Median/Select. Randomized algorithms • Quick-sort • Closest pair of points Inge Li Gørtz � 1 � 2 Randomized Algorithms • So far we dealt with deterministic algorithms: • Giving the same input to the algorithm repeatedly results in: • The same running time. • The same output. • Randomized algorithm: Randomized Algorithms • Can make random choices (using a random number generator) � 3 � 4

Random number generator Randomized algorithms • The algorithm can make random decisions using rand • A randomized algorithm has access to a random number generator rand(a, b) , where a, b are integers, a < b . • rand(a, b) returns a number x ∈ {a, (a + 1), . . . , (b − 1), b} . Running while (rand(0,1) = 0) do • Each of the b − a + 1 numbers is equally likely. print(“running”); end while • Calling rand(a, b) again results in a new, possibly di ff erent number. print(“stopped”); • rand(a, b) “has no memory”. The current number does not depend on the results of previous calls. Julefrokost • Statistically speaking: rand(a, b) generates numbers independently while (rand(1,4) = rand(1,4)) do according to uniform distribution. have another Christmas beer; end while go home; � 5 � 6 Properties of randomized algorithms Properties of randomized algorithms • A randomized algorithm might never stop. • A randomized algorithm might find the wrong solution. • Example: Find the position of 7 in a three-element array containing the numbers 4, 3 • Example: Find the position of the minimum element in a three-element array. and 7. MinOfThree(A[1…3]) i := rand(1,3) ; Find7(A[1…3]) j := rand(1,3) ; goon := true; return (min(A[i],A[j]); while (goon) do i := rand(1,3) ; if (A[i] = 7) then • If A = [5,6,4] and i = 2 and j = 1 then the algorithm will wrongly return 5. print(Bingo: 7 found at position i); goon := false; end if end while � 7 � 8

Properties of randomized algorithms • Both examples are not so bad as they look. • It is highly unlikely that Find7 runs a long time. • It is more likely that MinOfThree returns a correct result than a wrong one. Analysis of Randomized Algorithms � 9 � 10 Analysis of MinOfThree Analysis of MinOfThree MinOfThree(A[1…3]) MinOfThree(A[1…3]) i := rand(1,3) ; i := rand(1,3) ; j := rand(1,3) ; j := rand(1,3) ; return (min(A[i],A[j]); return (min(A[i],A[j]); • Running time: O(1). • Running MinOfThree once gives the correct answer with probaility 5/9 and the incorrect one with probability 4/9. • What is the chance that the answer of MinOfThree is correct? • Idea: Run MinOfThree many times and pick the smallest value returned. • 9 possible pairs all with probability 1/9: • The probability that MinOfThree fails every time in k runs is: • (1,1), (1,2), (1,3), (2,1), (2,2), (2,3), (3,1), (3,2), (3,3) • Assume wlog that A[1] is the minimum: • 5 of the 9 pairs contain the index 1 and give the correct minimum. • P[correct] = 5/9 > 1/2. • To be 99% sure to find the minimum, choose k such that � 11 � 12

Analysis of Find7 Analysis of Find7 Find7(A[1…3]) Find7(A[1…3]) goon := true; goon := true; while (goon) do while (goon) do i := rand(1,3) ; i := rand(1,3) ; if (A[i] = 7) then if (A[i] = 7) then print(Bingo: 7 found at position i); print(Bingo: 7 found at position i); goon := false; goon := false; end if end if end while end while • Running time is variable. What is the expected running time? • The while-loop stops after exactly two iterations if • Assume wlog that A[1] = 7. • rand(1, 3) ≠ 1 in the first iteration. • If rand(1, 3) = 1 then the while-loop is terminated. • rand(1, 3) = 1 in the second iteration. • P[rand(1,3) = 1] = 1/3 • This happens with probability 2/3 and 1/3, resp. • If rand(1, 3) ≠ 1 then the while-loop is not terminated. • The probability for both to happen is - by independence - • P[rand(1,3) ≠ 1] = 2/3 • The while-loop is terminated after one iteration with probability P[rand(1,3) = 1] = 1/3. � 13 � 14 Analysis of Find7 Expected running time • The while-loop stops after exactly three iterations if • We have • rand(1, 3) ≠ 1 in the first two iterations. • rand(1, 3) = 1 in the third iteration. • This happens with probability 2/3, 2/3 and 1/3, resp. • The expected running time of Find7 is • The probability for all three to happen is - by independence - • The while-loop stops after exactly k iterations if • Idea: Stop Las Vegas algorithm if it runs “much longer” than the • rand(1, 3) ≠ 1 in the first k-1 iterations. expected time and restart it. • rand(1, 3) = 1 in the k th iteration. • This happens with probability 2/3, 2/3 and 1/3, resp. ∞ x k ⋅ x k = (1 − x ) 2 for | x | < 1. ∑ • The probability for all three to happen is - by independence - k =0 � 15 � 16

Types of randomized algorithms Randomized algorithms • Analyse the expected number of times running is printed: • We have seen two kinds of algorithms: • Monte Carlo algorithms: stop after a fixed (polynomial) time and give the correct answer with probability greater 50%. 
 Running while (rand(0,1) = 0) do • Las Vegas algorithms: have variable running time but always give the print(“running”); end while correct answer. print(“stopped”); • Analyse the expected number of beers you get if you follow the algorithm: Julefrokost while (rand(1,4) = rand(1,4)) do have another Christmas beer; end while go home; � 17 � 18 Summations ∞ x k ⋅ x k = ∑ (1 − x ) 2 for | x | < 1. k =0 Median/Select ∞ 1 x k = ∑ (1 − x ) for | x | < 1. k =0 ∞ 1 k ⋅ x k − 1 = ∑ (1 − x ) 2 for | x | < 1. k =0 � 20

Select Select • Given n numbers S = {a 1 , a 2 , …, a n }. Select(S, k) { • Median: number that is in the middle position if in sorted order. Choose a pivot s ∈ S uniformly at random. • Select(S,k): Return the kth smallest number in S. For each element e in S if e < s put e in S’ • Min(S) = Select(S,1), Max(S)= Select(S,n), Median = Select(S,n/2). if e > s put e in S’’ if |S’| = k-1 then return s • Assume the numbers are distinct. if |S’| ≥ k then call Select(S’, k) if |S’| < k then call Select(S’’, k - |S’| - 1) } Select(S, k) { Choose a pivot s ∈ S uniformly at random. • Worst case running time: T ( n ) = cn + c ( n − 1) + c ( n − 2) + · · · = Θ ( n 2 ) . For each element e in S • If there is at least an fraction of elements both larger and smaller than s: if e < s put e in S’ ε if e > s put e in S’’ cn + (1 − ε ) cn + (1 − ε ) 2 cn + · · · T ( n ) = if |S’| = k-1 then return s 1 + (1 − ε ) + (1 − ε ) 2 + · · · � � = cn if |S’| ≥ k then call Select(S’, k) cn/ ε . ≤ if |S’| < k then call Select(S’’, k - |S’| - 1) • Limit number of bad pivots. } • Intuition: A fairly large fraction of elements are “well-centered” => random pivot likely to be good. � 21 � 22 Select • Phase j: Size of set at most and at least . n (3 / 4) j +1 n (3 / 4) j • Element is central if at least a quarter of the elements in the current call are smaller and at least a quarter are larger. • At least half the elements are central. • Pivot central => size of set shrinks with by at least a factor 3/4 => current phase ends. Quicksort • Pr[s is central] = 1/2. • Expected number of iterations before a central pivot is found = 2 => expected number of iterations in phase j at most 2. • X: random variable equal to number of steps taken by algorithm. • X j : expected number of steps in phase j. • X = X 1 + X 2 + .… • Number of steps in one iteration in phase j is at most . cn (3 / 4) j • E[X j ] = . 2 cn (3 / 4) j • Expected running time: ◆ j ◆ j ✓ 3 ✓ 3 X X X E [ X ] = E [ X j ] ≤ 2 cn = 2 cn ≤ 8 cn. 4 4 j j j � 23 � 24