Introduction CSCE423/823 CSCE423/823 Given an array A of n distinct - PDF document

Introduction CSCE423/823 CSCE423/823 Given an array A of n distinct numbers, the i th order statistic of A Computer Science & Engineering 423/823 is its i th smallest element Introduction Introduction Design and Analysis of Algorithms i = 1 ) minimum Finding Finding i = n ) maximum Minimum and Minimum and Maximum Maximum Lecture 01 — Medians and Order Statistics (Chapter 9) i = b ( n + 1) / 2 c ) (lower) median Selection of Selection of Arbitrary Arbitrary E.g. if A = [8 , 5 , 3 , 10 , 4 , 12 , 6] then min = 3, max = 12, median = Order Statistic Order Statistic 6, 3rd order stat = 5 Stephen Scott Problem: Given array A of n elements and a number i 2 { 1 , . . . , n } , (Adapted from Vinodchandran N. Variyam) find the i th order statistic of A There is an obvious solution to this problem. What is it? What is its time complexity? Can we do better? What if we only focus on i = 1 or i = n ? Spring 2012 1 / 24 2 / 24 Finding Minimum E ffi ciency of Minimum( A ) CSCE423/823 CSCE423/823 Introduction Introduction Loop is executed n � 1 times, each with one comparison small = A [1] Finding 1 Finding Minimum and Minimum and ) Total n � 1 comparisons Maximum for i = 2 to n do Maximum 2 Can we do better? Selection of if small > A [ i ] then Selection of 3 Arbitrary Arbitrary Order Statistic small = A [ i ] Order Statistic Lower Bound: Any algorithm finding minimum of n elements will 4 need at least n � 1 comparisons end 5 Proof of this comes from fact that no element of A can be considered return small 6 for elimination as the minimum until it’s been compared at least once Algorithm 1: Minimum( A, n ) 3 / 24 4 / 24 Correctness of Minimum( A ) Simultaneous Minimum and Maximum CSCE423/823 CSCE423/823 Introduction Introduction Finding Finding Observe that the algorithm always maintains the invariant that at Minimum and Minimum and Given array A with n elements, find both its minimum and maximum Maximum Maximum the end of each loop iteration, small holds the minimum of A [1 · · · i ] Selection of Selection of What is the obvious algorithm? What is its (non-asymptotic) time Arbitrary Easily shown by induction Arbitrary Order Statistic Order Statistic complexity? Correctness follows by observing that i == n before return Can we do better? statement 5 / 24 6 / 24

Simultaneous Minimum and Maximum Explanation of MinAndMax CSCE423/823 CSCE423/823 large = max( A [1] , A [2]) 1 Introduction Introduction small = min( A [1] , A [2]) 2 Finding Finding Idea: For each pair of values examined in the loop, compare them for i = 2 to b n/ 2 c do Minimum and 3 Minimum and Maximum Maximum large = max( large, max( A [2 i � 1] , A [2 i ])) directly 4 Selection of Selection of Arbitrary small = min( small, min( A [2 i � 1] , A [2 i ])) Arbitrary 5 For each such pair, compare the smaller one to small and the larger Order Statistic Order Statistic end one to large 6 if n is odd then 7 Example: A = [8 , 5 , 3 , 10 , 4 , 12 , 6] large = max( large, A [ n ]) 8 small = min( small, A [ n ]) 9 return ( large, small ) 10 Algorithm 2: MinAndMax( A, n ) 7 / 24 8 / 24 E ffi ciency of MinAndMax Selection of the i th Smallest Value CSCE423/823 CSCE423/823 Introduction How many comparisons does MinAndMax make? Introduction Finding Finding Initialization on Lines 1 and 2 requires only one comparison Now to the general problem: Given A and i , return the i th smallest Minimum and Minimum and Maximum Maximum value in A Each iteration through the loop requires one comparison between Selection of Selection of Arbitrary A [2 i � 1] and A [2 i ] and then one comparison to each of large and Arbitrary Obvious solution is sort and return i th element Order Statistic Order Statistic small , for a total of three Algorithm Time complexity is Θ ( n log n ) Overview Algorithm Lines 8 and 9 require one comparison each Pseudocode Can we do better? Example Total is at most 1 + 3( b n/ 2 c � 1) + 2  3 b n/ 2 c , which is better than Time Complexity Master Theorem 2 n � 3 for finding minimum and maximum separately 9 / 24 10 / 24 Selection of the i th Smallest Value (2) Procedure Select CSCE423/823 CSCE423/823 1 if p == r then Introduction Introduction 2 return A [ p ] Finding Finding New algorithm: Divide and conquer strategy 3 q = Partition( A, p, r ) // Like Partition in Quicksort Minimum and Minimum and Maximum Maximum Idea: Somehow discard a constant fraction of the current array after 4 k = q � p + 1 // Size of A [ p · · · q ] Selection of Selection of spending only linear time 5 if i == k then Arbitrary Arbitrary 6 return A [ q ] // Pivot value is the answer Order Statistic Order Statistic If we do that, we’ll get a better time complexity Algorithm Algorithm 7 else if i < k then Overview Overview More on this later 8 return Select ( A, p, q � 1 , i ) // Answer is in left subarray Algorithm Algorithm Pseudocode Pseudocode 9 else Example Which fraction do we discard? Example Time Complexity Time Complexity 10 return Select ( A, q + 1 , r, i � k ) // Answer is in right subarray Master Theorem Master Theorem Algorithm 3: Select( A, p, r, i ), which returns i th smallest element from A [ p · · · r ] 11 / 24 12 / 24

What is Select Doing? Partitioning the Array CSCE423/823 CSCE423/823 Like in Quicksort, Select first calls Partition, which chooses a pivot 1 x = ChoosePivotElement( A, p, r ) // Returns index of pivot Introduction Introduction element q , then reorders A to put all elements < A [ q ] to the left of 2 exchange A [ x ] with A [ r ] Finding Finding A [ q ] and all elements > A [ q ] to the right of A [ q ] 3 i = p � 1 Minimum and Minimum and Maximum Maximum 4 for j = p to r � 1 do E.g. if A = [1 , 7 , 5 , 4 , 2 , 8 , 6 , 3] and pivot element is 5, then result is Selection of Selection of 5 if A [ j ]  A [ r ] then A 0 = [1 , 4 , 2 , 3 , 5 , 7 , 8 , 6] Arbitrary Arbitrary 6 i = i + 1 Order Statistic Order Statistic exchange A [ i ] with A [ j ] 7 Algorithm If A [ q ] is the element we seek, then return it Algorithm Overview Overview 8 end Algorithm Algorithm Pseudocode If sought element is in left subarray, then recursively search it, and Pseudocode 9 exchange A [ i + 1] with A [ r ] Example Example Time Complexity ignore right subarray Time Complexity 10 return i + 1 Master Theorem Master Theorem If sought element is in right subarray, then recursively search it, and Algorithm 4: Partition( A, p, r ), which chooses a pivot element and ignore left subarray partitions A [ p · · · r ] around it 13 / 24 14 / 24 Partitioning the Array: Example (Fig 7.1) Choosing a Pivot Element CSCE423/823 CSCE423/823 Introduction Introduction Finding Finding Minimum and Minimum and Maximum Maximum Choice of pivot element is critical to low time complexity Selection of Selection of Why? Arbitrary Arbitrary Order Statistic Order Statistic Compare each element A [ j ] to x ( = 4 ) and swap with A [ i ] if A [ j ]  x What is the best choice of pivot element to partition A [ p · · · r ] ? Algorithm Algorithm Overview Overview Algorithm Algorithm Pseudocode Pseudocode Example Example Time Complexity Time Complexity Master Theorem Master Theorem 15 / 24 16 / 24 Choosing a Pivot Element (2) Median of Medians CSCE423/823 CSCE423/823 Introduction Introduction Given (sub)array A of n elements, partition A into m = b n/ 5 c Finding Finding Minimum and Minimum and groups of 5 elements each, and at most one other group with the Want to pivot on an element that it as close as possible to being the Maximum Maximum remaining n mod 5 elements median Selection of Selection of Arbitrary Arbitrary Make an array A 0 = [ x 1 , x 2 , . . . , x m +1 ] , where x i is median of group Order Statistic Of course, we don’t know what that is Order Statistic Algorithm Algorithm i , found by sorting (in constant time) group i Overview Overview Will do median of medians approach to select pivot element Algorithm Algorithm Pseudocode Pseudocode Call Select ( A 0 , 1 , m + 1 , b ( m + 1) / 2 c ) and use the returned element Example Example Time Complexity Time Complexity as the pivot Master Theorem Master Theorem 17 / 24 18 / 24