The Sorting Problem Inf 2B: Sorting, MergeSort and - PowerPoint PPT Presentation

The Sorting Problem Inf 2B: Sorting, MergeSort and Divide-and-Conquer Input: Array A of items with comparable keys . Lecture 7 of ADS thread Sort the items in A by increasing keys. Task: Kyriakos Kalorkoti School of Informatics The number of items to be sorted is usually denoted by n . University of Edinburgh What is important? Insertion Sort Algorithm insertionSort ( A ) Worst-case running-time: 1. for j 1 to A . length � 1 do What are the bounds on T Sort ( n ) for our Sorting Algorithm Sort. 2. a A [ j ] In-place or not?: 3. i j � 1 A sorting algorithm is in-place if it can be (simply) implemented 4. while i � 0 and A [ i ] . key > a . key do on the input array, with only O ( 1 ) extra space (extra variables). 5. A [ i + 1 ] A [ i ] Stable or not?: 6. i i � 1 7. A [ i + 1 ] a A sorting algorithm is stable if for every pair of indices with A [ i ] . key = A [ j ] . key and i < j , the entry A [ i ] comes before A [ j ] in the output array. I Asymptotic worst-case running time: Θ ( n 2 ) . I The worst-case (which gives Ω ( n 2 ) ) is h n , n � 1 , . . . , 1 i . I Both stable and in-place.

2nd sorting algorithm - Merge Sort Merge Sort - recursive structure Algorithm mergeSort ( A , i , j ) 12 6 4 9 13 8 5 1. if i < j then split in Divide mid b i + j 2. 2 c the middle 3. mergeSort ( A , i , mid ) 12 6 4 9 13 8 5 4. mergeSort ( A , mid + 1 , j ) sort & 5. merge ( A , i , mid , j ) recursively 4 6 9 12 5 8 13 Running Time: merge solutions together Conquer ( Θ ( 1 ) , for n  1 ; 4 5 6 8 9 12 13 T ( n ) = T ( d n / 2 e ) + T ( b n / 2 c ) + T merge ( n ) + Θ ( 1 ) , for n � 2 . How do we perform the merging? Merging the two subarrays Merge pseudocode Algorithm merge ( A , i , mid , j ) 13. while k  mid do A 8 11 12 4 9 21 4 B 1. new array B of length j � i + 1 14. B [ m ] A [ k ] 2. k i m k=i l=mid+1 3. ` mid + 1 15. k k + 1 A 8 11 12 4 9 21 4 8 B 4. m 0 16. m m + 1 k l 5. while k  mid and `  j do 17. while `  j do 4 8 9 B A 8 11 12 4 9 21 6. if A [ k ] . key < = A [ ` ] . key then 18. B [ m ] A [ ` ] 7. B [ m ] A [ k ] l k 19. ` ` + 1 8. k k + 1 A 8 11 12 4 9 21 9. else 20. m m + 1 k l 10. B [ m ] A [ ` ] 21. for m = 0 to j � i do New array B for output. 11. ` ` + 1 A [ m + i ] B [ m ] 22. Θ ( j � i + 1 ) time (linear time) always (best and worst cases). 12. m m + 1

Question on mergeSort Analysis of Mergesort I merge What is the status of mergeSort in regard to stability and T merge ( n ) = Θ ( n ) in-place sorting ? I mergeSort 1. Both stable and in-place. ( Θ ( 1 ) , for n  1 ; 2. Stable but not in-place. T ( n ) = T ( d n 2 e ) + T ( b n 2 c ) + T merge ( n ) + Θ ( 1 ) , for n � 2 . 3. Not stable, but is in-place. 4. Neither stable nor in-place. ( Θ ( 1 ) , for n  1 ; = T ( d n 2 e ) + T ( b n 2 c ) + Θ ( n ) , for n � 2 . Answer: not in-place but it is stable. If line 6 reads < instead of < = , we have sorting but NOT Solution to recurrence: Stability. T ( n ) = Θ ( n lg n ) . Solving the mergeSort recurrence Solving the mergeSort recurrence Put ` = lg n (hence 2 ` = n ). Write with constants c , d : T ( n ) = 2 T ( n / 2 ) + dn 2 T ( n / 2 2 ) + d ( n / 2 ) � � ( = + dn 2 c , for n  1 ; T ( n ) = 2 2 T ( n / 2 2 ) + 2 dn T ( d n 2 e ) + T ( b n = 2 c ) + dn , for n � 2 . 2 2 � 2 T ( n / 2 3 ) + d ( n / 2 2 ) � = + 2 dn 2 3 T ( n / 2 3 ) + 3 dn = Suppose n = 2 k for some k . Then no floors/ceilings. . . . 2 k T ( n / 2 k ) + kdn = ( c , for n = 1 ; 2 ` T ( n / 2 ` ) + ` dn = T ( n ) = 2 T ( n 2 ) + dn , for n � 2 . = nT ( 1 ) + ` dn = cn + dn lg ( n ) = Θ ( n lg ( n )) . Can extend to n not a power of 2 (see notes).

Merge Sort vs. Insertion Sort Divide-and-Conquer Algorithms I Merge Sort is much more efficient I Divide the input instance into several instances But: P 1 , P 2 , . . . P a of the same problem of smaller size - I If the array is “almost” sorted, Insertion Sort only needs “setting-up". “almost” linear time, while Merge Sort needs time I Recursively solve the problem on these smaller instances. Θ ( n lg ( n )) even in the best case. I Solve small enough instances directly. I For very small arrays, Insertion Sort is better because I Combine the solutions for the smaller instances Merge Sort has overhead from the recursive calls. P 1 , P 2 , . . . P a to a solution for the original instance. Do I Insertion Sort sorts in place , mergeSort does not (needs some “extra work" for this. Ω ( n ) additional memory cells). Analysing Divide-and-Conquer Algorithms The Master Theorem Theorem: Let n 0 2 N , k 2 N 0 and a , b 2 R with a > 0 and Analysis of divide-and-conquer algorithms yields recurrences b > 1, and let T : N ! R satisfy the following recurrence: like this: ( Θ ( 1 ) , if n < n 0 ; ( Θ ( 1 ) , if n < n 0 ; T ( n ) = T ( n ) = aT ( n / b ) + Θ ( n k ) , if n � n 0 . T ( n 1 ) + . . . + T ( n a ) + f ( n ) , if n � n 0 . f ( n ) is the time for “setting-up" and “extra work." Let e = log b ( a ) ; we call e the critical exponent . Then Usually recurrences can be simplified: Θ ( n e ) , 8 if k < e (I) ; < Θ ( n e lg ( n )) , T ( n ) = if k = e (II) ; ( Θ ( 1 ) , if n < n 0 ; Θ ( n k ) , if k > e (III) . T ( n ) = : aT ( n / b ) + Θ ( n k ) , if n � n 0 , I Theorem still true if we replace aT ( n / b ) by where n 0 , a , k 2 N , b 2 R with n 0 > 0, a > 0 and b > 1 are constants. a 1 T ( b n / b c ) + a 2 T ( d n / b e ) (Disregarding floors and ceilings.) for a 1 , a 2 � 0 with a 1 + a 2 = a .

Master Theorem in use . . . Master Theorem Example 1: We can “read off” the recurrence for mergeSort: Example 2: Let T be a function satisfying ( Θ ( 1 ) , n  1 ; T mergeSort ( n ) = T mergeSort ( d n 2 e ) + T mergeSort ( b n ( 2 c ) + Θ ( n ) , n � 2 . Θ ( 1 ) , if n  1 ; T ( n ) = 7 T ( n / 2 ) + Θ ( n 4 ) , if n � 2 . In Master Theorem terms, we have n 0 = 2 , k = 1 , a = 2 , b = 2 . e = log b ( a ) = log 2 ( 7 ) < 3 Thus e = log b ( a ) = log 2 ( 2 ) = 1 . So T ( n ) = Θ ( n 4 ) by case (III) . Hence T mergeSort ( n ) = Θ ( n lg ( n )) by case (II). Further Reading I If you have [GT], the “Sorting Sets and Selection" chapter has a section on mergeSort ( . ) I If you have [CLRS], there is an entire chapter on recurrences.