CSE 332: Sorting HW 4 due Wednesday no new HW out this week - PDF document

Announcements (2/3/14) • Reading for this lecture: Chapter 7. CSE 332: Sorting • HW 4 due Wednesday – no new HW out this week • Midterm next Monday Richard Anderson, Steve Seitz Winter 2014 2 Sorting Consistent Ordering • Input • The comparison function must provide a consistent – an array A of data records ordering on the set of possible keys – a key value in each data record – You can compare any two keys and get back an – a comparison function which imposes a consistent indication of a < b, a > b, or a = b (trichotomy) ordering on the keys – The comparison functions must be consistent • Output • If compare(a,b) says a<b, then compare(b,a) must say b>a – “sorted” array A such that • If compare(a,b) says a=b, then compare(b,a) must say b=a • For any i and j, if i < j then A[i]  A[j] • If compare(a,b) says a=b, then equals(a,b) and equals(b,a) must say a=b 3 4 Why Sort? Space • How much space does the sorting algorithm require? • Provides fast search: – In-place: no more than the array or at most O(1) addition space • Find k th largest element in: – out-of-place: use separate data structures, copy back – External memory sorting – data so large that does not fit in memory 5 6 1

Stability Time A sorting algorithm is stable if: How fast is the algorithm? – Items in the input with the same value end up in the – requirement: for any i<j, A[i] < A[j] same order as when they began. – This means that you need to at least check on each element at the very minimum Input Unstable sort Stable Sort • Complexity is at least: Adams 1 Adams 1 Adams 1 – And you could end up checking each element against Black 2 Smith 1 Smith 1 every other element Brown 4 Washington 2 Black 2 • Complexity could be as bad as: Jackson 2 Jackson 2 Jackson 2 Jones 4 Black 2 Washington 2 Smith 1 White 3 White 3 The big question: How close to O ( n ) can you get? Thompson 4 Wilson 3 Wilson 3 Washington 2 Thompson 4 Brown 4 White 3 Brown 4 Jones 4 Wilson 3 Jones 4 Thompson 4 7 8 [Sedgewick] Demo (with sound!) Sorting: The Big Picture • http://www.youtube.com/watch?v=kPRA0W1kECg Simple Fancier Comparison Specialized Handling algorithms: algorithms: lower bound: algorithms: huge data  ( n log n ) O( n 2 ) O( n log n ) O( n ) sets Insertion sort Heap sort Bucket sort External Selection sort Merge sort Radix sort sorting … Quick sort (avg) … 9 10 Selection Sort: idea Try it out: Selection Sort • 31, 16, 54, 4, 2, 17, 6 1. Find the smallest element, put it 1 st Find the next smallest element, put it 2 nd 2. 3. Find the next smallest, put it 3 rd And so on … 4. 11 12 2

Bubble Sort Selection Sort: Code • Take a pass through the array void SelectionSort (Array a[0..n-1]) { – If neighboring elements are out of order, swap them. for (i=0; i<n; ++i) { j = Find index of • Repeat until no swaps needed smallest entry in a[i..n-1] Swap(a[i],a[j]) } } • Wost & avg case: O(n 2 ) – pretty much no reason to ever use this algorithm Runtime: worst case : best case : average case : 13 14 Insertion Sort How to do the insertion? 1. Sort first 2 elements. Suppose my sequence is: Insert 3 rd element in order. 2. • (First 3 elements are now sorted.) 16, 31, 54, 78, 32, 17, 6 Insert 4 th element in order 3. • (First 4 elements are now sorted.) And I’ve already sorted up to 78. How to insert 32? And so on… 4. 15 16 Try it out: Insertion sort Insertion Sort: Code void InsertionSort (Array a[0..n-1]) { • 31, 16, 54, 4, 2, 17, 6 for (i=1; i<n; i++) { for (j=i; j>0; j--) { if (a[j] < a[j-1]) Swap(a[j],a[j-1]) else break } } Runtime: Note: can instead move the worst case : “hole” to minimize copying, best case : as with a binary heap. average case : 17 18 3

Insertion Sort vs. Selection Sort Sorting: The Big Picture • Same worst case, avg case complexity • Insertion better best-case Simple Fancier Comparison Specialized Handling – preferable when input is “almost sorted” algorithms: algorithms: lower bound: algorithms: huge data • one of the best sorting algs for almost sorted case (also for  ( n log n ) O( n 2 ) O( n log n ) O( n ) sets small arrays) Insertion sort Heap sort Bucket sort External Selection sort Merge sort Radix sort sorting … Quick sort (avg) … 19 20 In-place heap sort Heap Sort: Sort with a Binary Heap – Treat the initial array as a heap (via buildHeap ) – When you delete the i th element, put it at arr[n-i] • It’s not part of the heap anymore! 4 7 5 9 8 6 10 3 2 1 heap part sorted part 5 7 6 9 8 10 4 3 2 1 arr[n-i]= deleteMin() heap part sorted part Worst Case Runtime: 21 10/21/2013 22 “Divide and Conquer” AVL Sort • Very important strategy in computer science: – Divide problem into smaller parts – Independently solve the parts – Combine these solutions to get overall solution • Idea 1 : Divide array in half, recursively sort left and right halves, then merge two halves  known as Mergesort • Idea 2 : Partition array into small items and large items, then recursively sort the two sets  known as Quicksort Worst Case Runtime: 23 24 4

Mergesort Example Mergesort 8 2 9 4 5 3 1 6 Divide 8 2 9 4 5 3 1 6 8 2 9 4 5 3 1 6 Divide 8 2 9 4 5 3 1 6 • Divide it in two at the midpoint Divide • Sort each half (recursively) 1 element 8 2 9 4 5 3 1 6 • Merge two halves together Merge 2 8 4 9 3 5 1 6 Merge 2 4 8 9 1 3 5 6 Merge 1 2 3 4 5 6 8 9 25 26 Merging: Two Pointer Method Merging: Two Pointer Method • Perform merge using an auxiliary array • Perform merge using an auxiliary array 2 4 8 9 1 3 5 6 2 4 8 9 1 3 5 6 Auxiliary array Auxiliary array 1 27 28 Merging: Finishing Up Merging: Two Pointer Method Starting from here… • Perform merge using an auxiliary array i j target 2 4 8 9 1 3 5 6 Left finishes up copy i j target or Auxiliary array 1 2 3 4 5 first copy this… Right finishes up …then this i j 29 30 target 5

Merging Merge(A[], Temp[], left, mid, right) { Int i, j, k, l, target Merging: Two Pointer Method i = left j = mid + 1 target = left • Final result while (i < mid && j < right) { if (A[i] < A[j]) Temp[target] = A[i++] else Temp[target] = A[j++] 1 2 3 4 5 6 8 9 target++ } if (i > mid) //left completed// for (k = left to target-1) A[k] = Temp[k]; if (j > right) //right completed// Auxiliary array 1 2 3 4 5 6 k = mid l = right while (k > i) A[l--] = A[k--] Complexity? Stability? for (k = left to target-1) A[k] = Temp[k] 31 32 } Mergesort: Complexity Recursive Mergesort MainMergesort(A[1..n], n) { Array Temp[1..n] Mergesort[A, Temp, 1, n] } Mergesort(A[], Temp[], left, right) { if (left < right) { mid = (left + right)/2 Mergesort(A, Temp, left, mid) Mergesort(A, Temp, mid+1, right) Merge(A, Temp, left, mid, right) } } What is the recurrence relation? 33 34 Iterative Mergesort Iterative Mergesort Merge by 1 Merge by 1 Merge by 2 Merge by 2 Merge by 4 Merge by 4 Merge by 8 Merge by 8 Merge by 16 copy Iterative Mergesort reduces copying 35 36 Complexity? 6

Quicksort Properties of Mergesort Quicksort uses a divide and conquer strategy, but does not require the O(N) extra space that • In-place? MergeSort does. • Stable? Here’s the idea for sorting array S : • Sorted list complexity? 1. Pick an element v in S . This is the pivot value. • Nicely extends to handle linked lists. 2. Partition S -{ v } into two disjoint subsets, S 1 and S 2 • Multi-way merge is basis of big data sorting. such that: elements in S 1 are all  v • • Java uses Mergesort on Collections and on elements in S 2 are all  v • Arrays of Objects. 3. Return concatenation of QuickSort( S 1 ), v , QuickSort( S 2 ) Recursion ends when Quicksort( ) receives an array of length 0 or 1. 37 38 The steps of Quicksort Quicksort Example 4 6 3 8 1 9 2 5 S select pivot value 81 31 57 43 13 75 92 0 Divide 65 26 5 4 2 3 1 6 9 8 Divide S 1 S 2 2 partition S 0 8 1 3 4 9 31 75 6 43 65 13 Divide 81 92 26 57 1 element 3 4 QuickSort(S 1 ) and S 1 S 2 QuickSort(S 2 ) Conquer 3 4 0 13 26 31 43 57 65 75 81 92 Conquer 1 2 3 4 6 8 9 S 0 13 26 31 43 57 65 75 81 92 Presto! S is sorted Conquer 1 2 3 4 5 6 8 9 [Weiss] 39 40 Pivot Picking and Partitioning Picking the Pivot The tricky parts are: • Picking the pivot – Goal: pick a pivot value so that |S 1 | and |S 2 | are roughly equal in size. • Partitioning – Preferably in-place – Dealing with duplicates 41 42 7