Sorting Algorithms Mark Redekopp David Kempe Sandra Batista 2 - PowerPoint PPT Presentation

1 CSCI 104 Sorting Algorithms Mark Redekopp David Kempe Sandra Batista

2 Algorithm Efficiency SORTING

3 Sorting • If we have an unordered list, sequential search becomes our only choice List 7 3 8 6 5 1 index 0 1 2 3 4 5 • If we will perform a lot of searches it may Original be beneficial to sort the list, then use binary search List 1 3 5 6 7 8 index 0 1 2 3 4 5 • Many sorting algorithms of differing Sorted complexity (i.e. faster or slower) • Sorting provides a "classical" study of algorithm analysis because there are many implementations with different pros and cons

4 Applications of Sorting • Find the set_intersection of the 2 A 7 3 8 6 5 1 0 1 2 3 4 5 lists to the right B 9 3 4 2 7 8 11 – How long does it take? 0 1 2 3 4 5 6 Unsorted • Try again now that the lists are A 1 3 5 6 7 8 sorted 0 1 2 3 4 5 – How long does it take? 2 3 4 7 8 9 11 B 0 1 2 3 4 5 6 Sorted

5 Sorting Stability • A sort is stable if the order of equal items in the original list is maintained List 7,a 3,b 5,e 8,c 5,d in the sorted list index 1 0 2 3 4 Original – Good for searching with multiple criteria – Example: Spreadsheet search List 3,b 5,e 5,d 7,a 8,c index 1 0 2 3 4 • List of students in alphabetical order first Stable Sorting • Then sort based on test score • I'd want student's with the same test score to List 3,b 5,d 5,e 7,a 8,c index 1 0 2 3 4 appear in alphabetical order still Unstable Sorting • As we introduce you to certain sort algorithms consider if they are stable or not

6 Bubble Sorting • Main Idea: Keep comparing neighbors, moving larger item up and smaller item List 7 3 8 6 5 1 down until largest item is at the top. Original Repeat on list of size n-1 List 3 7 6 5 1 8 • Have one loop to count each pass, (a.k.a. i) After Pass 1 to identify which index we need to stop at List 3 6 5 1 7 8 • Have an inner loop start at the lowest After Pass 2 index and count up to the stopping List 3 5 1 6 7 8 location comparing neighboring elements After Pass 3 and advancing the larger of the neighbors List 3 1 5 6 7 8 After Pass 4 List 1 3 5 6 7 8 After Pass 5

7 Bubble Sort Algorithm void bsort(vector<int>& mylist) { int i ; for(i=mylist.size()-1; i > 0; i--){ for(j=0; j < i; j++){ if(mylist[j] > mylist[j+1]) { swap(mylist[j], mylist[j+1]); } } } } Pass 1 Pass 2 Pass n-2 7 3 8 6 5 1 3 7 6 5 1 8 3 1 5 6 7 8 j j j i i i 3 7 8 6 5 1 swap 3 7 6 5 1 8 no swap 1 3 5 6 7 8 swap j j i i … 3 7 8 6 5 1 no swap 3 6 7 5 1 8 swap j j i i 3 7 6 8 5 1 swap 3 6 5 7 1 8 swap j j i i 3 7 6 5 8 1 swap 3 6 5 1 7 8 swap j i 3 7 6 5 1 8 swap

8 Bubble Sort Value List Index Courtesy of wikipedia.org

9 Bubble Sort Analysis • Best Case Complexity: – When already ______________ but still have to ______________ – O(____) • Worst Case Complexity: – When ________________________ – O(____) void bsort(vector<int>& mylist) { int i ; for(i=mylist.size()-1; i > 0; i--){ for(j=0; j < i; j++){ if(mylist[j] > mylist[j+1]) { swap(mylist[j], mylist[j+1]); } } } }

10 Bubble Sort Analysis • Best Case Complexity: – When already sorted (no swaps) but still have to do all compares – O(n 2 ) • Worst Case Complexity: – When sorted in descending order – O(n 2 ) void bsort(vector<int>& mylist) { int i ; for(i=mylist.size()-1; i > 0; i--){ for(j=0; j < i; j++){ if(mylist[j] > mylist[j+1]) { swap(mylist[j], mylist[j+1]); } } } }

11 Loop Invariants • Loop invariant is a void bsort(vector<int>& mylist) { statement about what is int i ; for(i=mylist.size()-1; i > 0; i--){ true either before an for(j=0; j < i; j++){ if(mylist[j] > mylist[j+1]) { iteration begins or after swap(mylist[j], mylist[j+1]); } } } one ends } • Consider bubble sort and Pass 1 Pass 2 7 3 8 6 5 1 3 7 6 5 1 8 look at the data after j j i i each iteration (pass) 3 7 8 6 5 1 swap 3 7 6 5 1 8 no swap j j i i – What can we say about 3 7 8 6 5 1 no swap 3 6 7 5 1 8 swap j j i i the patterns of data after 3 7 6 8 5 1 swap 3 6 5 7 1 8 swap the k-th iteration? j j i i 3 7 6 5 8 1 swap 3 6 5 1 7 8 swap j i 3 7 6 5 1 8 swap

12 Loop Invariants • What is true after the k- void bsort(vector<int>& mylist) { int i ; th iteration? for(i=mylist.size()-1; i > 0; i--){ for(j=0; j < i; j++){ • All data at indices n-k and if(mylist[j] > mylist[j+1]) { swap(mylist[j], mylist[j+1]); above ___________ } } } } – ∀𝑗, 𝑗 ≥ 𝑜 − 𝑙: Pass 1 Pass 2 • All data at indices below 7 3 8 6 5 1 3 7 6 5 1 8 j j i i n-k are ______________ 3 7 8 6 5 1 swap 3 7 6 5 1 8 no swap ___________________ j j i i 3 7 8 6 5 1 no swap 3 6 7 5 1 8 swap – ∀𝑗, 𝑗 < 𝑜 − 𝑙: j j i i 3 7 6 8 5 1 swap 3 6 5 7 1 8 swap j j i i 3 7 6 5 8 1 swap 3 6 5 1 7 8 swap j i 3 7 6 5 1 8 swap

13 Loop Invariants • What is true after the k- void bsort(vector<int>& mylist) { int i ; th iteration? for(i=mylist.size()-1; i > 0; i--){ for(j=0; j < i; j++){ • All data at indices n-k and if(mylist[j] > mylist[j+1]) { swap(mylist[j], mylist[j+1]); above are sorted } } } } – ∀𝑗, 𝑗 ≥ 𝑜 − 𝑙: Pass 1 Pass 2 𝑏 𝑗 ≤ 𝑏 𝑗 + 1 7 3 8 6 5 1 3 7 6 5 1 8 • All data at indices below j j i i 3 7 8 6 5 1 swap 3 7 6 5 1 8 no swap n-k are less than the j j i i 3 7 8 6 5 1 no swap 3 6 7 5 1 8 swap value at n-k j j i i – ∀𝑗, 𝑗 < 𝑜 − 𝑙: 3 7 6 8 5 1 swap 3 6 5 7 1 8 swap j j i i 𝑏 𝑗 ≤ 𝑏 𝑜 − 𝑙 3 7 6 5 8 1 swap 3 6 5 1 7 8 swap j i 3 7 6 5 1 8 swap

14 Selection Sort • Selection sort does away with the many swaps and just records where the min or max value is and performs one swap at the end • The list/array can again be thought of in two parts – Sorted – Unsorted • The problem starts with the whole array unsorted and slowly the sorted portion grows • We could find the max and put it at the end of the list or we could find the min and put it at the start of the list – Just for variation let's choose the min approach

15 Selection Sort Algorithm void ssort(vector<int>& mylist) { Note: One can choose to find the for(i=0; i < mylist.size()-1; i++){ min and place at the start of the int min = i; list or max and place at the end. for(j=i+1; j < mylist.size; j++){ if(mylist[j] < mylist[min]) { min = j; } } swap(mylist[i], mylist[min]); } Pass 1 Pass 2 Pass n-2 min=0 min=1 min=4 7 3 8 6 5 1 min=1 1 3 8 6 5 7 min=1 1 3 5 6 7 8 min=4 j i j i j i 7 3 8 6 5 1 min=1 1 3 8 6 5 7 min=1 j i j i 7 3 8 6 5 1 min=1 1 3 8 6 5 7 min=1 j i j i 7 3 8 6 5 1 min=1 1 3 8 6 5 7 min=1 j i j i 7 3 8 6 5 1 min=5 1 3 8 6 5 7 swap i j 1 3 8 6 5 7 swap

16 Selection Sort Value Courtesy of wikipedia.org List Index

17 Selection Sort Analysis • Best Case Complexity: – __________________ – O(___) • Worst Case Complexity: – ____________________ – O(___) void ssort(vector<int>& mylist) { for(i=0; i < mylist.size()-1; i++){ int min = i; for(j=i+1; j < mylist.size; j++){ if(mylist[j] < mylist[min]) { min = j; } } swap(mylist[i], mylist[min]); }

18 Selection Sort Analysis Note: a+b+c = c+b+a (the order you sum doesn't matter so we can perform reordering or • Best Case Complexity: substitutions for our summations) 𝑜−2 𝑜−1 𝑜−2 – Sorted already 𝑈 𝑜 = ෍ ෍ 𝜄(1) = ෍ 𝜄(𝑜 − 𝑗 − 1) 𝑗=0 𝑘=𝑗+1 𝑗=0 – O(n 2 ) • Worst Case When i=0, we sum Θ (n-1). When i=n-2, we sum Θ (1). Thus, Complexity: 𝑜−1 𝜄(𝑙) = 𝜄(𝑜 2 ) – When sorted in = ෍ 𝑙=1 descending order void ssort(vector<int>& mylist) – O(n 2 ) { for(i=0; i < mylist.size()-1; i++){ int min = i; for(j=i+1; j < mylist.size; j++){ if(mylist[j] < mylist[min]) { min = j; } } swap(mylist[i], mylist[min]); }

19 Loop Invariant • What is true after the void ssort(vector<int>& mylist) { k-th iteration? for(i=0; i < mylist.size()-1; i++){ int min = i; for(j=i+1; j < mylist.size; j++){ • All data at indices less if(mylist[j] < mylist[min]) { min = j; than k are } } swap(mylist[i], mylist[min]); } _____________ Pass 1 Pass 2 min=0 min=1 – ∀𝑗, 𝑗 < 𝑙: 7 3 8 6 5 1 min=1 1 3 8 6 5 7 min=1 j i j i • All data at indices k 7 3 8 6 5 1 min=1 1 3 8 6 5 7 min=1 j i j i and above are 7 3 8 6 5 1 min=1 1 3 8 6 5 7 min=1 j i j i ____________ 7 3 8 6 5 1 min=1 1 3 8 6 5 7 min=1 j i j i __________________ 7 3 8 6 5 1 min=5 1 3 8 6 5 7 swap i j – ∀𝑗, 𝑗 ≥ 𝑙: 1 3 8 6 5 7 swap