Sorting and recurrence analysis techniques Autumn 2018 Shrirang - PowerPoint PPT Presentation

CSE 373: Data Structures and Algorithms Sorting and recurrence analysis techniques Autumn 2018 Shrirang (Shri) Mare shri@cs.washington.edu Thanks to Kasey Champion, Ben Jones, Adam Blank, Michael Lee, Evan McCarty, Robbie Weber, Whitaker Brand, Zora Fung, Stuart Reges, Justin Hsia, Ruth Anderson, and many others for sample slides and materials ...

Review Sorting problem statement Given n comparable elements, rearrange them in an increasing order. Input - An array ! that contains n elements - Each element has a key " and an associated data - Keys support a comparison function (e.g., keys implement a Comparable interface) Expected output - An output array ! such that for any # and $ , - ![#] ≤ ![$] if # < $ (increasing order) - Array ! can also have elements in reverse order (decreasing order) CSE 373 AU 18 2

Review Desired properties in a sorting algorithm Stable - In the output, equal elements (i.e., elements with equal keys) appear in their original order In-place - Algorithm uses a constant additional space, !(1) extra space Adaptive - Performs better when input is almost sorted or nearly sorted - (Likely different big-O for best-case and worst-case) Fast. ! (% log %) No algorithm has all of these properties. So choice of algorithm depends on the situation. CSE 373 AU 18 3

Sorting algorithms – High-level view - ! " # - Insertion sort - Selection sort - Quick sort (worst) - !(" log ") - Merge sort - Heap sort - Quick sort (avg) - Ω(" log ") -- lower bound on comparison sorts - !(") – non-comparison sorts - Bucket sort (avg) CSE 373 AU 18 4

A framework to think about sorting algos Some questions to consider when analyzing a sorting algorithm. What is the state of the data during each step while sorting Loop/step invariant: _______________________________________________________________________ Runtime: Worst ______________ Average ______________ Best ______________ Input: Worst ______________________ Best ______________________ Stable ____________________ In-place ____________________ Adaptive ____________________ Yes/No Yes/No/Can-be Yes/No/Can-be Operations: Comparisons __________________________ Moves > or < or approx. equal Data structure ______________________________________________________ Which data structure is better suited for this algo CSE 373 AU 18 5

Visualization: https://visualgo.net/en/sorting Insertion sort dea: At step ! , insert the ! "# element in the correct position among the first ! elements. Ide Loop/step invariant: _______________________________________________________________________ Runtime: Worst ______________ Average ______________ Best ______________ Input: Worst ______________________ Best ______________________ Stable ____________________ In-place ____________________ Adaptive ____________________ Operations: Comparisons __________________________ Moves Data structure ______________________________________________________ CSE 373 AU 18 6

Visualization: https://visualgo.net/en/sorting Selection sort dea: At step ! , find the smallest element among the not-yet-sorted elements ( ! … # ) and Ide swap it with the element at ! . Loop/step invariant: _______________________________________________________________________ Runtime: Worst ______________ Average ______________ Best ______________ Input: Worst ______________________ Best ______________________ Stable ____________________ In-place ____________________ Adaptive ____________________ Operations: Comparisons __________________________ Moves Data structure ______________________________________________________ CSE 373 AU 18 7

Heap sort Ide dea: Loop/step invariant: _______________________________________________________________________ Runtime: Worst ______________ Average ______________ Best ______________ Input: Worst ______________________ Best ______________________ Stable ____________________ In-place ____________________ Adaptive ____________________ Operations: Comparisons __________________________ Moves Data structure ______________________________________________________ CSE 373 AU 18 8

In-place heap sort Ide dea: 1. Treat initial array as a heap 2. When you call removeMin() , that frees up a slot towards the end in the array. Put the extract min element there. 3. More specifically, when you remove the ! "# element, put it at $ % − ! 4. This gives you a reverse sorted array. But easy to fix in-place. CSE 373 AU 18 9

Design technique: Divide-and-conquer Very important technique in algorithm to attack problems Three steps: 1. Divide: Split the original problem into smaller parts 2. Conquer: Solve individual parts independently (think recursion) 3. Combine: Put together individual solved parts to produce an overall solution Merge sort and Quick sort are classic examples of sorting algorithms that use this technique CSE 373 AU 18 10

Visualization: https://visualgo.net/en/sorting Merge sort To sort a given array, Divide: Split the input array into two halves Conquer: Sort each half independently Combine: Merge the two sorted halves into one sorted whole (HW3 Problem 6!) CSE 373 AU 18 11

0 1 2 3 4 Merge sort 8 2 57 91 22 0 1 0 1 2 Split array in the middle 8 2 57 91 22 0 1 0 0 0 Sort the two halves 91 22 2 57 8 Merge them together 0 0 91 22 0 1 22 91 0 1 0 1 2 % & if " ≤ 1 2! " 2 8 22 57 91 ! " = $ 2 + % - " otherwise 0 1 2 3 4 2 8 22 57 91

Review: Unfolding (technique 1) % & if " ≤ 1 2! " ! " = $ 2 + % - " otherwise CSE 373 AU 18 13

Technique 2: Tree method Number of Work per Work per Level Nodes Node Level at level 0 1 2 n ! n n 2 2 base Last recursive level: n n n n 4 4 4 4 . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 CSE 373 AU 18 14

Technique 2: Tree method Number of Work per Work per Level Nodes Node Level at level 0 1 2 n ! n n 2 2 base Last recursive level: n n n n 4 4 4 4 . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 CSE 373 AU 18 15

Technique 2: Tree method Number of Work per Work per Level Nodes Node Level at level 0 1 2 n ! n n 2 2 base Last recursive level: n n n n 4 4 4 4 . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 CSE 373 AU 18 16

Technique 2: Tree method Number of Work per Work per Level Nodes Node Level at level 0 1 2 n ! n n 2 2 base Last recursive level: n n n n 4 4 4 4 . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 CSE 373 AU 18 17

Technique 2: Tree method Number of Work per Work per Level Nodes Node Level at level 0 1 2 n ! n n 2 2 base Last recursive level: n n n n 4 4 4 4 . . . . . . . . . . . . . . . . . . . . . . . . 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 CSE 373 AU 18 18

Technique 2: Tree method Number of Work per Work per Level Nodes Node Level at level ! ! 0 1 ! ! 1 2 2 ! ! 2 4 n 4 ! 2 % $ ! 2 % n n 2 2 2 &'( ) * ! base 1 Last recursive level: log ! − 1 n n n n 4 4 4 4 Combining it all together… . . . . . . . . &'( 6 * 78 . . . . . . . . . . . . . . . . 0 ! = ! + 3 ! = ! + ! log ! %45 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 CSE 373 AU 18 19

Number of Work per Work per Level (i) Tree Method Practice Nodes Node Level 4 *ℎ,( ( ≤ 1 0 1 ' ( = 3' ( 4 + !( # 01ℎ,2*34, 1 ' ( 4 + ' ( 4 + ' ( 4 + !( # !n # 2 3 base ' ( ! n # ' ( ' ( ! n # ! n # 4 4 4 4 4 4 n n # n # n # # n # n # n # n # n # ! ! ! ! ! ! ! ! ! 16 16 16 16 16 16 16 16 16 … … … … … … … … … … … … … … … … … … … … … … … … … … … 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 EXAMPLE PROVIDED BY CS 161 – JESSICA SU 20 HTTPS://WEB.STANFORD.EDU/CLASS/ARCHIVE/CS/CS161/CS161.1168/LECTURE3.PDF

Number of Work per Work per Level (i) Tree Method Practice Nodes Node Level 4 *ℎ,( ( ≤ 1 !( 2 !( 2 0 1 ' ( = 3' ( 4 + !( # 01ℎ,2*34, 3 ! ( # 16 !( # 1 3 4 ' ( 4 + ' ( 4 + ' ( 4 + !( # !n # 9 ( # 256 !( # 2 9 ! 16 ( # 8 3 ! 3 8 3 !( # 4 8 16 ' ( ! n # ' ( ' ( ! n # ! n # 3 9:; < = 4 > 3 9:; < = base 4 4 4 4 4 4 4 Last recursive level: log < ( − 1 Combining it all together… n n # n # n # # n # n # n # n # n # ! ! ! ! ! ! ! ! ! 16 16 16 16 16 16 16 16 16 9:; C = DE 8 3 ' ( = 4 ( 9:; < ? + !( # @ … … … … … … … … … … … … … … … … … … … … … … … … … … … 16 8AB 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 EXAMPLE PROVIDED BY CS 161 – JESSICA SU 21 HTTPS://WEB.STANFORD.EDU/CLASS/ARCHIVE/CS/CS161/CS161.1168/LECTURE3.PDF