CS4102 Algorithms Fall 2018 Warm up Build a Max Heap from the following Elements: 4, 15, 22, 6, 18, 30, 14, 21 1
Heap • Heap Property: Each node must be larger than its children 30 1 22 21 3 2 4 14 19 15 6 7 4 5 6 30 21 22 19 15 4 14 6 8 0 1 2 3 4 5 6 7 8 2
Today’s Keywords • Sorting • Quicksort • Sorting Algorithm Characteristics • Insertion Sort • Bubble Sort • Heap Sort • Linear time Sorting • Counting Sort • Radix Sort 3
CLRS Readings • Chapter 6 • Chapter 8 4
Homeworks • Hw3 Due 11pm Monday Oct 1 – Divide and conquer – Written (use LaTeX!) • Hw4 released soon – Sorting – Written 5
Strategy: Decision Tree Θ(𝑜log 𝑜) • Conclusion: Worst Case Optimal run time of sorting is 𝑝(𝑜 log 𝑜) – There is no (comparison-based) sorting algorithm with run time One comparison Result of Possible comparison execution path < > >or<? >or<? >or<? < < > > log 𝑜! >or<? >or<? >or<? >or<? < < > < < > > Θ(𝑜 log 𝑜) > >or<? >or<? >or<? >or<? >or<? >or<? >or<? >or<? … … … … Permutation … … [5,2,4,1,3] [5,4,3,2,1] [1,2,3,4,5] [2,1,3,4,5] of sorted list 𝑜! Possible permutations 6
Sorting, so far 𝑃(𝑜 log 𝑜) • Sorting algorithms we have discussed: – Mergesort 𝑃(𝑜 log 𝑜) Optimal! – Quicksort Optimal! 𝑃(𝑜 2 ) • Other sorting algorithms 𝑃(𝑜 2 ) – Bubblesort 𝑃(𝑜 log 𝑜) – Insertionsort Optimal! – Heapsort 7
Speed Isn’t Everything • Important properties of sorting algorithms: • Run Time – Asymptotic Complexity – Constants • In Place (or In-Situ) – Done with only constant additional space • Adaptive – Faster if list is nearly sorted • Stable – Equal elements remain in original order • Parallelizable – Runs faster with multiple computers 8
Mergesort Θ(𝑜 log 𝑜) – Break 𝑜 -element list into two lists of Τ Run Time? 𝑜 2 elements Divide: • – If 𝑜 > 1 : Sort each sublist recursively Conquer: • – If 𝑜 = 1 : List is already sorted (base case) Optimal! Combine: • – Merge together sorted sublists into one sorted list In Place? Adaptive? Stable? No No Yes! (usually)
Merge • Combine: Merge sorted sublistsinto one sorted list – 2 sorted lists ( 𝑀 1 , 𝑀 2 ) • We have: – 1 output list ( 𝑀 𝑝𝑣𝑢 ) While ( 𝑀 1 and 𝑀 2 not empty): If 𝑀 1 0 ≤ 𝑀 2 [0] : Stable: 𝑀 𝑝𝑣𝑢 .append( 𝑀 1 .pop()) If elements are equal, leftmost comes first 𝑀 𝑝𝑣𝑢 .append( 𝑀 2 .pop()) Else: 𝑀 𝑝𝑣𝑢 .append( 𝑀 1 ) 𝑀 𝑝𝑣𝑢 .append( 𝑀 2 ) 10
Mergesort Θ(𝑜 log 𝑜) – Break 𝑜 -element list into two lists of Τ Run Time? 𝑜 2 elements Divide: • – If 𝑜 > 1 : Sort each sublist recursively Conquer: • – If 𝑜 = 1 : List is already sorted (base case) Optimal! Combine: • – Merge together sorted sublists into one sorted list In Place? Adaptive? Stable? Parallelizable? No No Yes! Yes! (usually) 11
Mergesort Parallelizable: – Break 𝑜 -element list into two lists of Τ 𝑜 2 elements • Divide: Allow different machines to work – If 𝑜 > 1 : on each sublist • Conquer: – If 𝑜 = 1 : • Sort each sublist recursively • List is already sorted (base case) • Combine: – Merge together sorted sublists into one sorted list 12
𝑈 𝑜 = 2𝑈(𝑜 Mergesort (Sequential) 2 ) + 𝑜 𝑜 𝑜 𝑜 total / level 𝑜 Τ Τ 𝑜 2 𝑜 𝑜 2 2 2 𝑜 log 2 𝑜 levels 𝑜 𝑜 Τ 𝑜 Τ Τ Τ 𝑜 4 𝑜 4 𝑜 4 𝑜 4 4 4 4 4 of recursion 1 1 1 1 1 1 1 1 1 … 1 1 1 … … … … Run Time: Θ(𝑜 log𝑜)
𝑈 𝑜 = 𝑈(𝑜 Mergesort (Parallel) 2 ) + 𝑜 𝑜 𝑜 𝑜 Done in Parallel 𝑜 Τ Τ 𝑜 2 𝑜 𝑜 2 2 2 2 𝑜 𝑜 𝑜 𝑜 Τ 𝑜 Τ Τ Τ 𝑜 4 𝑜 4 𝑜 4 𝑜 4 4 4 4 4 4 1 1 1 1 1 1 1 1 1 1 … 1 1 1 … … … … Run Time: Θ(𝑜)
Quicksort Θ(𝑜 log 𝑜) Run Time? Idea: pick a partition element, recursively sort • Divide: select an element 𝑞 , Partition( 𝑞 ) two sublists around that element • (almost always) Conquer: recursively sort left and right sublists • Better constants Combine: Nothing! • than Mergesort In Place? Adaptive? Stable? Parallelizable? kinda No! No Yes! Uses stack for recursive calls
Bubble Sort • Idea: March through list, swapping adjacent elements if out of order, repeat until sorted 8 5 7 9 12 10 1 2 4 3 6 11 5 8 7 9 12 10 1 2 4 3 6 11 5 7 8 9 12 10 1 2 4 3 6 11 5 7 8 9 12 10 1 2 4 3 6 11 16
Bubble Sort Θ(𝑜 2 ) Run Time? Idea: March through list, swapping adjacent • elements if out of order, repeat until sorted Constants worse than Insertion Sort In Place? Adaptive? “Compared to straight Yes Kinda insertion […], bubble sorting requires a more complicated program and takes about twice as long!” – Donald Knuth
Bubble Sort is “almost” Adaptive • Idea: March through list, swapping adjacent elements if out of order 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12 Only makes one “pass” 2 3 4 5 6 7 8 9 10 11 12 1 After one “pass” Requires 𝑜 passes, thus is 𝑃(𝑜 2 ) 2 3 4 5 6 7 8 9 10 11 1 12 18
Bubble Sort Θ(𝑜 2 ) Run Time? Idea: March through list, swapping adjacent • elements if out of order, repeat until sorted Constants worse than Insertion Sort In Place? Adaptive? Stable? Parallelizable? Yes! Kinda Yes No Not really "the bubble sort seems to have nothing to recommend it, except a catchy name and the fact that it leads to some interesting theoretical problems” – Donald Knuth, The Art of Computer Programming
Insertion Sort • Idea: Maintain a sorted list prefix, extend that prefix by “inserting” the next element Sorted Prefix 3 5 7 8 10 12 9 2 4 6 1 11 3 5 7 8 10 9 12 2 4 6 1 11 3 5 7 8 9 10 12 2 4 6 1 11 Sorted Prefix 3 5 7 8 9 10 12 2 4 6 1 11 20
Insertion Sort Θ(𝑜 2 ) Run Time? Idea: Maintain a sorted list prefix, extend that • prefix by “inserting” the next element (but with very small constants) Great for short lists! In Place? Adaptive? Yes! Yes
Insertion Sort is Adaptive • Idea: Maintain a sorted list prefix, extend that prefix by “inserting” the next element Sorted Prefix 1 2 3 4 5 6 7 8 9 10 11 12 Sorted Prefix 1 2 3 4 5 6 7 8 9 10 11 12 Runtime: 𝑃(𝑜) Only one comparison needed per element! 22
Insertion Sort Θ(𝑜 2 ) Run Time? Idea: Maintain a sorted list prefix, extend that • prefix by “inserting” the next element (but with very small constants) Great for short lists! In Place? Adaptive? Stable? Yes! Yes Yes
Insertion Sort is Stable • Idea: Maintain a sorted list prefix, extend that prefix by “inserting” the next element Sorted Prefix 3 5 7 8 10 12 10’ 2 4 6 1 11 3 5 7 8 10 10’ 12 2 4 6 1 11 Sorted Prefix 3 5 7 8 10 10’ 12 2 4 6 1 11 The “second” 10 will stay to the right 24
Insertion Sort Θ(𝑜 2 ) Run Time? Idea: Maintain a sorted list prefix, extend that • prefix by “inserting” the next element (but with very small constants) Great for short lists! In Place? Adaptive? Stable? Parallelizable? Yes! Yes Yes No Can sort a list as it is received, Online? i.e., don’t need the entire list Yes to begin sorting “All things considered, it’s actually a pretty good sorting algorithm!” – Nate Brunelle
Heap Sort • Idea: Build a Heap, repeatedly extract max element from the heap to build sorted list Right-to-Left 3 10 9 6 8 7 5 2 4 1 0 1 2 3 4 5 6 7 8 9 10 Max Heap 10 Property: Each node is larger 1 6 9 than its children 3 2 5 2 8 7 6 7 4 5 3 4 1 8 9 10 26
Heap Sort • Remove the Max element (i.e. the root) from the Heap: replace with last element, call Heapify(root) 3 9 6 8 7 5 2 4 1 0 1 2 3 4 5 6 7 8 9 10 Max Heap Property: Each 3 node is larger 1 than its children 6 9 3 2 5 2 8 7 6 7 4 5 4 1 Heapify(node): if node satisfies heap property, done. Else swap with largest 8 9 child and recurse on that subtree 27
Heap Sort • Remove the Max element (i.e. the root) from the Heap: replace with last element, call Heapify(root) 9 3 6 8 7 5 2 4 1 0 1 2 3 4 5 6 7 8 9 10 Max Heap Property: Each 9 node is larger 1 than its children 6 3 3 2 5 2 8 7 6 7 4 5 4 1 Heapify(node): if node satisfies heap property, done. Else swap with largest 8 9 child and recurse on that subtree 28
Heap Sort • Remove the Max element (i.e. the root) from the Heap: replace with last element, call Heapify(root) 9 8 6 3 7 5 2 4 1 0 1 2 3 4 5 6 7 8 9 10 Max Heap Property: Each 9 node is larger 1 than its children 6 8 3 2 5 2 3 7 6 7 4 5 4 1 Heapify(node): if node satisfies heap property, done. Else swap with largest 8 9 child and recurse on that subtree 29
Recommend
More recommend
