Floyd’s buildheap algorithm: example 9 5 5 0 6 3 7 2 8 1 6 6 10 3 11 2 12 1 13 4 14 15 4 3 A visualization: 5 10 9 7 3 2 6 1 0 8 3 7 2 0 1 4 10 0 9 1 8 2 13
Floyd’s buildheap algorithm: example 9 2 5 0 6 3 7 2 8 1 6 6 10 3 11 5 12 1 13 4 14 15 4 3 A visualization: 2 10 9 7 3 2 6 1 0 8 3 7 5 0 1 4 10 0 9 1 8 2 13
Floyd’s buildheap algorithm: example 9 2 5 0 6 3 7 2 8 1 6 0 10 3 11 5 12 1 13 4 14 15 4 3 A visualization: 2 10 9 7 3 2 0 1 6 8 3 7 5 0 1 4 10 0 9 1 8 2 13
Floyd’s buildheap algorithm: example 9 2 5 0 6 3 7 7 8 1 6 0 10 3 11 5 12 1 13 4 14 15 4 3 A visualization: 2 10 9 2 3 7 0 1 6 8 3 2 5 0 1 4 10 0 9 1 8 2 13
Floyd’s buildheap algorithm: example 9 2 5 1 6 3 7 7 8 1 6 0 10 3 11 5 12 8 13 4 14 15 4 3 A visualization: 2 10 9 2 3 7 0 1 6 0 3 2 5 1 8 4 10 0 9 1 0 2 13
Floyd’s buildheap algorithm: example 9 2 5 1 6 3 7 7 8 9 6 1 10 3 11 5 12 8 13 4 14 15 4 3 A visualization: 2 10 0 2 3 7 1 9 6 0 3 2 5 1 8 4 10 0 0 1 0 2 13
Floyd’s buildheap algorithm: example 9 2 5 1 6 3 7 7 8 9 6 1 10 3 11 5 12 8 13 4 14 15 4 3 A visualization: 2 0 1 2 3 7 6 9 10 0 3 2 5 1 8 4 10 0 0 1 0 2 13
Floyd’s buildheap algorithm Yes – algorithm is n log n , but with a more careful analysis, we can show it’s n ! 14 Wait... isn’t this still n log ( n ) ? We look at n nodes, and we run percolateDown(...) on each node, which takes log ( n ) time... right?
14 Floyd’s buildheap algorithm Wait... isn’t this still n log ( n ) ? We look at n nodes, and we run percolateDown(...) on each node, which takes log ( n ) time... right? Yes – algorithm is O ( n log ( n )) , but with a more careful analysis, we can show it’s O ( n ) !
Analyzing Floyd’s buildheap algorithm (1 node) (4 work) (2 nodes) (3 work) (4 nodes) (2 work) (8 nodes) (1 work) What’s the pattern? work n n n n 15 Question: How much work is percolateDown actually doing?
Analyzing Floyd’s buildheap algorithm (1 node) (4 work) (2 nodes) (3 work) (4 nodes) (2 work) What’s the pattern? work n n n n 15 Question: How much work is percolateDown actually doing? (8 nodes) × (1 work)
Analyzing Floyd’s buildheap algorithm (1 node) (4 work) (2 nodes) (3 work) What’s the pattern? work n n n n 15 Question: How much work is percolateDown actually doing? (4 nodes) × (2 work) (8 nodes) × (1 work)
Analyzing Floyd’s buildheap algorithm (1 node) (4 work) What’s the pattern? work n n n n 15 Question: How much work is percolateDown actually doing? (2 nodes) × (3 work) (4 nodes) × (2 work) (8 nodes) × (1 work)
What’s the pattern? Analyzing Floyd’s buildheap algorithm work n n n n 15 Question: How much work is percolateDown actually doing? (1 node) × (4 work) (2 nodes) × (3 work) (4 nodes) × (2 work) (8 nodes) × (1 work)
What’s the pattern? Analyzing Floyd’s buildheap algorithm work n n n n 15 Question: How much work is percolateDown actually doing? (1 node) × (4 work) (2 nodes) × (3 work) (4 nodes) × (2 work) (8 nodes) × (1 work)
What’s the pattern? Analyzing Floyd’s buildheap algorithm 15 Question: How much work is percolateDown actually doing? (1 node) × (4 work) (2 nodes) × (3 work) (4 nodes) × (2 work) (8 nodes) × (1 work) work ( n ) ≈ n 2 · 1 + n 4 · 2 + n 8 · 3 + · · ·
Analyzing Floyd’s buildheap algorithm What is i i i n i i i n work n (Seems hard to analyze...) So let’s just make it infjnity! It’s the height of the tree: so log n . supposed to be? i We had: i i n work n Can we write this in summation form? Yes. n work n Let’s rewrite bottom as powers of two, and factor out the n : 16 work ( n ) ≈ n 2 · 1 + n 4 · 2 + n 8 · 3 + · · ·
Analyzing Floyd’s buildheap algorithm i i i i n i i i n work n (Seems hard to analyze...) So let’s just make it infjnity! It’s the height of the tree: so log n . supposed to be? What is i i We had: n work n Can we write this in summation form? Yes. Let’s rewrite bottom as powers of two, and factor out the n : 16 work ( n ) ≈ n 2 · 1 + n 4 · 2 + n 8 · 3 + · · · � 1 2 1 + 2 2 2 + 3 � work ( n ) ≈ n 2 3 + · · ·
Analyzing Floyd’s buildheap algorithm We had: i i i n i i i n work n (Seems hard to analyze...) So let’s just make it infjnity! It’s the height of the tree: so log n . supposed to be? What is i 16 Can we write this in summation form? Yes. Let’s rewrite bottom as powers of two, and factor out the n : work ( n ) ≈ n 2 · 1 + n 4 · 2 + n 8 · 3 + · · · � 1 2 1 + 2 2 2 + 3 � work ( n ) ≈ n 2 3 + · · · ? � work ( n ) ≈ n 2 i i =1
Analyzing Floyd’s buildheap algorithm We had: i i i n i i i n work n (Seems hard to analyze...) So let’s just make it infjnity! It’s the height of the tree: so log n . i 16 Let’s rewrite bottom as powers of two, and factor out the n : Can we write this in summation form? Yes. work ( n ) ≈ n 2 · 1 + n 4 · 2 + n 8 · 3 + · · · � 1 2 1 + 2 2 2 + 3 � work ( n ) ≈ n 2 3 + · · · ? � work ( n ) ≈ n 2 i i =1 What is ? supposed to be?
Analyzing Floyd’s buildheap algorithm We had: i i i n i i i n work n So let’s just make it infjnity! (Seems hard to analyze...) i 16 Let’s rewrite bottom as powers of two, and factor out the n : Can we write this in summation form? Yes. work ( n ) ≈ n 2 · 1 + n 4 · 2 + n 8 · 3 + · · · � 1 2 1 + 2 2 2 + 3 � work ( n ) ≈ n 2 3 + · · · ? � work ( n ) ≈ n 2 i i =1 What is ? supposed to be? It’s the height of the tree: so log ( n ) .
Analyzing Floyd’s buildheap algorithm Can we write this in summation form? Yes. i i (Seems hard to analyze...) So let’s just make it infjnity! i We had: 16 Let’s rewrite bottom as powers of two, and factor out the n : work ( n ) ≈ n 2 · 1 + n 4 · 2 + n 8 · 3 + · · · � 1 2 1 + 2 2 2 + 3 � work ( n ) ≈ n 2 3 + · · · ? � work ( n ) ≈ n 2 i i =1 What is ? supposed to be? It’s the height of the tree: so log ( n ) . ? ∞ � � work ( n ) ≈ n 2 i ≤ n 2 i i =1 i =1
So buildHeap runs in Analyzing Floyd’s buildheap algorithm Strategy: prove the summation is upper-bounded by something n time! n i i i n work n Using an identity (see page 4 of Weiss): i 17 i upper-bounded by the same thing. If we can do this, then our original summation must defjnitely be even when the summation goes on for infjnity. ? ∞ � � work ( n ) ≈ n 2 i ≤ n 2 i i =1 i =1
So buildHeap runs in Analyzing Floyd’s buildheap algorithm i n time! i Using an identity (see page 4 of Weiss): i Strategy: prove the summation is upper-bounded by something 17 upper-bounded by the same thing. even when the summation goes on for infjnity. If we can do this, then our original summation must defjnitely be ? ∞ � � work ( n ) ≈ n 2 i ≤ n 2 i i =1 i =1 ∞ � work ( n ) ≤ n 2 i = n · 2 i =1
Analyzing Floyd’s buildheap algorithm i i Using an identity (see page 4 of Weiss): i Strategy: prove the summation is upper-bounded by something 17 If we can do this, then our original summation must defjnitely be upper-bounded by the same thing. even when the summation goes on for infjnity. ? ∞ � � work ( n ) ≈ n 2 i ≤ n 2 i i =1 i =1 ∞ � work ( n ) ≤ n 2 i = n · 2 i =1 So buildHeap runs in O ( n ) time!
Analyzing Floyd’s buildheap algorithm Lessons learned: So design an algorithm that does less work closer to ‘bottom’ More careful analysis can reveal tighter bounds Strategy: rather then trying to show a b directly, it can sometimes be simpler to show a t then t b . (Similar to what we did when fjnding c and n questions when doing asymptotic analysis!) 18 ◮ Most of the nodes near leaves (almost 1 2 of nodes are leaves!)
Analyzing Floyd’s buildheap algorithm Lessons learned: So design an algorithm that does less work closer to ‘bottom’ Strategy: rather then trying to show a b directly, it can sometimes be simpler to show a t then t b . (Similar to what we did when fjnding c and n questions when doing asymptotic analysis!) 18 ◮ Most of the nodes near leaves (almost 1 2 of nodes are leaves!) ◮ More careful analysis can reveal tighter bounds
Analyzing Floyd’s buildheap algorithm Lessons learned: So design an algorithm that does less work closer to ‘bottom’ doing asymptotic analysis!) 18 ◮ Most of the nodes near leaves (almost 1 2 of nodes are leaves!) ◮ More careful analysis can reveal tighter bounds ◮ Strategy: rather then trying to show a ≤ b directly, it can sometimes be simpler to show a ≤ t then t ≤ b . (Similar to what we did when fjnding c and n 0 questions when
Analyzing Floyd’s buildheap algorithm What we’re skipping Other kinds of heaps (leftist heaps, skew heaps, binomial queues) 19 ◮ How do we merge two heaps together?
Analyzing Floyd’s buildheap algorithm What we’re skipping queues) 19 ◮ How do we merge two heaps together? ◮ Other kinds of heaps (leftist heaps, skew heaps, binomial
On to sorting And now on to sorting... 20
You should just use Collections.sort(...) Why study sorting? A vehicle for talking about a technique called “divide-and-conquer” Difgerent sorts have difgerent purposes/tradeofgs. (General purpose sorts work well most of the time, but you might need something more effjcient in niche cases) It’s a “thing everybody knows”. 21 Why not just use Collections.sort(...) ?
Why study sorting? A vehicle for talking about a technique called “divide-and-conquer” Difgerent sorts have difgerent purposes/tradeofgs. (General purpose sorts work well most of the time, but you might need something more effjcient in niche cases) It’s a “thing everybody knows”. 21 Why not just use Collections.sort(...) ? ◮ You should just use Collections.sort(...)
Why study sorting? “divide-and-conquer” Difgerent sorts have difgerent purposes/tradeofgs. (General purpose sorts work well most of the time, but you might need something more effjcient in niche cases) It’s a “thing everybody knows”. 21 Why not just use Collections.sort(...) ? ◮ You should just use Collections.sort(...) ◮ A vehicle for talking about a technique called
Why study sorting? “divide-and-conquer” (General purpose sorts work well most of the time, but you might need something more effjcient in niche cases) It’s a “thing everybody knows”. 21 Why not just use Collections.sort(...) ? ◮ You should just use Collections.sort(...) ◮ A vehicle for talking about a technique called ◮ Difgerent sorts have difgerent purposes/tradeofgs.
Why study sorting? “divide-and-conquer” (General purpose sorts work well most of the time, but you might need something more effjcient in niche cases) 21 Why not just use Collections.sort(...) ? ◮ You should just use Collections.sort(...) ◮ A vehicle for talking about a technique called ◮ Difgerent sorts have difgerent purposes/tradeofgs. ◮ It’s a “thing everybody knows”.
Less formally: the compareTo(...) method can’t be broken. Types of sorts b and b time at best. n log n Fact: comparison sorts will run in a is true (or both) b is true, or b Either a c c then a If a Two difgerent kinds of sorts: b a then a b and b If a must be true. Formally: for every element a , b , and c in the list, the following Assumes elements in list form a consistent, total ordering : Works by comparing two elements at a time. Comparison sorts 22
Less formally: the compareTo(...) method can’t be broken. Types of sorts Two difgerent kinds of sorts: Comparison sorts Works by comparing two elements at a time. Assumes elements in list form a consistent, total ordering : Formally: for every element a , b , and c in the list, the following must be true. Fact: comparison sorts will run in n log n time at best. 22 ◮ If a ≤ b and b ≤ a then a = b ◮ If a ≤ b and b ≤ c then a ≤ c ◮ Either a ≤ b is true, or b ≤ a is true (or both)
Types of sorts Two difgerent kinds of sorts: Comparison sorts Works by comparing two elements at a time. Assumes elements in list form a consistent, total ordering : Formally: for every element a , b , and c in the list, the following must be true. Fact: comparison sorts will run in n log n time at best. 22 ◮ If a ≤ b and b ≤ a then a = b ◮ If a ≤ b and b ≤ c then a ≤ c ◮ Either a ≤ b is true, or b ≤ a is true (or both) Less formally: the compareTo(...) method can’t be broken.
Types of sorts Two difgerent kinds of sorts: Comparison sorts Works by comparing two elements at a time. Assumes elements in list form a consistent, total ordering : Formally: for every element a , b , and c in the list, the following must be true. 22 ◮ If a ≤ b and b ≤ a then a = b ◮ If a ≤ b and b ≤ c then a ≤ c ◮ Either a ≤ b is true, or b ≤ a is true (or both) Less formally: the compareTo(...) method can’t be broken. Fact: comparison sorts will run in O ( n log ( n )) time at best.
Types of sorts Two difgerent kinds of sorts: Niche sorts (aka “linear sorts”) Exploits certain properties about the items in the list to reach Faster, but less general-purpose. We’ll focus on comparison sorts, will cover a few linear sorts if time. 23 faster runtimes (typically, O ( n ) time).
Types of sorts Two difgerent kinds of sorts: Niche sorts (aka “linear sorts”) Exploits certain properties about the items in the list to reach Faster, but less general-purpose. We’ll focus on comparison sorts, will cover a few linear sorts if time. 23 faster runtimes (typically, O ( n ) time).
Types of sorts Two difgerent kinds of sorts: Niche sorts (aka “linear sorts”) Exploits certain properties about the items in the list to reach Faster, but less general-purpose. We’ll focus on comparison sorts, will cover a few linear sorts if time. 23 faster runtimes (typically, O ( n ) time).
More defjnitions In-place sort space to sort the array. 24 A sorting algorithm is in-place if it requires only O (1) extra ◮ Usually modifjes input array ◮ Can be useful: lets us minimize memory
More defjnitions Stable sort A sorting algorithm is stable if any equal items remain in the same relative order before and after the sort. attribute of an item 25 ◮ Observation: We sometimes want to sort on some, but not all ◮ Items that ’compare’ the same might not be exact duplicates ◮ Sometimes useful to sort on one attribute fjrst, then another
[(4, "wolf"), (8, "fox"), (8, "cow"), (9, "dog")] [(4, "wolf"), (8, "cow"), (8, "fox"), (9, "dog")] Stable sort: Example Input: Output; stable sort: Output; unstable sort: 26 ◮ Array: [(8, "fox"), (9, "dog"), (4, "wolf"), (8, "cow")] ◮ Compare function: compare pairs by number only
[(4, "wolf"), (8, "cow"), (8, "fox"), (9, "dog")] Stable sort: Example Input: Output; stable sort: Output; unstable sort: 26 ◮ Array: [(8, "fox"), (9, "dog"), (4, "wolf"), (8, "cow")] ◮ Compare function: compare pairs by number only [(4, "wolf"), (8, "fox"), (8, "cow"), (9, "dog")]
Stable sort: Example Input: Output; stable sort: Output; unstable sort: 26 ◮ Array: [(8, "fox"), (9, "dog"), (4, "wolf"), (8, "cow")] ◮ Compare function: compare pairs by number only [(4, "wolf"), (8, "fox"), (8, "cow"), (9, "dog")] [(4, "wolf"), (8, "cow"), (8, "fox"), (9, "dog")]
Overview of sorting algorithms There are many sorts... Quicksort, Merge sort, In-place merge sort, Heap sort, Insertion sort, Intro sort, Selection sort, Timsort, Cubesort, Shell sort, Bubble sort, Binary tree sort, Cycle sort, Library sort, Patience sorting, Smoothsort, Strand sort, Tournament sort, Cocktail sort, Comb sort, Gnome sort, Block sort, Stackoverfmow sort, Odd-even sort, Pigeonhole sort, Bucket sort, Counting sort, Radix sort, Spreadsort, Burstsort, Flashsort, Postman sort, Bead sort, Simple pancake sort, Spaghetti sort, Sorting network, Bitonic sort, Bogosort, Stooge sort, Insertion sort, Slow sort, Rainbow sort... ...we’ll focus on a few 27
Overview of sorting algorithms There are many sorts... Quicksort, Merge sort, In-place merge sort, Heap sort, Insertion sort, Intro sort, Selection sort, Timsort, Cubesort, Shell sort, Bubble sort, Binary tree sort, Cycle sort, Library sort, Patience sorting, Smoothsort, Strand sort, Tournament sort, Cocktail sort, Comb sort, Gnome sort, Block sort, Stackoverfmow sort, Odd-even sort, Pigeonhole sort, Bucket sort, Counting sort, Radix sort, Spreadsort, Burstsort, Flashsort, Postman sort, Bead sort, Simple pancake sort, Spaghetti sort, Sorting network, Bitonic sort, Bogosort, Stooge sort, Insertion sort, Slow sort, Rainbow sort... ...we’ll focus on a few 27
a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] 10 4 2 5 8 2 3 6 7 8 1 4 10 2 8 1 Insertion Sort 8 8 3 6 7 8 5 1 4 2 2 10 28 INSERT current item into sorted region 7 6 5 3 2 Current item Unsorted Already sorted a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10]
a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] 10 4 2 5 8 2 3 6 7 8 1 4 10 2 8 1 Insertion Sort 8 8 3 6 7 8 5 1 4 2 2 10 28 INSERT current item into sorted region 7 6 5 3 2 Current item Unsorted Already sorted a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10]
a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] 1 4 10 8 2 8 2 2 3 Insertion Sort 5 6 6 7 8 1 4 10 2 8 7 28 7 10 8 4 Already sorted 8 Unsorted Current item 6 INSERT current item into sorted region 2 3 1 3 5 5 2 a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10]
Insertion Sort 8 3 5 6 7 2 8 1 4 10 2 2 INSERT current item into sorted region 3 5 6 7 8 1 4 10 2 8 2 28 Current item 1 Unsorted 5 10 8 2 7 8 6 3 Already sorted 4 a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10]
Insertion Sort 1 Pseudocode INSERT current item into sorted region Unsorted Already sorted 8 2 10 4 2 5 7 6 3 8 29 a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] ◮ Worst case runtime? for ( int i = 1; i < n; i++) { ◮ Best case runtime? // Find index to insert into int newIndex = findPlace(i); ◮ Average runtime? // Insert and shift nodes over ◮ Stable? shift(newIndex, i); } ◮ In-place?
a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] Selection Sort 11 14 15 6 10 2 3 7 8 9 18 14 11 15 10 18 8 9 10 3 6 7 8 15 18 14 2 9 11 31 Next smallest 7 6 3 2 SELECT next min and swap with current Current item Unsorted Already sorted a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10]
a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] Selection Sort 11 14 15 6 10 2 3 7 8 9 18 14 11 15 10 18 8 9 10 3 6 7 8 15 18 14 2 9 11 31 Next smallest 7 6 3 2 SELECT next min and swap with current Current item Unsorted Already sorted a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10]
a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] 2 Selection Sort 10 15 11 14 18 9 3 8 6 7 7 9 18 14 11 15 10 2 8 6 Current item 3 6 7 8 15 18 14 11 9 Already sorted Unsorted 10 31 2 Next smallest 3 SELECT next min and swap with current a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10]
Selection Sort 10 2 3 6 7 2 9 18 14 11 15 2 Next smallest 3 6 7 8 9 18 14 11 15 10 SELECT next min and swap with current 8 Current item 11 3 6 7 8 15 Unsorted 18 14 31 Already sorted 9 10 a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10]
Selection Sort 18 Pseudocode SELECT next min and swap with current Unsorted Already sorted 10 2 11 14 9 15 7 6 3 8 32 a[0] a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] ◮ Worst case runtime? for ( int i = 0; i < n; i++) { ◮ Best case runtime? // Find next smallest int newIndex = findNextMin(i); ◮ Average runtime? // Swap current and next smallest ◮ Stable? swap(newIndex, i); } ◮ In-place?
Idea: run buildHeap then call removeMin n times. E[] input = buildHeap(...); E[] output = new E[n]; for ( int i = 0; i < n; i++) { output[i] = removeMin(input); } Heap sort Can we use heaps to help us sort? Pseudocode Worst case runtime? Best case runtime? Average runtime? Stable? In-place? 34
E[] input = buildHeap(...); E[] output = new E[n]; for ( int i = 0; i < n; i++) { output[i] = removeMin(input); } Heap sort Can we use heaps to help us sort? Pseudocode Worst case runtime? Best case runtime? Average runtime? Stable? In-place? 34 Idea: run buildHeap then call removeMin n times.
Heap sort Can we use heaps to help us sort? Pseudocode 34 Idea: run buildHeap then call removeMin n times. ◮ Worst case runtime? ◮ Best case runtime? E[] input = buildHeap(...); E[] output = new E[n]; ◮ Average runtime? for ( int i = 0; i < n; i++) { output[i] = removeMin(input); ◮ Stable? } ◮ In-place?
Recommend
More recommend