CS4102 Algorithms Fall 2018 Warm up Compare + with + () When - PowerPoint PPT Presentation

CS4102 Algorithms Fall 2018 Warm up Compare 𝑔 𝑜 + 𝑛 with 𝑔 𝑜 + 𝑔(𝑛) When 𝑔 𝑜 = 𝑃(𝑜) When 𝑔 𝑜 = Ω(𝑜) 1

𝑔 𝑜 = O(𝑜) 𝑔(𝑛) 𝑔(𝑛) 𝑔(𝑜) 𝑔(𝑜) 𝑜 𝑛 𝑜 + 𝑛 𝑔 𝑜 + 𝑛 ≤ 𝑔 𝑜 + 𝑔(𝑛) 2

𝑔 𝑜 = Ω(𝑜) 𝑔(𝑛) 𝑔(𝑛) 𝑔(𝑜) 𝑔(𝑜) 𝑜 𝑛 𝑜 + 𝑛 𝑔 𝑜 + 𝑛 ≥ 𝑔 𝑜 + 𝑔(𝑛) 3

𝑔 𝑜 = Θ(𝑜) 𝑔(𝑛) 𝑔(𝑜) 𝑔(𝑛) 𝑔(𝑜) 𝑜 𝑛 𝑜 + 𝑛 𝑔 𝑜 + 𝑛 = 𝑔 𝑜 + 𝑔(𝑛) 4

Today’s Keywords • Divide and Conquer • Sorting • Quicksort • Median • Order statistic • Quickselect • Median of Medians 5

CLRS Readings • Chapter 7 6

More on HW2 Homeworks • You must read from garden.txt file automatically (it’s a fixed filename) • That file has a list of pairs of floats (not ints) • Hw2 due 11pm Friday! • You must only output one floating point number (minimum distance) – Programming (use Python or Java!) • Uploaded files: • One python file, or – Divide and conquer • One or more java files (uploaded individually) – Closest pair of points • Don’t use packages! • Don’t use subdirectories! • Hw3 released soon • DO NOT upload a zip file! • Try it yourself: – Divide and conquer • Put the files you are going to upload in a directory (with a garden.txt file) – Written (use LaTeX!) • python closestpair_mst3k.py • javac *.java java ClosestPair • Use the one for your language and you should get a result 7

Quicksort • Idea: pick a pivot element, recursively sort two sublists around that element • Divide: select an element 𝑞 , Partition( 𝑞 ) • Conquer: recursively sort left and right sublists • Combine: Nothing! 8

Partition (Divide step) • Given: a list, a pivot 𝑞 Start: unordered list 8 5 7 3 12 10 1 2 4 9 6 11 Goal: All elements < 𝑞 on left, all > 𝑞 on right 5 7 3 1 2 4 6 8 12 10 9 11 9

Partition Summary 1. Put 𝑞 at beginning of list 2. Put a pointer (Begin) just after 𝑞 , and a pointer (End) at the end of the list 3. While Begin < End: 1. If Begin value < 𝑞 , move Begin right 2. Else swap Begin value with End value, move End Left 4. If pointers meet at element < 𝑞 : Swap 𝑞 with pointer position 5. Else If pointers meet at element > 𝑞 : Swap 𝑞 with value to the left 10

Quicksort Run Time • If the pivot is always the median: 2 5 1 3 6 4 7 8 10 9 11 12 2 1 3 5 6 4 7 8 9 10 11 12 • Then we divide in half each time 𝑈 𝑜 = 2𝑈 𝑜 2 + 𝑜 𝑈 𝑜 = 𝑃(𝑜 log 𝑜) 11

Quicksort Run Time • If the partition is always unbalanced: 1 5 2 3 6 4 7 8 10 9 11 12 1 2 3 5 6 4 7 8 10 9 11 12 • Then we shorten by 1 each time 𝑈 𝑜 = 𝑈 𝑜 − 1 + 𝑜 𝑈 𝑜 = 𝑃(𝑜 2 ) 12

Good Pivot • What makes a good Pivot? – Roughly even split between left and right – Ideally: median • Can we find median in linear time? – Yes! – Quickselect 13

Quickselect • Finds 𝑗 th order statistic – 𝑗 th smallest element in the list – 1 st order statistic: minimum – 𝑜 th order statistic: maximum 𝑜 – th order statistic: median 2 14

Quickselect • Finds 𝑗 th order statistic • Idea: pick a pivot element, partition, then recurse on sublist containing index 𝑗 • Divide: select an element 𝑞 , Partition( 𝑞 ) • Conquer: if 𝑗 = index of 𝑞 , done! – if 𝑗 < index of 𝑞 recurse left. Else recurse right • Combine: Nothing! 15

Partition (Divide step) • Given: a list, a pivot value 𝑞 Start: unordered list 8 5 7 3 12 10 1 2 4 9 6 11 Goal: All elements < 𝑞 on left, all > 𝑞 on right 5 7 3 1 2 4 6 8 12 10 9 11 16

Conquer 2 5 7 3 6 4 1 8 10 9 11 12 All elements > 𝑞 All elements < 𝑞 Exactly where it belongs! Recurse on sublist that contains index 𝑗 (add index of the pivot to 𝑗 if recursing right) 17

Quickselect Run Time • If the pivot is always the median: 2 5 1 3 6 4 7 8 10 9 11 12 2 1 3 5 6 4 7 8 9 10 11 12 • Then we divide in half each time 𝑇 𝑜 = 𝑇 𝑜 2 + 𝑜 𝑇 𝑜 = 𝑃(𝑜) 18

Quickselect Run Time • If the partition is always unbalanced: 1 5 2 3 6 4 7 8 10 9 11 12 1 2 3 5 6 4 7 8 10 9 11 12 • Then we shorten by 1 each time 𝑇 𝑜 = 𝑇 𝑜 − 1 + 𝑜 𝑇 𝑜 = 𝑃(𝑜 2 ) 19

Good Pivot • What makes a good Pivot? – Roughly even split between left and right – Ideally: median • Here’s what’s next: – An algorithm for finding a “rough” split – This algorithm uses Quickselect as a subroutine Déjà vu? 20

Good Pivot • What makes a good Pivot? – Both sides of Pivot >30% >30% Or Select Pivot from this range >30% 21

Median of Medians • Fast way to select a “good” pivot • Guarantees pivot is greater than 30% of elements and less than 30% of the elements • Idea: break list into chunks, find the median of each chunk, use the median of those medians 22

Median of Medians 1. Break list into chunks of size 5 2. Find the median of each chunk 3. Return median of medians (using Quickselect) 23

Why is this good? Each chunk sorted, chunks ordered by their medians MedianofMedians < < < < is Greater than all of these < < < < < 5 < < < < < < < < < < < < < < 𝑜 5 24

Why is this good? MedianofMedians < < < < is larger than all of these < < < < < < < < < < < < < < < < < < < Larger than 3 𝑜 things in each 5 (but one) list to 1 𝑜 3𝑜 < 3 2 ⋅ 5 − 2 ≈ 10 − 6 elements the left 1 𝑜 3𝑜 > Similarly: 3 2 ⋅ 5 − 2 ≈ 10 − 6 elements 25

Quickselect • Divide: select an element 𝑞 using Median of Medians, Partition( 𝑞 ) 𝑁 𝑜 + Θ(𝑜) • Conquer: if 𝑗 = index of 𝑞 , done, if 𝑗 < index of 𝑞 recurse left. Else recurse right 7 ≤ 𝑇 𝑜 10 • Combine: Nothing! 7 𝑇 𝑜 ≤ 𝑇 𝑜 + 𝑁 𝑜 + Θ(𝑜) 10 26

Median of Medians, Run Time Θ(𝑜) 1. Break list into chunks of 5 Θ(𝑜) 2. Find the median of each chunk 3. Return median of medians (using Quickselect) 𝑇 𝑜 5 𝑁 𝑜 = 𝑇 𝑜 5 + Θ(𝑜) 27

Quickselect 𝑁 𝑜 = 𝑇 𝑜 𝑇 𝑜 ≤ 𝑇 7𝑜 5 + Θ(𝑜) 10 + 𝑁 𝑜 + Θ(𝑜) = 𝑇 7𝑜 10 + 𝑇 𝑜 5 + Θ(𝑜) = 𝑇 7𝑜 10 + 𝑇 2𝑜 + Θ(𝑜) 10 ≤ 𝑇 9𝑜 + Θ(𝑜) Because 𝑇 𝑜 = Ω(𝑜) 10 Master theorem Case 3! 𝑇 𝑜 = O(𝑜) 28

Phew! Back to Quicksort • Using Quickselect, with a median-of-medians partition: 2 5 1 3 6 4 7 8 10 9 11 12 2 1 3 5 6 4 7 8 9 10 11 12 • Then we divide in half each time 𝑈 𝑜 = 2𝑈 𝑜 2 + Θ(𝑜) 𝑈 𝑜 = Θ(𝑜 log 𝑜) 29

Is it worth it? • Using Quickselect to pick median guarantees Θ(𝑜 log 𝑜) run time • Approach has very large constants – If you really want Θ(𝑜 log 𝑜) , better off using MergeSort • Better approach: Random pivot – Very small constant (very fast algorithm) – Expected to run in Θ(𝑜 log 𝑜) time • Why? Unbalanced partitions are very unlikely 30

Quicksort Run Time • If the pivot is always 𝑜 th order statistic: 10 10 + 𝑈 9𝑜 𝑜 𝑈 𝑜 = 𝑈 + 𝑜 10 31

10 + 𝑈 9𝑜 𝑜 𝑈 𝑜 = 𝑈 10 + 𝑜 𝑜 𝑜 𝑜 𝑜/10 9𝑜/10 𝑜 𝑜 10 9𝑜 10 + 81𝑜/100 9𝑜/ 100 9𝑜/100 𝑜/100 𝑜 100 9𝑜 100 9𝑜 100 81𝑜 100 𝑜 + + + log 10 𝑜 … 9 … … 1 1 + … 1 1 1 + 1 1 + 1

Quicksort Run Time • If the pivot is always 𝑜 th order statistic: 10 10 + 𝑈 9𝑜 𝑜 𝑈 𝑜 = 𝑈 + 𝑜 10 𝑈 𝑜 = Θ(𝑜 log 𝑜) 33

Quicksort Run Time • If the pivot is always 𝑒 th order statistic: 1 5 2 3 6 4 7 8 10 9 11 12 1 2 3 5 6 4 7 8 10 9 11 12 • Then we shorten by 𝑒 each time 𝑈 𝑜 = 𝑈 𝑜 − 𝑒 + 𝑜 𝑈 𝑜 = 𝑃(𝑜 2 ) What’s the probability of this occurring? 34

Probability of 𝑜 2 run time We must consistently select pivot from within the first 𝑒 terms 𝑒 Probability first pivot is among 𝑒 smallest: 𝑜 𝑒 Probability second pivot is among 𝑒 smallest: 𝑜−𝑒 Probability all pivots are among 𝑒 smallest: 𝑒 𝑒 𝑜 − 2𝑒 ⋅ … ⋅ 𝑒 𝑒 1 𝑜 ⋅ 𝑜 − 𝑒 ⋅ 2𝑒 ⋅ 1 = 𝑜 𝑒 ! 35

Formal Argument for 𝑜 log 𝑜 Average • Remember, run time counts comparisons! • Quicksort only compares against a pivot – Element 𝑗 only compared to element 𝑘 if one of them was the pivot 36

Formal Argument for 𝑜 log 𝑜 Average • What is the probability of comparing two given elements? 1 2 3 4 5 6 7 8 9 10 11 12 • (Probability of comparing 3 and 4) = 1 – Why? Otherwise I wouldn’t know which came first – ANY sorting algorithm must compare adjacent elements 37