Quicksort Proposed by C.A.R. Hoare in 1962. Divide-and-conquer - PDF document

CS 3343 -- Fall 2007 Quicksort • Proposed by C.A.R. Hoare in 1962. • Divide-and-conquer algorithm. • Sorts “in place” (like insertion sort, but not like merge sort). • Very practical (with tuning). Quicksort Carola Wenk Slides courtesy of Charles Leiserson with small changes by Carola Wenk 9/27/07 CS 3343 Analysis of Algorithms 1 9/27/07 CS 3343 Analysis of Algorithms 2 Divide and conquer Partitioning subroutine P ARTITION ( A , p , q ) // A [ p . . q ] Quicksort an n -element array: x ← A [ p ] // pivot = A [ p ] Running time Running time 1. Divide: Partition the array into two subarrays i ← p = O ( n ) for n = O ( n ) for n around a pivot x such that elements in lower for j ← p + 1 to q elements. elements. subarray ≤ x ≤ elements in upper subarray. do if A [ j ] ≤ x then i ← i + 1 ≤ x ≥ x x ≤ x ≥ x x exchange A [ i ] ↔ A [ j ] exchange A [ p ] ↔ A [ i ] 2. Conquer: Recursively sort the two subarrays. return i 3. Combine: Trivial. ≤ x ≥ x Invariant: x ? ≤ x ≥ x x ? Key: Linear-time partitioning subroutine. p i j q 9/27/07 CS 3343 Analysis of Algorithms 3 9/27/07 CS 3343 Analysis of Algorithms 4 Example of partitioning Example of partitioning 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 i j i j 9/27/07 CS 3343 Analysis of Algorithms 5 9/27/07 CS 3343 Analysis of Algorithms 6 1

Example of partitioning Example of partitioning 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 i j 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 i j 9/27/07 CS 3343 Analysis of Algorithms 7 9/27/07 CS 3343 Analysis of Algorithms 8 Example of partitioning Example of partitioning 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 i j i j 9/27/07 CS 3343 Analysis of Algorithms 9 9/27/07 CS 3343 Analysis of Algorithms 10 Example of partitioning Example of partitioning 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 10 8 13 2 11 6 5 3 10 8 13 2 11 6 5 3 10 8 13 2 11 i j i j 9/27/07 CS 3343 Analysis of Algorithms 11 9/27/07 CS 3343 Analysis of Algorithms 12 2

Example of partitioning Example of partitioning 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 10 8 13 2 11 6 5 3 10 8 13 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 6 5 3 2 8 13 10 11 6 5 3 2 8 13 10 11 6 5 3 2 8 13 10 11 i j i j 9/27/07 CS 3343 Analysis of Algorithms 13 9/27/07 CS 3343 Analysis of Algorithms 14 Example of partitioning Example of partitioning 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 10 13 5 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 13 10 8 3 2 11 6 5 3 10 8 13 2 11 6 5 3 10 8 13 2 11 6 5 3 10 8 13 2 11 6 5 3 10 8 13 2 11 6 5 3 2 8 13 10 11 6 5 3 2 8 13 10 11 6 5 3 2 8 13 10 11 6 5 3 2 8 13 10 11 i j 2 5 3 6 8 13 10 11 2 5 3 6 8 13 10 11 i 9/27/07 CS 3343 Analysis of Algorithms 15 9/27/07 CS 3343 Analysis of Algorithms 16 Pseudocode for quicksort Analysis of quicksort Q UICKSORT ( A , p, r ) • Assume all input elements are distinct. if p < r then q ← P ARTITION ( A , p, r ) • In practice, there are better partitioning algorithms for when duplicate input Q UICKSORT ( A , p, q –1) elements may exist. Q UICKSORT ( A , q+ 1 , r ) • Let T(n) = worst-case running time on an array of n elements. Initial call: Q UICKSORT ( A , 1 , n ) 9/27/07 CS 3343 Analysis of Algorithms 17 9/27/07 CS 3343 Analysis of Algorithms 18 3

Worst-case of Worst-case recursion tree quicksort T ( n ) = T (0) + T ( n –1) + cn • Input sorted or reverse sorted. • Partition around min or max element. • One side of partition always has no elements. = + − + Θ T ( n ) T ( 0 ) T ( n 1 ) ( n ) = Θ + − + Θ ( 1 ) T ( n 1 ) ( n ) = − + Θ T ( n 1 ) ( n ) = Θ ( n 2 ) (arithmetic series) 9/27/07 CS 3343 Analysis of Algorithms 19 9/27/07 CS 3343 Analysis of Algorithms 20 Worst-case recursion tree Worst-case recursion tree T ( n ) = T (0) + T ( n –1) + cn T ( n ) = T (0) + T ( n –1) + cn T ( n ) cn T (0) T ( n –1) 9/27/07 CS 3343 Analysis of Algorithms 21 9/27/07 CS 3343 Analysis of Algorithms 22 Worst-case recursion tree Worst-case recursion tree T ( n ) = T (0) + T ( n –1) + cn T ( n ) = T (0) + T ( n –1) + cn cn cn T (0) c ( n –1) T (0) c ( n –1) T (0) T ( n –2) T (0) c ( n –2) T (0) O T (0) 9/27/07 CS 3343 Analysis of Algorithms 23 9/27/07 CS 3343 Analysis of Algorithms 24 4

Worst-case recursion tree Worst-case recursion tree T ( n ) = T (0) + T ( n –1) + cn T ( n ) = T (0) + T ( n –1) + cn  height  ( )  n  ( ) Θ ∑ Θ ∑ cn cn     = Θ 2 = Θ 2 k n k n         T (0) c ( n –1) T (0) c ( n –1) = = k 1 k 1 T (0) c ( n –2) T (0) c ( n –2) height = n height = n T (0) T (0) O O T (0) T (0) 9/27/07 CS 3343 Analysis of Algorithms 25 9/27/07 CS 3343 Analysis of Algorithms 26 Worst-case recursion tree Best-case analysis (For intuition only!) T ( n ) = T (0) + T ( n –1) + cn If we’re lucky, P ARTITION splits the array evenly:  n  ( ) Θ ∑ cn   = Θ 2 k n   T ( n ) = 2 T ( n /2) + Θ ( n )   Θ (1) c ( n –1) k = 1 = Θ ( n log n ) (same as merge sort) Θ (1) c ( n –2) height = n T ( n ) = Θ ( n ) + Θ ( n 2 ) 1 : 9 Θ (1) What if the split is always ? = Θ ( n 2 ) 10 10 O ( ) ( ) = + + Θ T ( n ) T 1 n T 9 n ( n ) Θ (1) 10 10 What is the solution to this recurrence? 9/27/07 CS 3343 Analysis of Algorithms 27 9/27/07 CS 3343 Analysis of Algorithms 28 Analysis of “almost-best” case Analysis of “almost-best” case T ( n ) cn ( ) ( ) T 10 n 1 T 10 n 9 9/27/07 CS 3343 Analysis of Algorithms 29 9/27/07 CS 3343 Analysis of Algorithms 30 5

Analysis of “almost-best” case Analysis of “almost-best” case cn cn cn 1 cn 1 cn 9 cn 9 cn cn 10 10 10 10 log 10/9 n ( ) ( ) ( ) ( ) cn cn cn cn T 100 1 n T 100 9 n T 100 9 n T 100 81 n 1 9 9 81 cn 100 100 100 100 … … … O ( n ) leaves Θ (1) O ( n ) leaves Θ (1) 9/27/07 CS 3343 Analysis of Algorithms 31 9/27/07 CS 3343 Analysis of Algorithms 32 Analysis of “almost-best” case Quicksort Runtimes • Best case runtime T best ( n ) ∈ O( n log n ) cn cn • Worst case runtime T worst ( n ) ∈ O( n 2 ) cn 1 cn cn 9 10 log 10 n 10 log 10/9 n • Worse than mergesort? Why is it called cn 9 cn 9 cn 81 cn cn 1 100 100 100 100 quicksort then? … … • Its average runtime T avg ( n ) ∈ O( n log n ) … Θ (1) O ( n ) leaves O ( n ) leaves • Better even, the expected runtime of Θ (1) randomized quicksort is O( n log n ) Θ ( n log n ) cn log 10 n ≤ T ( n ) ≤ cn log 10/9 n + Ο ( n ) 9/27/07 CS 3343 Analysis of Algorithms 33 9/27/07 CS 3343 Analysis of Algorithms 34 Average Runtime Average Runtime The average runtime T avg ( n ) for Quicksort is • What kind of inputs are there? the average runtime over all possible inputs • Do [1,2,…, n ] and [5,6,…, n +5] cause of length n. different runtimes of Quicksort? • No. Therefore only consider all • What kind of inputs are there? permutations of [1,2,…, n ] . • How many inputs are there? • How many inputs are there? • There are n ! different permutations of [1,2,…, n ] 9/27/07 CS 3343 Analysis of Algorithms 35 9/27/07 CS 3343 Analysis of Algorithms 36 6