Recurrence Relations Sorting overview After today, you should be - PowerPoint PPT Presentation

Recurrence Relations Sorting overview After today, you should be able to…  θ ( N log b a ) if a > b k …write recurrences for code snippets   …solve recurrences using telescoping, θ ( N k logN ) if a = b k T ( N ) = recurrence trees, and the master method  θ ( N k ) if a < b k 

A technique for analyzing recursive algorithms

} An equation (or inequality) that relates the N th element of a sequence to certain of its predecessors (recursive case) } Includes an initial condition (base case) } So Solu lutio ion: A function of N. Example. Solve using backward substitution. T(N) = 2T(N/2) + N T(1) = 1

For Forwar ard su subst stitution on Ba Backward substitution Simple Re Recu currence ce trees Often can’t solve difficult relations Visual Great intuition for div-and-conquer Te Telesco coping Widely applicable Difficult to formulate Master The Ma Theorem Not intuitive Immediate Only for div-and-conquer Only gives Big-Theta

What’s N?

1 void sort(a) { sort(a, a.length-1); } void sort(a, last) { if (last == 0) return; find max value in a from 0 to last swap max to last sort(a, last-1) } What’s N?

2–3 } Basic idea: tweak the relation somehow so successive terms cancel } Example: T(1) = 1, T(N) = 2T(N/2) + N where N = 2 k for some k } Divide by N to get a “piece of the telescope”:

4 Recurrence: Level T(N) = 2T(N/2) + N T(1) = 1 0 N 1 N/2 N/2 2 N/4 N/4 N/4 N/4 ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ? 1 1 1 1 1 1 1 1 1 … 1 1 1 How many nodes at level i? 2 i • How much work at level i? 2 i (N/2 i ) = N • Index of last level? log 2 N • Total:

5 } For Divide-and-conquer algorithms ◦ Divide data into two or more parts of the same size ◦ Solve problem on one or more of those parts ◦ Combine "parts" solutions to solve whole problem } Examples ◦ Binary search ◦ Merge Sort ◦ MCSS recursive algorithm we studied last time Th Theorem m 7. 7.5 5 in Weiss

5–7 } For any recurrence in the form: T ( N ) = aT ( N/b ) + θ ( N k ) with a ≥ 1 , b > 1 } The solution is  θ ( N log b a ) if a > b k   θ ( N k logN ) if a = b k T ( N ) = θ ( N k ) if a < b k   Example: 2T(N/4) + N Theorem 7.5 in Weiss

Recurrence: Level T(N) = aT(N/b) + cN k T(1) ≤ c 0 cN k … 1 c(N/b) k c(N/b) k c(N/b) k … … … … 2 c(N/b 2 ) k c(N/b 2 ) k c(N/b 2 ) k c(N/b 2 ) k c(N/b 2 ) k c(N/b 2 ) k c(N/b 2 ) k c(N/b 2 ) k c(N/b 2 ) k ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ? c c c c c c c c c c c c … c c c How many nodes at level i? a i • a i c(N/b i ) k = cN k (a/b k ) i How much work at level i? • Index of last level? log b N • Summation:

} Upper bound on work at level i: } a = “Rate of subproblem proliferation” 😠 } b k = “Rate of work shrinkage” 😄 😠 a < b k 😄 😠 a = b k 😄 😠 a > b k 😄 Ca Case As As level i 😄 work goes 😑 work stays 😠 work goes in increases… down! same up! T( T(N) dominated Root of tree Every level Leaves of tree by by work do done similar at… at Θ (N k lo Ma Master Th Theorem Θ (N k ) log N) Θ (N lo log b a ) sa says s T(N) N) in in…

log b N ⇣ a ⌘ i } Case 1. a < b k X cN k b k i =0 ✓ 1 − ( a/b k ) log b N +1 ◆ ✓ ◆ 1 cN k ≈ cN k 1 − ( a/b k ) 1 − ( a/b k ) } Case 2. a = b k log b N X cN k 1 = cN k (log b N + 1) i =0 } Case 3. a > b k ✓ ( a/b k ) log b N +1 − 1 ◆ ≈ cN k ( a/b k ) log b N = ca log b N = cN log b a cN k ( a/b k ) − 1

} Analyze code to determine relation ◦ Base case in code gives base case for relation ◦ Number and “size” of recursive calls determine recursive part of recursive case ◦ Non-recursive code determines rest of recursive case } Apply a strategy ◦ Guess and check (substitution) ◦ Telescoping ◦ Recurrence tree ◦ Master theorem

Quick look at several sorting methods Focus on quicksort Quicksort average case analysis

8–10 10 } Name as many as you can } How does each work? } Running time for each (sorting N items)? ◦ best ◦ worst ◦ average ◦ extra space requirements } Spend 10 minutes with a group of three, answering these questions. Then we will summarize Put list on board

} Some possible answers (C (Collect them on the boar ard) ◦ Bubble sort (Do Don't 't say ay th the b-wo word rd!) ◦ Insertion sort Li Like so sorting files s in manila fo fold lders ◦ Selection sort Se Sele lect the lar largest, t , then th the seco cond lar largest, … ◦ Merge sort Sp Split lit, r , recursively s sort, , me merge ◦ Binary tree sort In Insert all all in into to BST ST, th then in inOrder trav aversal al ◦ (Quicksort) Not Not so so elementary. We’ll do it in det detail – http://students.ceid.upatras.gr/~pirot/java/Quicksort/ ◦ (Heapsort) We We’ll ll als also do th this is one in in det detail ◦ (Shellsort) In Interesting va variation on on ins nsertion on so sort ◦ (Radix sort) Anoth An ther one one tha hat we we’ll consider r in some det detail Best, worst, average time? Extra space requirements?

http://www.xkcd.com/1185/ Stacksort connects to StackOverflow, searches for “sort a list”, and downloads and runs code snippets until the list is sorted.