Mat 2345 Discrete Math Recursion Practice MergeSort Week 12 - PowerPoint PPT Presentation

Mat 2345 — Discrete Math Week 12 Week 12 Mat 2345 — Discrete Math Recursion Practice MergeSort Week 12 Complexity Correctness Loop Invariants Fall 2013

Student Responsibilities — Week 12 Mat 2345 — Discrete Math Reading : Textbook, Section 4.4 & 4.5 Week 12 Assignments : Week 12 Sec 4.4 8, 10, 24, 28 Recursion Sec 4.5 2, 4, 7, 12 Practice MergeSort Attendance : Frostily Encouraged Complexity Correctness Loop Week 12 Overview Invariants Sec 4.4. Recursive Algorithms Sec 4.5. Program Correctness

Section 4.4 Recursive Algorithms Mat 2345 — Discrete Math A recursive procedure to find the max in a non–empty list. Week 12 Week 12 Recursion We will assume we have built–in functions: Practice Length() — which returns the number of elements in the list MergeSort Complexity Max() — which returns the larger of two values Correctness Listhead() — which returns the first element in a list Loop Invariants Note: Max() requires one comparison

Mat 2345 — Discrete procedure Maxlist(...list...){ Math // PRE: list is not empty Week 12 // POST: returns the largest element in list Week 12 // strip off list head and pass on the remainder Recursion Practice if Length(list) is 1 then MergeSort return Listhead(list) Complexity else Correctness return Max(Listhead(list), Loop Maxlist(remainder_of_list)) Invariants } What happens with the list { 29 } ? With the list { 3 , 8 , 5 } ?

How Many Comparisons? Mat 2345 — Discrete Math Week 12 The recurrence equation for the number of comparisons Week 12 required for a list of length n , C ( n ) is: Recursion Practice C (1) = 0 the initial condition MergeSort Complexity C ( n ) = 1 + C ( n − 1) the recurrence equation Correctness Loop So, C ( n ) ∈ O ( n ) as we would expect Invariants

A Variant of Maxlist() Mat 2345 — Discrete Math Assuming the list length is a power of 2, here is a variant of Week 12 Maxlist() using a Divide–and–Conquer approach. Week 12 Recursion Divide the list in half, and find the maximum of each half Practice MergeSort Find the Max() of the maximum of the two halves Complexity Correctness Loop Apply these steps to each list half recursively. Invariants What could the base case(s) be?

Maxlist2() Algorithm Mat 2345 — procedure Maxlist2(...list...){ Discrete Math // PRE: list is not empty Week 12 // POST: returns the largest element in list // Divide list into two lists, take the max of Week 12 // the two halves (recursively) Recursion Practice if Length(list) is 1 then MergeSort return Listhead(list) Complexity else Correctness a = Maxlist2(first half of list) Loop b = Maxlist2(second half of list) Invariants return Max(a, b) } What happens with the list { 29 , 7 } ? With the list { 3 , 8 , 5 , 7 } ?

How Many Comparisons in Maxlist2() ? Mat 2345 — Discrete Math There are two calls to Maxlist2() , each of which requires Week 12 C ( n 2 ) operations to find maximum. Week 12 Recursion One comparison is required by the Max() function Practice MergeSort Complexity The recurrence equation for the number of comparisons Correctness required for a list of length n , C ( n ), is: Loop Invariants C (1) = 0 the initial condition 2 C ( n c ( n ) = 2 ) + 1 the recurrence equation

Consider A Sampling Mat 2345 — C ( n ) = 2 C ( n Discrete n 2 ) + 1 Math Week 12 2 1 − 1 2 0 = 1 1 = Week 12 2 2 − 1 2 1 = 2 3 = Recursion 2 3 − 1 Practice 2 2 = 4 7 = MergeSort 2 4 − 1 2 3 = 8 15 = Complexity Correctness 2 5 − 1 2 4 = 16 31 = Loop . . Invariants . . . . 2 log( n )+1 − 1 2 log n = = 2 n − 1 ∈ O ( n ) n Thus, C ( n ) = 2 log( n )+1 − 1 ∈ O ( n )

3 n 2 + 5 n + 4 ∈ O ( n 2 ) Practice I: Prove Mat 2345 — Discrete Math Definition of Big–Oh : f ( n ) ∈ O ( g ( n )) if there exists positive Week 12 constants c and N 0 such that ∀ n ≥ N 0 we have f ( n ) ≤ cg ( n ) Week 12 Recursion We need to find c > 0 and N 0 > 0 such that: Practice MergeSort 3 n 2 + 5 n + 4 ≤ cn 2 ∀ n ≥ N 0 Complexity Correctness We note that Loop Invariants 3 n 2 + 5 n + 4 ≤ 3 n 2 + 5 n 2 + 4 n 2 , when n > 0 ≤ 12 n 2 and we can choose c = 12

Practice I, Cont. Mat 2345 — Discrete Math Week 12 3 n 2 + 5 n + 4 12 n 2 To find N 0 : = 9 n 2 − 5 n − 4 Week 12 0 = Recursion Practice MergeSort when n = 1, 9(1) 2 − 5(1) − 4 = 9 − 5 − 4 = 0 Complexity Correctness Loop Thus, 3 n 2 + 5 n + 4 ≤ 12 n 2 ∀ n ≥ 1, and Invariants therefore, 3 n 2 + 5 n + 4 ∈ O ( n 2 )

Mat 2345 — Discrete Math Practice II. Week 12 Given T ( n ) = 2 n − 1, prove that T ( n ) ∈ O ( n ) Week 12 Recursion Practice MergeSort Complexity Practice III. Correctness Prove that T ( n ) = 3 n + 2 if Loop � 2 Invariants n = 0 T ( n ) = 3 + T ( n − 1) n > 0

MergeSort Algorithm Mat 2345 — Discrete list MergeSort(list[1..n]){ Math // PRE: none Week 12 // POST: returns list[1..n] in sorted order // Functional dependency: Merge() Week 12 Recursion if n is 0 Practice return an empty list MergeSort else if n is 1 Complexity return list[1] Correctness else { Loop Invariants list A = MergeSort(list[1..n/2]) list B = MergeSort(list[n/2 + 1..n]) list C = Merge(A, B) return C } }

Time Complexity of MergeSort() Prove by induction that the time complexity of MergeSort() , Mat 2345 — Discrete T ( n ) ∈ O ( n log n ) Math Week 12 What we need to do: Week 12 Establish a Base Case for some small n Recursion Prove T ( k ) ≤ cf ( k ) → T (2 k ) ≤ cf (2 k ) Practice MergeSort In particular, we need to prove ∀ k ≥ N 0 that: Complexity Correctness T ( k ) ≤ ck log k → T (2 k ) ≤ c 2 k log(2 k ) Loop = c 2 k (log 2 + log k ) Invariants = c 2 k log k + c 2 k where k = 2 m for some m ≥ 0, wlog ∗ ∗ without loss of generality

Base Case Mat 2345 — Let n = 1. Discrete Math n log n = (1) log(1) = 1(0) = 0 Week 12 But, T ( n ) is always positive, so this is not a good base case. Try a larger number. Week 12 Recursion Let n = 2. Practice T (2) = Time to divide MergeSort + time to MergeSort halves Complexity Correctness + time to Merge Loop = 1 + 1 + 1 + 2 = 5 Invariants while n log n = 2 log 2 = 2(1) = 2 Can we find a constant c > 0 such that 5 ≤ 2 c ? 5 2 ≤ c , so 5 2 is a lower bound on c

Inductive Hypothesis Mat 2345 — Discrete Math Week 12 Week 12 Recursion Practice Assume for some arbitrary k ≥ 2 that T ( k ) ≤ ck log k MergeSort Complexity Correctness Loop Invariants

Inductive Step — Show T (2 k ) ≤ 2 ck log k + 2 ck Mat 2345 — Discrete Math Week 12 1 + T ( ⌈ 2 k 2 ⌉ ) + T ( ⌊ 2 k T (2 k ) ≤ 2 ⌋ ) + 2 k Week 12 Recursion ≤ T ( k ) + T ( k ) + 2 k + 1 Practice MergeSort ≤ 2 T ( k ) + 2 k + 1 Complexity ≤ 2( ck log k ) + 2 k + 1 Correctness Loop ≤ 2 ck log k + 2 k + 1 Invariants

Inductive Step, Cont. Mat 2345 — Discrete Now, can we find a c such that Math Week 12 2 ck log k + 2 k + 1 ≤ 2 ck log k + 2 ck Week 12 2 k + 1 ≤ 2 ck Recursion Practice 1 ≤ 2 ck − 2 k MergeSort 1 ≤ 2 k ( c − 1) Complexity Correctness Loop Since k ≥ 2 from base case, ( c − 1) ≥ 1 4 or c ≥ 5 Invariants 4 We had a lower bound of 5 2 , so we can choose c = 3. Thus, T ( n ) ∈ O ( n log n ) ∀ n ≥ 2.

More on Complexity Mat 2345 — Discrete Math Week 12 If an algorithm is composed of several parts, then its time complexity is the sum of the complexities of its parts. Week 12 Recursion Practice MergeSort We must be able to evaluate these summations . Complexity Correctness Loop Things become even more complicated when the algorithm Invariants contains loops, each iteration of which is a different complexity.

An Example: Suppose S n = � n i =1 i 2 Mat 2345 — Discrete ( n 2 + n ) n ( n +1) We saw � n ≤ n 2 , and is, in fact, i =1 i = = Math 2 2 Θ( n 2 ) Week 12 Week 12 i =1 i 2 ≤ � n i =1 n 2 = n 3 . So, we guess � n Maybe Recursion S n ∈ Θ( n 3 ). Practice MergeSort Complexity We can prove our guess correct, and find the minimum Correctness constant of difference between S n and n 3 by induction: Loop Invariants i =1 i 2 = an 3 + bn 2 + cn + d = P ( n ) Guess: � n i =1 i 2 − � n i =1 i 2 = ( n + 1) 2 Notice that � n +1

Mat 2345 — Discrete So, P ( n + 1) = P ( n ) + ( n + 1) 2 Math Week 12 Thus, Week 12 a ( n + 1) 3 + b ( n + 1) 2 + c ( n + 1) + d = Recursion an 3 + bn 2 + cn + d + ( n + 1) 2 Practice MergeSort Complexity a ( n 3 + 3 n 2 + 3 n + 1) + b ( n 2 + 2 n + 1) + cn + c + d = Correctness an 3 + bn 2 + cn + d + n 2 + 2 n + 1 Loop Invariants an 3 + 3 an 2 + 3 an + a + bn 2 + 2 bn + b + cn + c + d = an 3 + bn 2 + cn + d + n 2 + 2 n + 1

Mat 2345 — Hence: Discrete Math 3 an 2 + 3 an + a + 2 bn + b + c n 2 + 2 n + 1, Week 12 = or 3 an 2 + (3 a + 2 b ) n + ( a + b + c ) n 2 + 2 n + 1 = Week 12 Recursion Practice Since coefficients of the same power of n must be equal: MergeSort Complexity 3a = 1 (3a+2b) = 2 a + b + c = 1 Correctness 1 3( 1 3 + 1 1 a = 3 ) + 2b = 2 2 + c = 1 Loop 3 Invariants 1 - 1 3 - 1 2b = 1 c = 2 1 1 b = c = 2 6 And we can choose d = 0