General remarks Algorithms Algorithms Week 3 Oliver Oliver - PowerPoint PPT Presentation

CS 270 CS 270 General remarks Algorithms Algorithms Week 3 Oliver Oliver Kullmann Kullmann Divide-and- Divide-and- Conquer Conquer Solving Recurrences Merge Sort Merge Sort First we continue with an important example for Solving Solving Recurrences Recurrences Divide-and-Conquer, namely Merge Sort . Recursion Recursion Divide-and-Conquer 1 Trees Trees Then we present a basic tool for analysing algorithms by Master Master Theorem Theorem Solving Recurrences . Merge Sort Divide-and- Divide-and- Conquer Conquer We conclude by considering another example, namely Matrix Matrix Solving Recurrences 2 multiplication multiplication Matrix Multiplication . Tutorial Tutorial Recursion Trees Reading from CLRS for week 3 Master Theorem Chapter 4 Divide-and-Conquer 3 Matrix multiplication Tutorial 4 CS 270 CS 270 Another example: Merge-Sort Another example: Merge-Sort Algorithms Algorithms A sorting algorithm based on divide and conquer. The worst-case Oliver Oliver Kullmann Kullmann running time has a lower order of growth than insertion sort. Divide-and- Divide-and- Conquer Conquer Again we are dealing with subproblems of sorting subarrays Merge Sort Merge Sort Solving Solving A [ p . . q ] Initially, p = 1 and q = A . length , but these values Merge-Sort ( A , p , q ) Recurrences Recurrences Recursion Recursion change again as we recurse through subproblems. Trees Trees 1 if p < q / check for base case / Master Master Theorem Theorem 2 r = ⌊ ( p + q ) / 2 ⌋ / / divide To sort A [ p . . q ]: Divide-and- Divide-and- Conquer 3 Merge-Sort ( A , p , r ) / / conquer Conquer Matrix Matrix 4 Merge-Sort ( A , r +1 , q ) / conquer multiplication / multiplication Divide by splitting into two subarrays A [ p . . . r ] and Tutorial Tutorial 5 Merge ( A , p , r , q ) / combine / A [ r +1 . . . q ], where r is the halfway point of A [ p . . . q ]. Conquer by recursively sorting the two subarrays A [ p . . . r ] and A [ r +1 . . . q ]. Initial call: Merge-Sort ( A , 1 , A . length ) Combine by merging the two sorted subarrays A [ p . . . r ] and A [ r +1 . . . q ] to produce a single sorted subarray A [ p . . . q ]. The recursion bottoms out when the subarray has just 1 element, so that it is trivially sorted.

CS 270 CS 270 Merge Algorithms Algorithms Merge ( A , p , r , q ) Oliver Oliver Kullmann Kullmann 1 n 1 = r − p + 1 Divide-and- Divide-and- 2 n 2 = q − r Conquer Conquer Input: Array A and indices p , r , q such that Merge Sort 3 let L [1 . . n 1 +1] and R [1 . . n 2 +1] be new arrays Merge Sort Solving Solving 4 for i = 1 to n 1 Recurrences Recurrences p ≤ r < q Recursion 5 L [ i ] = A [ p + i − 1] Recursion Trees Trees Subarrays A [ p . . r ] and subarray A [ r +1 . . q ] are sorted. By Master Master 6 for j = 1 to n 2 Theorem Theorem the restriction on p , r , q neither subarray is empty. Divide-and- Divide-and- 7 R [ j ] = A [ r + j ] Conquer Conquer 8 L [ n 1 +1] = R [ n 2 +1] = ∞ Matrix Matrix Output: The two subarrays are merged into a single sorted multiplication multiplication 9 i = j = 1 Tutorial Tutorial subarray in A [ p . . q ]. 10 for k = p to q 11 if L [ i ] ≤ R [ j ] We implement is so that it takes Θ( n ) time, with 12 A [ k ] = L [ i ] n = q − p + 1 = the number of elements being merged. 13 i = i +1 14 else A [ k ] = R [ j ] 15 j = j +1 CS 270 CS 270 Analysis of Merge-Sort Analysing divide-and-conquer algorithms Algorithms Algorithms Oliver Oliver Recall the divide-and-conquer paradigm: Kullmann Kullmann Divide-and- Divide the problem into a number of subproblems that are Divide-and- Conquer Conquer The runtime T ( n ), where n = q − p +1 > 1, satisfies: smaller instances of the same problem. Merge Sort Merge Sort Solving Solving Conquer the subproblems by solving them recursively. Recurrences Recurrences T ( n ) = 2 T ( n / 2) + Θ( n ) . Recursion Recursion Base case: If the subproblem are small enough, just solve Trees Trees Master Master them by brute force. Theorem Theorem We will show that T ( n ) = Θ( n lg n ). Divide-and- Divide-and- Combine the subproblem solutions to give a solution to the Conquer Conquer It can be shown (see tutorial-section) that Ω( n lg n ) Matrix Matrix original problem. multiplication multiplication comparisons are necessary in the worst case to sort n Tutorial Tutorial We use recurrences to characterise the running time of a numbers for any comparison-based algorithm: this is thus divide-and-conquer algorithm. Solving the recurrence gives us an (asymptotic) lower bound on the problem. the asymptotic running time. Hence Merge-Sort is provably (asymptotically) optimal. A recurrence is a function defined in terms of one or more base cases, and itself, with smaller arguments

CS 270 CS 270 Examples for recurrences Main technical issues with recurrences Algorithms Algorithms Oliver Oliver Kullmann Kullmann � Floors and ceilings: The recurrence describing worst-case 1 if n = 1 T ( n ) = running time of Merge-Sort is really Divide-and- Divide-and- T ( n − 1) + 1 if n > 1 Conquer Conquer � Θ(1) if n = 1 Merge Sort Merge Sort T ( n ) = Solution: T ( n ) = n . Solving Solving T ( ⌈ n / 2 ⌉ ) + T ( ⌊ n / 2 ⌋ ) + Θ( n ) if n > 1 Recurrences Recurrences � 1 if n = 1 Recursion Recursion Exact vs. asymptotic functions Sometimes we are interested in Trees Trees T ( n ) = Master Master 2 T ( n / 2) + n if n > 1 Theorem the exact analysis of an algorithm (as for the Theorem Divide-and- Divide-and- Min-Max-Problem), at other times we are concerned with the Conquer Conquer Solution: T ( n ) ≈ n lg n + n . Matrix Matrix asymptotic analysis (as for the Sorting Problem). multiplication multiplication � 0 if n = 2 Tutorial Tutorial T ( n ) = Boundary conditions Running time on small inputs is bounded T ( √ n ) + 1 if n > 2 by a constant: T ( n ) = Θ(1) for small n . We usually do not mention this constant, as it typically doesn’t change the order Solution: T ( n ) ≈ lg lg n . of growth of T ( n ). Such constants only play a role if we are � 1 if n = 1 T ( n ) = interested in exact solutions. T ( n / 3) + T (2 n / 3) + n if n > 1 When we state and solve recurrences, we often omit floors, Solution: T ( n ) = Θ( n lg n ). ceilings, and boundary conditions, as they usually do not matter. CS 270 CS 270 Recursion trees (quadratic growth) Recursion trees (quasi-linear and linear growth) Algorithms Algorithms What about the “merge-sort” recurrence Oliver Oliver Draw the unfolding of the recurrence Kullmann Kullmann T ( n ) = n + 2 T ( n / 2) ? Divide-and- Divide-and- T ( n ) = n + 4 T ( n / 2) . Conquer Conquer Merge Sort Merge Sort Again the height of the tree is lg n . Solving Solving T ( n ) n Recurrences Recurrences However now the “workload” of each level is equal to n . Recursion Recursion Trees Trees T ( n T ( n 2 ) T ( n T ( n 2 ) 2 ) 2 ) Master Master Theorem Theorem So here we get · · · · · · · · · Divide-and- Divide-and- T ( n ) = Θ( n · lg n ) . ✲ Conquer Conquer n 2 0 n ✻ Matrix Matrix multiplication multiplication ✲ n n n n 2 1 n And what about the recurrence Tutorial Tutorial 2 2 2 2 ✲ lg n n n n n n n n n n n n n n n n n 2 2 n 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 T ( n ) = 1 + 2 T ( n / 2) ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ❄ ✲ . . . . . . . . . . . . . . . . Again the height of the tree is lg n . 2 lg n n Θ(2 lg n n ) The “workload” of the level is 1 , 2 , . . . , 2 lg n . Total: = Θ( n 2 ) So here we get We exploited that 1 + 2 + 4 + · · · + 2 k = 2 k +1 − 1 = Θ(2 k ). T ( n ) = Θ( n ) .