More Divide & Conquer Carola Wenk Slides courtesy of Charles - PowerPoint PPT Presentation

CS 3343 – Fall 2011 More Divide & Conquer Carola Wenk Slides courtesy of Charles Leiserson with small Slides courtesy of Charles Leiserson with small changes by Carola Wenk 9/20/11 CS 3343 Analysis of Algorithms 1

Powering a number Powering a number Problem: Compute a n where n ∈ N Problem: Compute a n , where n ∈ N . Naive algorithm: Θ ( n ). Divide-and-conquer algorithm: (recursive squaring) a n/ 2 ⋅ a n/ 2 if n is even; a n = a ( n– 1) / 2 ⋅ a ( n– 1) / 2 ⋅ a if n is odd. T ( n ) = T ( n /2) + Θ (1) ⇒ T ( n ) = Θ (log n ) . ( ) ( ) ( ) ( ) ( g ) 9/20/11 CS 3343 Analysis of Algorithms 2

Fibonacci numbers Fibonacci numbers Recursive definition: Recursive definition: 0 if n = 0; 1 1 if n = 1; if n = 1; F n = F = F n– 1 + F n –2 if n ≥ 2. 8 13 21 34 L 0 1 1 2 3 5 Naive recursive algorithm: Ω ( φ n ) Ω ( φ ) N i i l i h ( + (exponential time), where φ = 1 5 ) / 2 is the golden ratio . ld i i h 9/20/11 CS 3343 Analysis of Algorithms 3

Computing Fibonacci numbers Naive recursive squaring: Naive recursive squaring: F n = φ n / 5 rounded to the nearest integer. • Recursive squaring: Θ (log n ) time • Recursive squaring: Θ (log n ) time. • This method is unreliable, since floating-point arithmetic is prone to round off errors arithmetic is prone to round-off errors. Bottom-up (one-dimensional dynamic programming) : • Compute F 0 , F 1 , F 2 , …, F n in order, forming each number by summing the two previous. • Running time: Θ ( n ). 9/20/11 CS 3343 Analysis of Algorithms 4

Convex Hull Convex Hull • Given a set of pins on a pinboard • Given a set of pins on a pinboard 4 4 5 3 • And a rubber band around them • How does the rubber band look 6 when it snaps tight? 2 • We represent convex hull as the 1 0 sequence of points on the convex sequence of points on the convex hull polygon, in counter-clockwise order. 9/20/11 CS 3343 Analysis of Algorithms 5

Convex Hull: Divide & Conquer Convex Hull: Divide & Conquer • Preprocessing: sort the points by x- coordinate • Divide the set of points into two sets A and B : sets A and B : • A contains the left ⎣ n/2 ⎦ points, • B contains the right ⎡ n/2 ⎤ points • B contains the right ⎡ n/2 ⎤ points • Recursively compute the convex hull of A A A B B • Recursively compute the convex hull of B • Merge the two convex hulls 9/20/11 CS 3343 Analysis of Algorithms 6

Merging Merging • Find upper and lower tangent • With those tangents the convex hull of A ∪ B can be computed from the convex hulls of A and the convex hull convex hulls of A and the convex hull of B in O(n) linear time A A B B 9/20/11 CS 3343 Analysis of Algorithms 7

Finding the lower tangent Finding the lower tangent 3 a = rightmost point of A 4=b b = leftmost point of B 4 2 while T=ab not lower tangent to both 3 3 convex hulls of A and B do{ convex hulls of A and B do{ 5 5 5 a=2 while T not lower tangent to 6 1 convex hull of A do{ 7 7 a=a-1 1 1 } 0 0 while T not lower tangent to convex hull of B do{ convex hull of B do{ A A B B b=b+1 } right turn or can be checked } } l ft t left turn? ? in constant time 9/20/11 CS 3343 Analysis of Algorithms 8

Convex Hull: Runtime Convex Hull: Runtime • Preprocessing: sort the points by x- O(n log n) just once O(n log n) just once coordinate • Divide the set of points into two O(1) ( ) sets A and B : sets A and B : • A contains the left ⎣ n/2 ⎦ points, • B contains the right ⎡ n/2 ⎤ points • B contains the right ⎡ n/2 ⎤ points • Recursively compute the convex T(n/2) hull of A • Recursively compute the convex T(n/2) hull of B • Merge the two convex hulls O(n) 9/20/11 CS 3343 Analysis of Algorithms 9

Convex Hull: Runtime Convex Hull: Runtime • Runtime Recurrence: T(n) = 2 T(n/2) + dn • Solves to T(n) = Θ (n log n) 9/20/11 CS 3343 Analysis of Algorithms 10

Matrix multiplication Matrix multiplication Input: Input: A = [ a ] B = [ b ] A = [ a ij ], B = [ b ij ]. i , j = 1, 2,… , n . Output: C = [ c ij ] = A ⋅ B . ⎡ ⎤ ⎡ ⎤ ⎡ ⎤ L L L c c c a a a b b b 11 12 1 n 11 12 1 n 11 12 1 n ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ L L L c c c a a a b b b ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ 21 22 2 n = 21 22 2 n ⋅ 21 22 2 n ⎢ ⎢ ⎥ ⎥ ⎢ ⎢ ⎥ ⎥ ⎢ ⎢ ⎥ ⎥ M M O M M M O M M M O M ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ L ⎦ ⎣ L ⎦ ⎣ L ⎦ c c c a a a b b b n 1 n 2 nn n 1 n 2 nn n 1 n 2 nn n ∑ = ⋅ c a b ij j ik kj j = k 1 9/20/11 CS 3343 Analysis of Algorithms 11

Standard algorithm Standard algorithm for i ← 1 to n for i ← 1 to n do for j ← 1 to n do c ← 0 do c ij ← 0 for k ← 1 to n do c ← c + a do c ij ← c ij + a ik ⋅ b kj b Running time = Θ ( n 3 ) Running time Θ ( n ) 9/20/11 CS 3343 Analysis of Algorithms 12

Divide-and-conquer algorithm Divide-and-conquer algorithm I DEA : I DEA : n × n matrix = 2 × 2 matrix of ( n /2) ×( n /2) submatrices: ⎡ ⎡ ⎤ ⎤ ⎡ ⎡ ⎤ ⎤ ⎡ ⎡ ⎤ ⎤ r r s s a a b b e e f f = ⋅ ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ t u c d g h ⋅ C = A B r = a·e+b·g 8 recursive mults of ( n /2) ×( n /2) submatrices s = a·f+ b·h 4 adds of ( n /2) ×( n /2) submatrices t = c·e+d·g u = c·f +d·h 9/20/11 CS 3343 Analysis of Algorithms 13

Analysis of D&C algorithm Analysis of D&C algorithm T ( n ) = 8 T ( n /2) + Θ ( n 2 ) 8 T ( /2) + Θ ( 2 ) T ( ) work adding k ddi # submatrices # b submatrices submatrix size n log ba = n log 2 8 = n 3 ⇒ C ASE 1 ⇒ T ( n ) = Θ ( n 3 ). No better than the ordinary algorithm. 9/20/11 CS 3343 Analysis of Algorithms 14

Strassen’s idea Strassen s idea • Multiply 2 × 2 matrices with only 7 recursive mults. u t p y at ces w t o y 7 ecu s ve u ts. P 1 = a ⋅ ( f – h ) r = P 5 + P 4 – P 2 + P 6 P 2 = ( a + b ) ⋅ h s = P 1 + P 2 P + P P ( + b ) h P 3 = ( c + d ) ⋅ e t = P 3 + P 4 P 4 = d ⋅ ( g – e ) P d ( ) u = P 5 + P 1 – P 3 – P 7 P + P P P P 5 = ( a + d ) ⋅ ( e + h ) 7 mults 18 adds/subs 7 mults, 18 adds/subs. P 6 = ( b – d ) ⋅ ( g + h ) P ( b d ) ( + h ) Note: No reliance on P 7 = ( a – c ) ⋅ ( e + f ) commutativity of mult! commutativity of mult! 9/20/11 CS 3343 Analysis of Algorithms 15

Strassen’s idea Strassen s idea • Multiply 2 × 2 matrices with only 7 recursive mults. u t p y at ces w t o y 7 ecu s ve u ts. P 1 = a ⋅ ( f – h ) r = P 5 + P 4 – P 2 + P 6 P 2 = ( a + b ) ⋅ h = ( a + d )( e + h ) ( + d )( + h ) P ( + b ) h P 3 = ( c + d ) ⋅ e + d ( g – e ) – ( a + b ) h P 4 = d ⋅ ( g – e ) P d ( ) + ( b + ( b – d )( g + h ) d )( + h ) P 5 = ( a + d ) ⋅ ( e + h ) = ae + ah + de + dh P 6 = ( b – d ) ⋅ ( g + h ) P ( b d ) ( + h ) + d + dg –de – ah – bh d h bh P 7 = ( a – c ) ⋅ ( e + f ) + bg + bh – dg – dh = ae + bg + b 9/20/11 CS 3343 Analysis of Algorithms 16

Strassen’s algorithm Strassen s algorithm 1. Divide: Partition A and B into ( n /2) × ( n /2) submatrices. Form P -terms to be multiplied using + and – . 2. Conquer: Perform 7 multiplications of ( n /2) × ( n /2) submatrices recursively. 3. Combine: Form C using + and – on ( n /2) × ( n /2) submatrices. ( ) ( ) T ( n ) = 7 T ( n /2) + Θ ( n 2 ) 9/20/11 CS 3343 Analysis of Algorithms 17

Analysis of Strassen Analysis of Strassen T ( n ) = 7 T ( n /2) + Θ ( n 2 ) T ( n ) 7 T ( n /2) Θ ( n ) n log ba = n log 2 7 ≈ n 2.81 ⇒ C ASE 1 ⇒ T ( n ) = Θ ( n log 7 ). The number 2.81 may not seem much smaller than 3, but because the difference is in the exponent, the 3, bu because e d e e ce s e e po e , e impact on running time is significant. In fact, Strassen’s algorithm beats the ordinary algorithm g y g on today’s machines for n ≥ 30 or so. Best to date (of theoretical interest only): Θ ( n 2.376 L ). l ) Θ ( 2 376 ) B t t d t ( f th ti l i t t 9/20/11 CS 3343 Analysis of Algorithms 18

Conclusion Conclusion • Divide and conquer is just one of several Di id d i j f l powerful techniques for algorithm design. • Divide-and-conquer algorithms can be analyzed using recurrences and the master method (so practice this math). th d ( ti thi th) • Can lead to more efficient algorithms 9/20/11 CS 3343 Analysis of Algorithms 19