Powering a number (a bit easier than the recursive mystery question - PowerPoint PPT Presentation

CS 3343 -- Spring 2006 Powering a number (a bit easier than the recursive mystery question on the homework) Problem: Compute a n , where n ∈ N . Naive algorithm: Θ ( n ). Divide-and-conquer algorithm: (recursive squaring) a n/ 2 ⋅ a n/ 2 if n is even; a n = Matrix Multiplication a ( n– 1) / 2 ⋅ a ( n– 1) / 2 ⋅ a if n is odd. Carola Wenk T ( n ) = T ( n /2) + Θ (1) ⇒ T ( n ) = Θ (log n ) . Slides courtesy of Charles Leiserson with small changes by Carola Wenk 2/16/06 CS 3343 Analysis of Algorithms 1 2/16/06 CS 3343 Analysis of Algorithms 2 Matrix multiplication Standard algorithm Input: A = [ a ij ], B = [ b ij ]. for i ← 1 to n i , j = 1, 2,… , n . Output: C = [ c ij ] = A ⋅ B . do for j ← 1 to n do c ij ← 0  c c c   a a a   b b b  L L L 11 12 1 n 11 12 1 n 11 12 1 n for k ← 1 to n       c c c a a a b b b L L L  21 22 2 n   21 22 2 n   21 22 2 n  = ⋅ do c ij ← c ij + a ik ⋅ b kj       M M O M M M O M M M O M       c c c a a a b b b  L   L   L  n 1 n 2 nn n 1 n 2 nn n 1 n 2 nn Running time = Θ ( n 3 ) n ∑ = ⋅ c a b ij ik kj = k 1 2/16/06 CS 3343 Analysis of Algorithms 3 2/16/06 CS 3343 Analysis of Algorithms 4 1

Divide-and-conquer algorithm Analysis of D&C algorithm I DEA : T ( n ) = 8 T ( n /2) + Θ ( n 2 ) n × n matrix = 2 × 2 matrix of ( n /2) ×( n /2) submatrices:  r s   a b   e f  work adding = ⋅ # submatrices       t u c d g h       submatrices submatrix size ⋅ C = A B n log ba = n log 2 8 = n 3 ⇒ C ASE 1 ⇒ T ( n ) = Θ ( n 3 ). r = a·e+b·g s = a·f+ b·h 8 recursive mults of ( n /2) ×( n /2) submatrices 4 adds of ( n /2) ×( n /2) submatrices No better than the ordinary algorithm. t = c·e+d·h u = c·f +d·g 2/16/06 CS 3343 Analysis of Algorithms 5 2/16/06 CS 3343 Analysis of Algorithms 6 Strassen’s idea Strassen’s idea • Multiply 2 × 2 matrices with only 7 recursive mults. • Multiply 2 × 2 matrices with only 7 recursive mults. P 1 = a ⋅ ( f – h ) P 1 = a ⋅ ( f – h ) r = P 5 + P 4 – P 2 + P 6 s = P 1 + P 2 P 2 = ( a + b ) ⋅ h P 2 = ( a + b ) ⋅ h = a ⋅ ( f – h ) + ( a + b ) ⋅ h s = P 1 + P 2 P 3 = ( c + d ) ⋅ e P 3 = ( c + d ) ⋅ e t = P 3 + P 4 = af – ah + ah + bh P 4 = d ⋅ ( g – e ) P 4 = d ⋅ ( g – e ) u = P 5 + P 1 – P 3 – P 7 = af + bh P 5 = ( a + d ) ⋅ ( e + h ) P 5 = ( a + d ) ⋅ ( e + h ) 7 mults, 18 adds/subs. 7 mults, 18 adds/subs. P 6 = ( b – d ) ⋅ ( g + h ) P 6 = ( b – d ) ⋅ ( g + h ) Note: No reliance on Note: No reliance on P 7 = ( a – c ) ⋅ ( e + f ) P 7 = ( a – c ) ⋅ ( e + f ) commutativity of mult! commutativity of mult! 2/16/06 CS 3343 Analysis of Algorithms 7 2/16/06 CS 3343 Analysis of Algorithms 8 2

Strassen’s algorithm Analysis of Strassen 1. Divide: Partition A and B into T ( n ) = 7 T ( n /2) + Θ ( n 2 ) ( n /2) × ( n /2) submatrices. Form P -terms n log ba = n log 2 7 ≈ n 2.81 ⇒ C ASE 1 ⇒ T ( n ) = Θ ( n log 7 ). to be multiplied using + and – . 2. Conquer: Perform 7 multiplications of The number 2.81 may not seem much smaller than ( n /2) × ( n /2) submatrices recursively. 3, but because the difference is in the exponent, the 3. Combine: Form C using + and – on impact on running time is significant. In fact, ( n /2) × ( n /2) submatrices. Strassen’s algorithm beats the ordinary algorithm on today’s machines for n ≥ 30 or so. T ( n ) = 7 T ( n /2) + Θ ( n 2 ) Best to date (of theoretical interest only): Θ ( n 2.376 L ). 2/16/06 CS 3343 Analysis of Algorithms 9 2/16/06 CS 3343 Analysis of Algorithms 10 3