Representing Big Integers Multiple-precision integers cant be stored - PDF document

Representing Big Integers Multiple-precision integers can’t be stored in a single machine word like an ‘int‘. Why are these important computationally? Example: 4391354067575026 represented as an array: [6, 2, 0, 5, 7, 5, 7, 6, 0, 4, 5, 3, 1, 9, 3, 4] in base B = 10 [242, 224, 71, 203, 233, 153, 15] in base B = 256 CS 355 (USNA) Unit 4 Spring 2012 1 / 33 Base of representation General form of a multiple-precision integer: d 0 + d 1 B + d 2 B 2 + d 3 B 3 + · · · + d n − 1 B n − 1 , Does the choice of base B matter? CS 355 (USNA) Unit 4 Spring 2012 2 / 33 Addition How would you add two n -digit integers? Remember, every digit is in a separate machine word. How big can the “carries” get? What if the inputs don’t have the same size? How fast is your method? CS 355 (USNA) Unit 4 Spring 2012 3 / 33

Standard Addition carry := 0 1 A := new array of length n+1 2 f o r i from 0 to n-1 3 A[i] := (X[i] + Y[i] + carry) mod B 4 carry := (X[i] + Y[i] + carry) / B 5 end f o r 6 A[n] := carry 7 return A 8 CS 355 (USNA) Unit 4 Spring 2012 4 / 33 Linear-time lower bounds Remember the Ω( n log n ) lower bound for comparison-based sorting? Much easier lower bounds exist for many problems! Linear lower bounds For any problem with input size n , where changing any part of the input could change the answer , any correct algorithm must take Ω( n ) time. What does this tell us about integer addition? CS 355 (USNA) Unit 4 Spring 2012 5 / 33 Multiplication Let’s remember how we multiplied multi-digit integers in grade school. CS 355 (USNA) Unit 4 Spring 2012 6 / 33

Standard multiplication A := new array of length (2*n) 1 A := [0 0 .. 0] 2 T := new array of length (n+1) 3 f o r i from 0 to n-1 4 −− s e t T to X t i m e s Y [ i ] −− 5 carry := 0 6 f o r j from 0 to n-1 7 T[j] := (X[j] * Y[i] + carry) mod B 8 carry := (X[j] * Y[i] + carry) / B 9 end f o r 10 T[n] := carry 11 −− Add T to A , the r u n n i n g sum −− 12 A[i..i+n] := add(A[i..i+n-1], T[0..n]) 13 end f o r 14 return A 15 CS 355 (USNA) Unit 4 Spring 2012 7 / 33 Divide and Conquer Maybe a divide-and-conquer approach will yield a faster multiplication algorithm. Let’s split the digit-lists in half. Let m = ⌊ n 2 ⌋ and write x = x 0 + B m x 1 and y = y 0 + B m y 1 . Then we multiply xy = x 0 y 0 + x 0 y 1 B m + x 1 y 0 B m + x 1 y 1 B 2 m . For example, if x = 7407 and y = 2915, then we get Integers | Array representation -----------|--------------------- x = 7407 | X = [7, 0, 4, 7] y = 2915 | Y = [5, 1, 9, 2] x0 = 07 | X0 = [7, 0] x1 = 74 | X1 = [4, 7] y0 = 15 | Y0 = [5, 1] y1 = 29 | Y1 = [9, 2] CS 355 (USNA) Unit 4 Spring 2012 8 / 33 Recurrences for Multiplication Standard multiplication has running time � 1 , n = 1 T ( n ) = n + T ( n − 1) , n ≥ 2 The divide-and-conquer way has running time � 1 , n = 1 T ( n ) = n + 4 T ( n 2 ) , n ≥ 2 CS 355 (USNA) Unit 4 Spring 2012 9 / 33

Karatsuba’s Algorithm The equation: ( x 0 + x 1 B m )( y 0 + y 1 B m ) = x 0 y 0 + x 1 y 1 B 2 m +(( x 0 + x 1 )( y 0 + y 1 ) − x 0 y 0 − x 1 y 1 ) B m leads to an algorithm: 1 Compute two sums: u = x 0 + x 1 and v = y 0 + y 1 . 2 Compute three m -digit products: x 0 y 0 , x 1 y 1 , and uv . 3 Sum them up and multiply by powers of B to get the answer: xy = x 0 y 0 + x 1 y 1 B 2 m + ( uv − x 0 y 0 − x 1 y 1 ) B m CS 355 (USNA) Unit 4 Spring 2012 10 / 33 Karatsuba Example x = 7407 = 7 + 74*100 y = 2915 = 15 + 29*100 u = x0 + x1 = 7 + 74 = 81 v = y0 + y1 = 15 + 29 = 44 x0*y0 = 7*15 = 105 x1*x1 = 74*29 = 2146 u*v = 81*44 = 3564 x*y = 105 + 2146*10000 + (3564 - 105 - 2146)*100 = 105 + 1313 + 2146 = 21591405 CS 355 (USNA) Unit 4 Spring 2012 11 / 33 Analyzing Karatsuba � 1 , n ≤ 1 T ( n ) = n + 3 T ( n 2 ) , n ≥ 2 Crucial difference : The coefficient of T ( n 2 ). CS 355 (USNA) Unit 4 Spring 2012 12 / 33

Beyond Karatsuba History Lesson : 1962: Karatsuba: O ( n 1 . 59 ) 1963: Toom & Cook: O ( n 1 . 47 ), O ( n 1+ ǫ ) 1971: Sch¨ onhage & Strassen: O ( n (log n )(log log n )) urer: O ( n (log n )2 log ∗ n ) 2007: F¨ Lots of work to do in algorithms! CS 355 (USNA) Unit 4 Spring 2012 13 / 33 Recurrences Algorithm Recurrence Asymptotic big-Θ BinarySearch 1 + T ( n / 2) log n LinearSearch 1 + T ( n − 1) n MergeSort (space) n + T ( n / 2) n MergeSort (time) n + 2 T ( n / 2) n log n n lg 3 KaratsubaMul n + 3 T ( n / 2) n 2 SelectionSort n + T ( n − 1) n 2 StandardMul n + 4 T ( n / 2) CS 355 (USNA) Unit 4 Spring 2012 14 / 33 Master Method A T ( n ) = aT ( n b ) + n c (log n ) d Write e = log b a = lg a lg b Three cases: 1 c = e . Then T ( n ) ∈ Θ( n c (log n ) d +1 ). 2 c < e . Then T ( n ) ∈ Θ( n e ) = Θ( n log b a ). 3 c > e . Then T ( n ) ∈ Θ( n c (log n ) d ). CS 355 (USNA) Unit 4 Spring 2012 15 / 33

Master Method B T ( n ) = aT ( n − b ) + n c (log n ) d Two cases: 1 a = 1 . Then T ( n ) ∈ Θ( n c +1 (log n ) d ). 2 a > 1 . Then T ( n ) ∈ Θ( e n ), where e is the positive constant a 1 / b . CS 355 (USNA) Unit 4 Spring 2012 16 / 33 Matrix Multiplication Review: Dimensions = number of rows and columns. Multiplication of 4 × 3 and 4 × 2 matrices:  7 1 2   28 9   2 0  6 2 8 56 30     6 3 =       9 6 3 66 27     4 3 1 1 4 24 15 A B = AB Middle dimensions must match. Running time : CS 355 (USNA) Unit 4 Spring 2012 17 / 33 Divide and Conquer Matrix Multiplication � S � � W � SW + TY T X � SX + TZ � = U V Y Z UW + VY UX + VZ Is this faster? CS 355 (USNA) Unit 4 Spring 2012 18 / 33

Strassen’s Algorithm Step 1: Seven products P 1 = S ( X − Z ) P 5 =( S + V )( W + Z ) P 2 =( S + T ) Z P 6 =( T − V )( Y + Z ) P 3 =( U + V ) W P 7 =( S − U )( W + X ) P 4 = V ( Y − W ) Step 2: Add and subtract � S � � W � P 5 + P 4 − P 2 + P 6 � � T X P 1 + P 2 = U V Y Z P 3 + P 4 P 1 + P 5 − P 3 − P 7 CS 355 (USNA) Unit 4 Spring 2012 19 / 33 Fibonacci Here’s a basic algorithm to compute f n : fib(n) Input: Non-negative integer n Output: f n i f n <= 1 then return n 1 e l s e return fib(n-1) + fib(n-2) 2 Is this fast? CS 355 (USNA) Unit 4 Spring 2012 20 / 33 Recursion tree for fib(6) fib(1) fib(2) fib(0) fib(3) fib(1) fib(1) fib(4) fib(2) fib(0) fib(2) fib(1) fib(5) fib(3) fib(1) fib(0) fib(6) fib(2) fib(1) fib(4) fib(3) fib(1) fib(0) fib(2) fib(1) fib(0) CS 355 (USNA) Unit 4 Spring 2012 21 / 33

Memoization How to avoid repeated, identical function calls ? Memoization means saving the results of calls in a table: fibmemo(n) Input: Non-negative integer n Output: f n i f T[n] i s unset then 1 i f n <= 1 then T[n] := n 2 e l s e T[n] := fibmemo(n-1) + fibmemo(n-2) 3 end i f 4 return T[n] 5 See the original function? CS 355 (USNA) Unit 4 Spring 2012 22 / 33 Recursion tree for fibmemo(6) fibmemo(6) fibmemo(5) fibmemo(4) fibmemo(4) fibmemo(3) fibmemo(3) fibmemo(2) fibmemo(2) fibmemo(1) fibmemo(1) fibmemo(0) CS 355 (USNA) Unit 4 Spring 2012 23 / 33 Cost of Memoization How should the table T be implemented? Analysis CS 355 (USNA) Unit 4 Spring 2012 24 / 33

Matrix Chain Multiplication Problem Given n matrices A 1 , A 2 , . . . , A n , find the best order of operations to compute the product A 1 A 2 · · · A n . Matrix multiplication is associative but not commutative. In summary: where should we put the parentheses? CS 355 (USNA) Unit 4 Spring 2012 25 / 33 Example  6 5 4   4 9  8 8 5 � 2 1 6       1 5 6 4 5 � 4 4 4     9 7 ∗ ∗     8 0 9 1 8 4 0 7 0     0 9     6 4 2   2 0 1 7 5 X Y Z 5 × 2 2 × 6 6 × 3 CS 355 (USNA) Unit 4 Spring 2012 26 / 33 Computing minimal mults Idea : Figure out the final multiplication, then use recursion to do the rest. mm(D) Input: Dimensions array D of length n + 1 Output: Least number of mults to compute the matrix chain product i f n = 1 then return 1 0 e l s e 2 fewest := infinity −− ( j u s t a p l a c e h o l d e r ) −− 3 f o r i from 1 to n-1 do 4 t := mm(D[0..i]) + D[0]*D[i]*D[n] + mm(D[i..n]) 5 i f then 6 t < fewest fewest := t end f o r 7 return fewest 8 end i f 9 CS 355 (USNA) Unit 4 Spring 2012 27 / 33

Analyzing mm(D) � 1 , n = 1 T ( n ) = n + � n − 1 i =1 ( T ( i ) + T ( n − i )) , n ≥ 2 CS 355 (USNA) Unit 4 Spring 2012 28 / 33 Memoized minimal mults Let’s use our general tool for avoiding repeated recursive calls: mmm(D) Input: Dimensions array D of length n + 1 Output: Least number of mults to compute the matrix chain product 1 i f T[D] i s unset then i f n = 1 then T[D] := 0 2 3 e l s e T[D] := infinity −− ( j u s t a p l a c e h o l d e r ) −− 4 5 f o r i from 1 to n-1 do 6 t := mmm(D[0..i]) + D[0]*D[i]*D[n] + mmm(D[i..n]) 7 i f t < T[D] then T[D] := t 8 end f o r end i f 9 10 end i f return T[D] 11 CS 355 (USNA) Unit 4 Spring 2012 29 / 33 Analyzing mmm(D) Cost of each call, not counting recursion: Total number of recursive calls: CS 355 (USNA) Unit 4 Spring 2012 30 / 33