Kartsubas Algorithm and Linear Time Selection Lecture 11 February - PowerPoint PPT Presentation

CS 374: Algorithms & Models of Computation, Spring 2017 Kartsuba’s Algorithm and Linear Time Selection Lecture 11 February 23, 2017 Chandra Chekuri (UIUC) CS374 1 Spring 2017 1 / 34

Part I Fast Multiplication Chandra Chekuri (UIUC) CS374 2 Spring 2017 2 / 34

Multiplying Numbers Problem Given two n -digit numbers x and y , compute their product. Grade School Multiplication Compute “partial product” by multiplying each digit of y with x and adding the partial products. 3141 × 2718 25128 3141 21987 6282 8537238 Chandra Chekuri (UIUC) CS374 3 Spring 2017 3 / 34

Time Analysis of Grade School Multiplication Each partial product: Θ( n ) 1 Number of partial products: Θ( n ) 2 Addition of partial products: Θ( n 2 ) 3 Total time: Θ( n 2 ) 4 Chandra Chekuri (UIUC) CS374 4 Spring 2017 4 / 34

A Trick of Gauss Carl Friedrich Gauss: 1777–1855 “Prince of Mathematicians” Observation: Multiply two complex numbers: ( a + bi ) and ( c + di ) ( a + bi )( c + di ) = ac − bd + ( ad + bc ) i Chandra Chekuri (UIUC) CS374 5 Spring 2017 5 / 34

A Trick of Gauss Carl Friedrich Gauss: 1777–1855 “Prince of Mathematicians” Observation: Multiply two complex numbers: ( a + bi ) and ( c + di ) ( a + bi )( c + di ) = ac − bd + ( ad + bc ) i How many multiplications do we need? Chandra Chekuri (UIUC) CS374 5 Spring 2017 5 / 34

A Trick of Gauss Carl Friedrich Gauss: 1777–1855 “Prince of Mathematicians” Observation: Multiply two complex numbers: ( a + bi ) and ( c + di ) ( a + bi )( c + di ) = ac − bd + ( ad + bc ) i How many multiplications do we need? Only 3! If we do extra additions and subtractions. Compute ac , bd , ( a + b )( c + d ) . Then ( ad + bc ) = ( a + b )( c + d ) − ac − bd Chandra Chekuri (UIUC) CS374 5 Spring 2017 5 / 34

Divide and Conquer Assume n is a power of 2 for simplicity and numbers are in decimal. Split each number into two numbers with equal number of digits x = x n − 1 x n − 2 . . . x 0 and y = y n − 1 y n − 2 . . . y 0 1 x = x n − 1 . . . x n / 2 0 . . . 0 + x n / 2 − 1 . . . x 0 2 x = 10 n / 2 x L + x R where x L = x n − 1 . . . x n / 2 and 3 x R = x n / 2 − 1 . . . x 0 Similarly y = 10 n / 2 y L + y R where y L = y n − 1 . . . y n / 2 and 4 y R = y n / 2 − 1 . . . y 0 Chandra Chekuri (UIUC) CS374 6 Spring 2017 6 / 34

Example 1234 × 5678 = (100 × 12 + 34) × (100 × 56 + 78) = 10000 × 12 × 56 +100 × (12 × 78 + 34 × 56) +34 × 78 Chandra Chekuri (UIUC) CS374 7 Spring 2017 7 / 34

Divide and Conquer Assume n is a power of 2 for simplicity and numbers are in decimal. x = x n − 1 x n − 2 . . . x 0 and y = y n − 1 y n − 2 . . . y 0 1 x = 10 n / 2 x L + x R where x L = x n − 1 . . . x n / 2 and 2 x R = x n / 2 − 1 . . . x 0 y = 10 n / 2 y L + y R where y L = y n − 1 . . . y n / 2 and 3 y R = y n / 2 − 1 . . . y 0 Therefore xy = (10 n / 2 x L + x R )(10 n / 2 y L + y R ) = 10 n x L y L + 10 n / 2 ( x L y R + x R y L ) + x R y R Chandra Chekuri (UIUC) CS374 8 Spring 2017 8 / 34

Time Analysis xy = (10 n / 2 x L + x R )(10 n / 2 y L + y R ) = 10 n x L y L + 10 n / 2 ( x L y R + x R y L ) + x R y R 4 recursive multiplications of number of size n / 2 each plus 4 additions and left shifts (adding enough 0’s to the right) Chandra Chekuri (UIUC) CS374 9 Spring 2017 9 / 34

Time Analysis xy = (10 n / 2 x L + x R )(10 n / 2 y L + y R ) = 10 n x L y L + 10 n / 2 ( x L y R + x R y L ) + x R y R 4 recursive multiplications of number of size n / 2 each plus 4 additions and left shifts (adding enough 0’s to the right) T ( n ) = 4 T ( n / 2) + O ( n ) T (1) = O (1) Chandra Chekuri (UIUC) CS374 9 Spring 2017 9 / 34

Time Analysis xy = (10 n / 2 x L + x R )(10 n / 2 y L + y R ) = 10 n x L y L + 10 n / 2 ( x L y R + x R y L ) + x R y R 4 recursive multiplications of number of size n / 2 each plus 4 additions and left shifts (adding enough 0’s to the right) T ( n ) = 4 T ( n / 2) + O ( n ) T (1) = O (1) T ( n ) = Θ( n 2 ) . No better than grade school multiplication! Chandra Chekuri (UIUC) CS374 9 Spring 2017 9 / 34

Time Analysis xy = (10 n / 2 x L + x R )(10 n / 2 y L + y R ) = 10 n x L y L + 10 n / 2 ( x L y R + x R y L ) + x R y R 4 recursive multiplications of number of size n / 2 each plus 4 additions and left shifts (adding enough 0’s to the right) T ( n ) = 4 T ( n / 2) + O ( n ) T (1) = O (1) T ( n ) = Θ( n 2 ) . No better than grade school multiplication! Can we invoke Gauss’s trick here? Chandra Chekuri (UIUC) CS374 9 Spring 2017 9 / 34

Improving the Running Time xy = (10 n / 2 x L + x R )(10 n / 2 y L + y R ) = 10 n x L y L + 10 n / 2 ( x L y R + x R y L ) + x R y R Gauss trick: x L y R + x R y L = ( x L + x R )( y L + y R ) − x L y L − x R y R Chandra Chekuri (UIUC) CS374 10 Spring 2017 10 / 34

Improving the Running Time xy = (10 n / 2 x L + x R )(10 n / 2 y L + y R ) = 10 n x L y L + 10 n / 2 ( x L y R + x R y L ) + x R y R Gauss trick: x L y R + x R y L = ( x L + x R )( y L + y R ) − x L y L − x R y R Recursively compute only x L y L , x R y R , ( x L + x R )( y L + y R ) . Chandra Chekuri (UIUC) CS374 10 Spring 2017 10 / 34

Improving the Running Time xy = (10 n / 2 x L + x R )(10 n / 2 y L + y R ) = 10 n x L y L + 10 n / 2 ( x L y R + x R y L ) + x R y R Gauss trick: x L y R + x R y L = ( x L + x R )( y L + y R ) − x L y L − x R y R Recursively compute only x L y L , x R y R , ( x L + x R )( y L + y R ) . Time Analysis Running time is given by T ( n ) = 3 T ( n / 2) + O ( n ) T (1) = O (1) which means Chandra Chekuri (UIUC) CS374 10 Spring 2017 10 / 34

Improving the Running Time xy = (10 n / 2 x L + x R )(10 n / 2 y L + y R ) = 10 n x L y L + 10 n / 2 ( x L y R + x R y L ) + x R y R Gauss trick: x L y R + x R y L = ( x L + x R )( y L + y R ) − x L y L − x R y R Recursively compute only x L y L , x R y R , ( x L + x R )( y L + y R ) . Time Analysis Running time is given by T ( n ) = 3 T ( n / 2) + O ( n ) T (1) = O (1) which means T ( n ) = O ( n log 2 3 ) = O ( n 1 . 585 ) Chandra Chekuri (UIUC) CS374 10 Spring 2017 10 / 34

State of the Art Sch¨ onhage-Strassen 1971: O ( n log n log log n ) time using Fast-Fourier-Transform ( FFT ) urer 2007: O ( n log n 2 O (log ∗ n ) ) time Martin F¨ Conjecture There is an O ( n log n ) time algorithm. Chandra Chekuri (UIUC) CS374 11 Spring 2017 11 / 34

Analyzing the Recurrences Basic divide and conquer: T ( n ) = 4 T ( n / 2) + O ( n ) , 1 T (1) = 1 . Claim: T ( n ) = Θ( n 2 ) . Saving a multiplication: T ( n ) = 3 T ( n / 2) + O ( n ) , 2 T (1) = 1 . Claim: T ( n ) = Θ( n 1+log 1 . 5 ) Chandra Chekuri (UIUC) CS374 12 Spring 2017 12 / 34

Analyzing the Recurrences Basic divide and conquer: T ( n ) = 4 T ( n / 2) + O ( n ) , 1 T (1) = 1 . Claim: T ( n ) = Θ( n 2 ) . Saving a multiplication: T ( n ) = 3 T ( n / 2) + O ( n ) , 2 T (1) = 1 . Claim: T ( n ) = Θ( n 1+log 1 . 5 ) Use recursion tree method: In both cases, depth of recursion L = log n . 1 Work at depth i is 4 i n / 2 i and 3 i n / 2 i respectively: number of 2 children at depth i times the work at each child i =0 2 i and n � L Total work is therefore n � L i =0 (3 / 2) i 3 respectively. Chandra Chekuri (UIUC) CS374 12 Spring 2017 12 / 34

Recursion tree analysis Chandra Chekuri (UIUC) CS374 13 Spring 2017 13 / 34

Part II Selecting in Unsorted Lists Chandra Chekuri (UIUC) CS374 14 Spring 2017 14 / 34

Rank of element in an array A : an unsorted array of n integers Definition For 1 ≤ j ≤ n , element of rank j is the j ’th smallest element in A . 16 14 34 20 12 5 3 19 11 Unsorted array Ranks 5 4 2 1 3 6 9 8 7 Sort of array 3 5 11 12 14 16 19 20 34 Chandra Chekuri (UIUC) CS374 15 Spring 2017 15 / 34

Problem - Selection Input Unsorted array A of n integers and integer j Goal Find the j th smallest number in A ( rank j number) Median: j = ⌊ ( n + 1) / 2 ⌋ Chandra Chekuri (UIUC) CS374 16 Spring 2017 16 / 34

Problem - Selection Input Unsorted array A of n integers and integer j Goal Find the j th smallest number in A ( rank j number) Median: j = ⌊ ( n + 1) / 2 ⌋ Simplifying assumption for sake of notation: elements of A are distinct Chandra Chekuri (UIUC) CS374 16 Spring 2017 16 / 34

Algorithm I Sort the elements in A 1 Pick j th element in sorted order 2 Time taken = O ( n log n ) Chandra Chekuri (UIUC) CS374 17 Spring 2017 17 / 34

Algorithm I Sort the elements in A 1 Pick j th element in sorted order 2 Time taken = O ( n log n ) Do we need to sort? Is there an O ( n ) time algorithm? Chandra Chekuri (UIUC) CS374 17 Spring 2017 17 / 34

Algorithm II If j is small or n − j is small then Find j smallest/largest elements in A in O ( jn ) time. (How?) 1 Time to find median is O ( n 2 ) . 2 Chandra Chekuri (UIUC) CS374 18 Spring 2017 18 / 34

Divide and Conquer Approach Pick a pivot element a from A 1 Partition A based on a . 2 A less = { x ∈ A | x ≤ a } and A greater = { x ∈ A | x > a } | A less | = j : return a 3 | A less | > j : recursively find j th smallest element in A less 4 | A less | < j : recursively find k th smallest element in A greater 5 where k = j − | A less | . Chandra Chekuri (UIUC) CS374 19 Spring 2017 19 / 34

Example 16 14 34 20 12 5 3 19 11 Chandra Chekuri (UIUC) CS374 20 Spring 2017 20 / 34