CS 473: Algorithms Ruta Mehta University of Illinois, - PowerPoint PPT Presentation

Representing Polynomials by Samples Let p be a polynomial of degree n − 1 . Pick n distinct samples x 0 , x 1 , x 2 , . . . , x n − 1 Let y 0 = p ( x 0 ) , y 1 = p ( x 1 ) , . . . , y n − 1 = p ( x n − 1 ) . Representation Polynomials represented by ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . , ( x n − 1 , y n − 1 ) . Is the above a valid representation? Ruta (UIUC) CS473 15 Spring 2018 15 / 55

Representing Polynomials by Samples Let p be a polynomial of degree n − 1 . Pick n distinct samples x 0 , x 1 , x 2 , . . . , x n − 1 Let y 0 = p ( x 0 ) , y 1 = p ( x 1 ) , . . . , y n − 1 = p ( x n − 1 ) . Representation Polynomials represented by ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . , ( x n − 1 , y n − 1 ) . Is the above a valid representation? Why do we use 2 n numbers instead of n numbers for coefficient and root representation? Ruta (UIUC) CS473 15 Spring 2018 15 / 55

Sample Representation Theorem Given a list { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . ( x n − 1 , y n − 1 ) } there is exactly one polynomial p of degree n − 1 such that p ( x j ) = y j for j = 0 , 1 , . . . , n − 1 . Ruta (UIUC) CS473 16 Spring 2018 16 / 55

Sample Representation Theorem Given a list { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . ( x n − 1 , y n − 1 ) } there is exactly one polynomial p of degree n − 1 such that p ( x j ) = y j for j = 0 , 1 , . . . , n − 1 . So representation is valid. Ruta (UIUC) CS473 16 Spring 2018 16 / 55

Sample Representation Theorem Given a list { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . ( x n − 1 , y n − 1 ) } there is exactly one polynomial p of degree n − 1 such that p ( x j ) = y j for j = 0 , 1 , . . . , n − 1 . So representation is valid. Can use same x 0 , x 1 , . . . , x n − 1 for all polynomials of degree n − 1 . No need to store them explicitly and hence need only n numbers y 0 , y 1 , . . . , y n − 1 . ( Ruta (UIUC) CS473 16 Spring 2018 16 / 55

Lagrange Interpolation Given ( x 0 , y 0 ) , . . . , ( x n − 1 , y n − 1 ) the following polynomial p satisfies the property that p ( x j ) = y j for j = 0 , 1 , 2 , . . . , n − 1 .   n − 1 y j � � p ( x ) = ( x − x k )   � k � = j ( x j − x k ) j =0 k � = j Ruta (UIUC) CS473 17 Spring 2018 17 / 55

Lagrange Interpolation Given ( x 0 , y 0 ) , . . . , ( x n − 1 , y n − 1 ) the following polynomial p satisfies the property that p ( x j ) = y j for j = 0 , 1 , 2 , . . . , n − 1 .   n − 1 y j � � p ( x ) = ( x − x k )   � k � = j ( x j − x k ) j =0 k � = j For n = 3 , p ( x ) = ( x − x 1 )( x − x 2 ) ( x − x 0 )( x − x 2 ) ( x − x 0 )( x − x 1 ) y 0 ( x 0 − x 1 )( x 0 − x 2 )+ y 1 ( x 1 − x 0 )( x 1 − x 2 )+ y 2 ( x 2 − x 0 )( x 2 − x 1 ) Ruta (UIUC) CS473 17 Spring 2018 17 / 55

Lagrange Interpolation Given ( x 0 , y 0 ) , . . . , ( x n − 1 , y n − 1 ) the following polynomial p satisfies the property that p ( x j ) = y j for j = 0 , 1 , 2 , . . . , n − 1 .   n − 1 y j � � p ( x ) = ( x − x k )   � k � = j ( x j − x k ) j =0 k � = j For n = 3 , p ( x ) = ( x − x 1 )( x − x 2 ) ( x − x 0 )( x − x 2 ) ( x − x 0 )( x − x 1 ) y 0 ( x 0 − x 1 )( x 0 − x 2 )+ y 1 ( x 1 − x 0 )( x 1 − x 2 )+ y 2 ( x 2 − x 0 )( x 2 − x 1 ) Easy to verify that p ( x j ) = y j ! Thus there exists one polynomial of degree n − 1 that interpolates the values ( x 0 , y 0 ) , . . . , ( x n − 1 , y n − 1 ) . Ruta (UIUC) CS473 17 Spring 2018 17 / 55

Lagrange Interpolation Given ( x 0 , y 0 ) , . . . , ( x n − 1 , y n − 1 ) there is a polynomial p ( x ) such that p ( x i ) = y i for 0 ≤ i < n . Can there be two distinct polynomials? Ruta (UIUC) CS473 18 Spring 2018 18 / 55

Lagrange Interpolation Given ( x 0 , y 0 ) , . . . , ( x n − 1 , y n − 1 ) there is a polynomial p ( x ) such that p ( x i ) = y i for 0 ≤ i < n . Can there be two distinct polynomials? No! Use Fundamental Theorem of Algebra to prove it — exercise. Ruta (UIUC) CS473 18 Spring 2018 18 / 55

Addition and Multiplication with Sample Representation Let a = { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . ( x n − 1 , y n − 1 ) } and b = { ( x 0 , y ′ 0 ) , ( x 1 , y ′ 1 ) , . . . ( x n − 1 , y ′ n − 1 ) } be two polynomials of degree n − 1 in sample representation. Ruta (UIUC) CS473 19 Spring 2018 19 / 55

Addition and Multiplication with Sample Representation Let a = { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . ( x n − 1 , y n − 1 ) } and b = { ( x 0 , y ′ 0 ) , ( x 1 , y ′ 1 ) , . . . ( x n − 1 , y ′ n − 1 ) } be two polynomials of degree n − 1 in sample representation. a + b can be represented by Ruta (UIUC) CS473 19 Spring 2018 19 / 55

Addition and Multiplication with Sample Representation Let a = { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . ( x n − 1 , y n − 1 ) } and b = { ( x 0 , y ′ 0 ) , ( x 1 , y ′ 1 ) , . . . ( x n − 1 , y ′ n − 1 ) } be two polynomials of degree n − 1 in sample representation. a + b can be represented by { ( x 0 , ( y 0 + y ′ 0 )) , ( x 1 , ( y 1 + y ′ 1 )) , . . . ( x n − 1 , ( y n − 1 + y ′ n − 1 )) } Thus, can be computed in O ( n ) time Ruta (UIUC) CS473 19 Spring 2018 19 / 55

Addition and Multiplication with Sample Representation Let a = { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . ( x n − 1 , y n − 1 ) } and b = { ( x 0 , y ′ 0 ) , ( x 1 , y ′ 1 ) , . . . ( x n − 1 , y ′ n − 1 ) } be two polynomials of degree n − 1 in sample representation. a + b can be represented by { ( x 0 , ( y 0 + y ′ 0 )) , ( x 1 , ( y 1 + y ′ 1 )) , . . . ( x n − 1 , ( y n − 1 + y ′ n − 1 )) } Thus, can be computed in O ( n ) time a · b can be evaluated at n samples Ruta (UIUC) CS473 19 Spring 2018 19 / 55

Addition and Multiplication with Sample Representation Let a = { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . ( x n − 1 , y n − 1 ) } and b = { ( x 0 , y ′ 0 ) , ( x 1 , y ′ 1 ) , . . . ( x n − 1 , y ′ n − 1 ) } be two polynomials of degree n − 1 in sample representation. a + b can be represented by { ( x 0 , ( y 0 + y ′ 0 )) , ( x 1 , ( y 1 + y ′ 1 )) , . . . ( x n − 1 , ( y n − 1 + y ′ n − 1 )) } Thus, can be computed in O ( n ) time a · b can be evaluated at n samples { ( x 0 , ( y 0 · y ′ 0 )) , ( x 1 , ( y 1 · y ′ 1 )) , . . . ( x n − 1 , ( y n − 1 · y ′ n − 1 )) } Can be computed in O ( n ) time. Ruta (UIUC) CS473 19 Spring 2018 19 / 55

Addition and Multiplication with Sample Representation Let a = { ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . ( x n − 1 , y n − 1 ) } and b = { ( x 0 , y ′ 0 ) , ( x 1 , y ′ 1 ) , . . . ( x n − 1 , y ′ n − 1 ) } be two polynomials of degree n − 1 in sample representation. a + b can be represented by { ( x 0 , ( y 0 + y ′ 0 )) , ( x 1 , ( y 1 + y ′ 1 )) , . . . ( x n − 1 , ( y n − 1 + y ′ n − 1 )) } Thus, can be computed in O ( n ) time a · b can be evaluated at n samples { ( x 0 , ( y 0 · y ′ 0 )) , ( x 1 , ( y 1 · y ′ 1 )) , . . . ( x n − 1 , ( y n − 1 · y ′ n − 1 )) } Can be computed in O ( n ) time. But what if p , q are given in coefficient form? Convolution requires p , q to be in coefficient form. Ruta (UIUC) CS473 19 Spring 2018 19 / 55

Recall Goal: given polynomials a = ( a 0 , . . . , a n − 1 ) and b = ( b 0 , . . . , b n − 1 ) in coefficient representation, compute a · b in coefficient form (convolution) . Ruta (UIUC) CS473 20 Spring 2018 20 / 55

Recall Goal: given polynomials a = ( a 0 , . . . , a n − 1 ) and b = ( b 0 , . . . , b n − 1 ) in coefficient representation, compute a · b in coefficient form (convolution) . Sample representation: Fix x 0 , . . . , x n − 1 . a ′ = ( x 0 , a ( x 0 )) , . . . , ( x n − 1 , a ( x n − 1 )) , similarly b ′ from b . Theorem. Unique degree ( n − 1) polynomial corresponding to any given n samples. Ruta (UIUC) CS473 20 Spring 2018 20 / 55

Recall Goal: given polynomials a = ( a 0 , . . . , a n − 1 ) and b = ( b 0 , . . . , b n − 1 ) in coefficient representation, compute a · b in coefficient form (convolution) . Sample representation: Fix x 0 , . . . , x n − 1 . a ′ = ( x 0 , a ( x 0 )) , . . . , ( x n − 1 , a ( x n − 1 )) , similarly b ′ from b . Theorem. Unique degree ( n − 1) polynomial corresponding to any given n samples. a ′ is a valid representation of a . a ′ · b ′ requires O ( n ) multiplications. Ruta (UIUC) CS473 20 Spring 2018 20 / 55

Recall Goal: given polynomials a = ( a 0 , . . . , a n − 1 ) and b = ( b 0 , . . . , b n − 1 ) in coefficient representation, compute a · b in coefficient form (convolution) . Sample representation: Fix x 0 , . . . , x n − 1 . a ′ = ( x 0 , a ( x 0 )) , . . . , ( x n − 1 , a ( x n − 1 )) , similarly b ′ from b . Theorem. Unique degree ( n − 1) polynomial corresponding to any given n samples. a ′ is a valid representation of a . a ′ · b ′ requires O ( n ) multiplications. Plan. Convert to sample representation. Multiply. Convert back to coefficient representation. Ruta (UIUC) CS473 20 Spring 2018 20 / 55

Coefficient representation to Sample representation Given a polynomial a as ( a 0 , a 1 , . . . , a n − 1 ) can we obtain a sample representation ( x 0 , y 0 ) , . . . , ( x n − 1 , y n − 1 ) quickly? Also can we invert the representation quickly? Ruta (UIUC) CS473 21 Spring 2018 21 / 55

Coefficient representation to Sample representation Given a polynomial a as ( a 0 , a 1 , . . . , a n − 1 ) can we obtain a sample representation ( x 0 , y 0 ) , . . . , ( x n − 1 , y n − 1 ) quickly? Also can we invert the representation quickly? Suppose we choose x 0 , x 1 , . . . , x n − 1 arbitrarily. Take O ( n ) time to evaluate y j = a ( x j ) given ( a 0 , . . . , a n − 1 ) . Total time is Ω( n 2 ) Inversion via Lagrange interpolation also Ω( n 2 ) Ruta (UIUC) CS473 21 Spring 2018 21 / 55

Key Idea Can choose x 0 , x 1 , . . . , x n − 1 carefully! Total time to evaluate a ( x 0 ) , a ( x 1 ) , . . . , a ( x n − 1 ) should be better than evaluating each separately. Ruta (UIUC) CS473 22 Spring 2018 22 / 55

Key Idea Can choose x 0 , x 1 , . . . , x n − 1 carefully! Total time to evaluate a ( x 0 ) , a ( x 1 ) , . . . , a ( x n − 1 ) should be better than evaluating each separately. How do we choose x 0 , x 1 , . . . , x n − 1 to save work? Ruta (UIUC) CS473 22 Spring 2018 22 / 55

A Simple Start a ( x ) = a 0 + a 1 x + a 2 x 2 + a 3 x 3 + . . . + a n − 1 x n − 1 Assume n is a power of 2 for rest of the discussion. Observation: ( − x ) 2 j = x 2 j . Can we exploit this? Ruta (UIUC) CS473 23 Spring 2018 23 / 55

A Simple Start a ( x ) = a 0 + a 1 x + a 2 x 2 + a 3 x 3 + . . . + a n − 1 x n − 1 Assume n is a power of 2 for rest of the discussion. Observation: ( − x ) 2 j = x 2 j . Can we exploit this? Example 3+4 x +6 x 2 +2 x 3 + x 4 +10 x 5 = (3+6 x 2 + x 4 )+ x (4+2 x 2 +10 x 4 ) Ruta (UIUC) CS473 23 Spring 2018 23 / 55

A Simple Start a ( x ) = a 0 + a 1 x + a 2 x 2 + a 3 x 3 + . . . + a n − 1 x n − 1 Assume n is a power of 2 for rest of the discussion. Observation: ( − x ) 2 j = x 2 j . Can we exploit this? Example 3+4 x +6 x 2 +2 x 3 + x 4 +10 x 5 = (3+6 x 2 + x 4 )+ x (4+2 x 2 +10 x 4 ) a ( c ) = (3 + 6 c 2 + c 4 ) + c (4 + 2 c 2 + 10 c 4 ) Ruta (UIUC) CS473 23 Spring 2018 23 / 55

A Simple Start a ( x ) = a 0 + a 1 x + a 2 x 2 + a 3 x 3 + . . . + a n − 1 x n − 1 Assume n is a power of 2 for rest of the discussion. Observation: ( − x ) 2 j = x 2 j . Can we exploit this? Example 3+4 x +6 x 2 +2 x 3 + x 4 +10 x 5 = (3+6 x 2 + x 4 )+ x (4+2 x 2 +10 x 4 ) a ( c ) = (3 + 6 c 2 + c 4 ) + c (4 + 2 c 2 + 10 c 4 ) a ( − c ) = Ruta (UIUC) CS473 23 Spring 2018 23 / 55

A Simple Start a ( x ) = a 0 + a 1 x + a 2 x 2 + a 3 x 3 + . . . + a n − 1 x n − 1 Assume n is a power of 2 for rest of the discussion. Observation: ( − x ) 2 j = x 2 j . Can we exploit this? Example 3+4 x +6 x 2 +2 x 3 + x 4 +10 x 5 = (3+6 x 2 + x 4 )+ x (4+2 x 2 +10 x 4 ) a ( c ) = (3 + 6 c 2 + c 4 ) + c (4 + 2 c 2 + 10 c 4 ) a ( − c ) = (3 + 6 c 2 + c 4 ) − c (4 + 2 c 2 + 10 c 4 ) Ruta (UIUC) CS473 23 Spring 2018 23 / 55

Odd and Even Decomposition Let a = ( a 0 , a 1 , . . . a n − 1 ) be a polynomial. Let a odd = ( a 1 , a 3 , a 5 , . . . ) be the ( n / 2 − 1) degree polynomial defined by the odd coefficients; so a odd ( x ) = a 1 + a 3 x + a 5 x 2 + · · · Ruta (UIUC) CS473 24 Spring 2018 24 / 55

Odd and Even Decomposition Let a = ( a 0 , a 1 , . . . a n − 1 ) be a polynomial. Let a odd = ( a 1 , a 3 , a 5 , . . . ) be the ( n / 2 − 1) degree polynomial defined by the odd coefficients; so a odd ( x ) = a 1 + a 3 x + a 5 x 2 + · · · Similarly, let a even ( x ) = a 0 + a 2 x + . . . be the ( n / 2 − 1) degree polynomial defined by the even coefficients. Ruta (UIUC) CS473 24 Spring 2018 24 / 55

Odd and Even Decomposition Let a = ( a 0 , a 1 , . . . a n − 1 ) be a polynomial. Let a odd = ( a 1 , a 3 , a 5 , . . . ) be the ( n / 2 − 1) degree polynomial defined by the odd coefficients; so a odd ( x ) = a 1 + a 3 x + a 5 x 2 + · · · Similarly, let a even ( x ) = a 0 + a 2 x + . . . be the ( n / 2 − 1) degree polynomial defined by the even coefficients. Observe a ( x ) = a even ( x 2 ) + xa odd ( x 2 ) Thus, evaluating a at x can be reduced to evaluating lower degree polynomials plus constantly many arithmetic operations. Ruta (UIUC) CS473 24 Spring 2018 24 / 55

Exploiting Odd-Even Decomposition a ( x ) = a even ( x 2 ) + xa odd ( x 2 ) Choose n samples x 0 , x 1 , x 2 , . . . , x n / 2 − 1 , − x 0 , − x 1 , . . . , − x n / 2 − 1 Evaluate a even and a odd at x 2 0 , x 2 1 , x 2 2 , . . . , x 2 n / 2 − 1 . Ruta (UIUC) CS473 25 Spring 2018 25 / 55

Exploiting Odd-Even Decomposition a ( x ) = a even ( x 2 ) + xa odd ( x 2 ) Choose n samples x 0 , x 1 , x 2 , . . . , x n / 2 − 1 , − x 0 , − x 1 , . . . , − x n / 2 − 1 Evaluate a even and a odd at x 2 0 , x 2 1 , x 2 2 , . . . , x 2 n / 2 − 1 . For each i = 0 to ( n / 2 − 1) , evaluate a ( x i ) = a even ( x 2 i ) + x i a odd ( x 2 i ) a ( − x i ) = a even ( x 2 i ) − x i a odd ( x 2 i ) Ruta (UIUC) CS473 25 Spring 2018 25 / 55

Exploiting Odd-Even Decomposition a ( x ) = a even ( x 2 ) + xa odd ( x 2 ) Choose n samples x 0 , x 1 , x 2 , . . . , x n / 2 − 1 , − x 0 , − x 1 , . . . , − x n / 2 − 1 Evaluate a even and a odd at x 2 0 , x 2 1 , x 2 2 , . . . , x 2 n / 2 − 1 . For each i = 0 to ( n / 2 − 1) , evaluate a ( x i ) = a even ( x 2 i ) + x i a odd ( x 2 i ) a ( − x i ) = a even ( x 2 i ) − x i a odd ( x 2 i ) Total of O ( n ) work! Ruta (UIUC) CS473 25 Spring 2018 25 / 55

Exploiting Odd-Even Decomposition a ( x ) = a even ( x 2 ) + xa odd ( x 2 ) Choose n samples x 0 , x 1 , x 2 , . . . , x n / 2 − 1 , − x 0 , − x 1 , . . . , − x n / 2 − 1 Evaluate a even and a odd at x 2 0 , x 2 1 , x 2 2 , . . . , x 2 n / 2 − 1 . For each i = 0 to ( n / 2 − 1) , evaluate a ( x i ) = a even ( x 2 i ) + x i a odd ( x 2 i ) a ( − x i ) = a even ( x 2 i ) − x i a odd ( x 2 i ) Total of O ( n ) work! Suppose we can make this work recursively. Then T ( n ) = 2 T ( n / 2) + O ( n ) which implies T ( n ) = O ( n log n ) Ruta (UIUC) CS473 25 Spring 2018 25 / 55

Collapsible sets Definition Given a set X of numbers square ( X ) (for square of X ) is the set { x 2 | x ∈ X } . Ruta (UIUC) CS473 26 Spring 2018 26 / 55

Collapsible sets Definition Given a set X of numbers square ( X ) (for square of X ) is the set { x 2 | x ∈ X } . Definition A set X of n numbers is collapsible if square ( X ) ⊂ X and | square ( X ) | = n / 2 . Ruta (UIUC) CS473 26 Spring 2018 26 / 55

Collapsible sets Definition Given a set X of numbers square ( X ) (for square of X ) is the set { x 2 | x ∈ X } . Definition A set X of n numbers is collapsible if square ( X ) ⊂ X and | square ( X ) | = n / 2 . Definition A set X of n numbers (for n a power of 2 ) is recursively collapsible if n = 1 or if X is collapsible and square ( X ) is recursively collapsible. Ruta (UIUC) CS473 26 Spring 2018 26 / 55

Divide and Conquer assuming collapsible set Given a recursively collapsible set X of size n , compute sample representation of polynomial a of degree ( n − 1) as follows: SampleRepresentation ( a , X , n ) If n = 1 return a ( x 0 ) where X = { x 0 } Compute square ( X ) in O ( n ) time %note: | square ( X ) | = n / 2 Ruta (UIUC) CS473 27 Spring 2018 27 / 55

Divide and Conquer assuming collapsible set Given a recursively collapsible set X of size n , compute sample representation of polynomial a of degree ( n − 1) as follows: SampleRepresentation ( a , X , n ) If n = 1 return a ( x 0 ) where X = { x 0 } Compute square ( X ) in O ( n ) time %note: | square ( X ) | = n / 2 { y 0 , y 1 , . . . , y n / 2 − 1 } =SampleRepresentation( a o dd , square ( X ) , n / 2) { y ′ 0 , y ′ 1 , . . . , y ′ n / 2 − 1 } =SampleRepresentation( a e ven , square ( X ) , n / 2) Ruta (UIUC) CS473 27 Spring 2018 27 / 55

Divide and Conquer assuming collapsible set Given a recursively collapsible set X of size n , compute sample representation of polynomial a of degree ( n − 1) as follows: SampleRepresentation ( a , X , n ) If n = 1 return a ( x 0 ) where X = { x 0 } Compute square ( X ) in O ( n ) time %note: | square ( X ) | = n / 2 { y 0 , y 1 , . . . , y n / 2 − 1 } =SampleRepresentation( a o dd , square ( X ) , n / 2) { y ′ 0 , y ′ 1 , . . . , y ′ n / 2 − 1 } =SampleRepresentation( a e ven , square ( X ) , n / 2) For each i from 0 to ( n − 1) compute z i = a e ven ( x 2 i ) + x i a o dd ( x 2 i ) Return { z 0 , z 1 , . . . , z n − 1 } Ruta (UIUC) CS473 27 Spring 2018 27 / 55

Divide and Conquer assuming collapsible set Given a recursively collapsible set X of size n , compute sample representation of polynomial a of degree ( n − 1) as follows: SampleRepresentation ( a , X , n ) If n = 1 return a ( x 0 ) where X = { x 0 } Compute square ( X ) in O ( n ) time %note: | square ( X ) | = n / 2 { y 0 , y 1 , . . . , y n / 2 − 1 } =SampleRepresentation( a o dd , square ( X ) , n / 2) { y ′ 0 , y ′ 1 , . . . , y ′ n / 2 − 1 } =SampleRepresentation( a e ven , square ( X ) , n / 2) For each i from 0 to ( n − 1) compute z i = a e ven ( x 2 i ) + x i a o dd ( x 2 i ) Return { z 0 , z 1 , . . . , z n − 1 } Exercise: show that algorithm runs in O ( n log n ) time Ruta (UIUC) CS473 27 Spring 2018 27 / 55

Are there collapsible sets? n samples x 0 , x 1 , x 2 , . . . , x n / 2 − 1 , − x 0 , − x 1 , . . . , − x n / 2 − 1 Next step in recursion x 2 0 , x 2 1 , . . . , x 2 n / 2 − 1 Ruta (UIUC) CS473 28 Spring 2018 28 / 55

Are there collapsible sets? n samples x 0 , x 1 , x 2 , . . . , x n / 2 − 1 , − x 0 , − x 1 , . . . , − x n / 2 − 1 Next step in recursion x 2 0 , x 2 1 , . . . , x 2 n / 2 − 1 To continue recursion, we need { x 2 0 , x 2 1 , . . . , x 2 2 − 1 } = { z 0 , z 1 , . . . , z n 4 − 1 , − z 0 , − z 1 , . . . , − z n 4 − 1 } n Ruta (UIUC) CS473 28 Spring 2018 28 / 55

Are there collapsible sets? n samples x 0 , x 1 , x 2 , . . . , x n / 2 − 1 , − x 0 , − x 1 , . . . , − x n / 2 − 1 Next step in recursion x 2 0 , x 2 1 , . . . , x 2 n / 2 − 1 To continue recursion, we need { x 2 0 , x 2 1 , . . . , x 2 2 − 1 } = { z 0 , z 1 , . . . , z n 4 − 1 , − z 0 , − z 1 , . . . , − z n 4 − 1 } n n / 4 then x 0 = √− 1 x n / 4 That is If z 0 = x 2 0 and − z 0 = x 2 x 0 = ix n / 4 where i is the imaginary number. Ruta (UIUC) CS473 28 Spring 2018 28 / 55

Are there collapsible sets? n samples x 0 , x 1 , x 2 , . . . , x n / 2 − 1 , − x 0 , − x 1 , . . . , − x n / 2 − 1 Next step in recursion x 2 0 , x 2 1 , . . . , x 2 n / 2 − 1 To continue recursion, we need { x 2 0 , x 2 1 , . . . , x 2 2 − 1 } = { z 0 , z 1 , . . . , z n 4 − 1 , − z 0 , − z 1 , . . . , − z n 4 − 1 } n n / 4 then x 0 = √− 1 x n / 4 That is If z 0 = x 2 0 and − z 0 = x 2 x 0 = ix n / 4 where i is the imaginary number. Can continue recursion but need to go to complex numbers. Ruta (UIUC) CS473 28 Spring 2018 28 / 55

Complex Numbers Notation For the rest of lecture, i stands for √− 1 Definition Complex numbers are points lying in the complex plane represented as √ a 2 + b 2 e (arctan( b / a )) i Cartesian a + ib = Polar re θ i = r (cos θ + i sin θ ) Thus, e π i = − 1 and e 2 π i = 1 . Ruta (UIUC) CS473 29 Spring 2018 29 / 55

Power Series for Functions (Recall) What is e z when z is a real number? When z is a complex number? e z = 1 + z / 1! + z 2 / 2! + . . . + z j / j ! + . . . Therefore e i θ 1 + i θ/ 1! + ( i θ ) 2 / 2! + ( i θ ) 3 / 3! + . . . = (1 − θ 2 / 2! + θ 4 / 4! − . . . +) + i ( θ − θ 3 / 3! + . . . +) = = cos θ + i sin θ Ruta (UIUC) CS473 30 Spring 2018 30 / 55

Complex Roots of Unity What are the roots of the polynomial x k − 1 ? ( e 2 π i = 1 ) Clearly 1 is a root. Ruta (UIUC) CS473 31 Spring 2018 31 / 55

Complex Roots of Unity What are the roots of the polynomial x k − 1 ? ( e 2 π i = 1 ) Clearly 1 is a root. Suppose re θ i is a root then r k e k θ i = 1 which implies that r = 1 and k θ = 2 π ⇒ θ = 2 π/ k Ruta (UIUC) CS473 31 Spring 2018 31 / 55

Complex Roots of Unity What are the roots of the polynomial x k − 1 ? ( e 2 π i = 1 ) Clearly 1 is a root. Suppose re θ i is a root then r k e k θ i = 1 which implies that r = 1 and k θ = 2 π ⇒ θ = 2 π/ k k , . . . , ω k − 1 Let ω k = e 2 π i / k . The roots are 1 = ω 0 k , ω 2 where k ω j k = e 2 π ji / k . Ruta (UIUC) CS473 31 Spring 2018 31 / 55

Complex Roots of Unity What are the roots of the polynomial x k − 1 ? ( e 2 π i = 1 ) Clearly 1 is a root. Suppose re θ i is a root then r k e k θ i = 1 which implies that r = 1 and k θ = 2 π ⇒ θ = 2 π/ k k , . . . , ω k − 1 Let ω k = e 2 π i / k . The roots are 1 = ω 0 k , ω 2 where k ω j k = e 2 π ji / k . Proposition Let ω k be e 2 π i / k . The equation x k = 1 has k distinct complex roots k = e (2 π j ) i / k for j = 0 , 1 , . . . k − 1 given by ω j Proof. k ) k = ( e 2 π ji / k ) k = e 2 π ji = ( e 2 π i ) j = (1) j = 1 ( ω j Ruta (UIUC) CS473 31 Spring 2018 31 / 55

Roots of unity form a collapsible set Observation 1: ω j k = ω j mod k k Ruta (UIUC) CS473 32 Spring 2018 32 / 55

Roots of unity form a collapsible set Observation 1: ω j k = ω j mod k k Lemma Assume n is a power of 2 . The n ’th roots of unity are a recursively collapsible set. Proof. Let X n = { 1 , ω n , ω 2 n , . . . , ω n − 1 } . n Ruta (UIUC) CS473 32 Spring 2018 32 / 55

Roots of unity form a collapsible set Observation 1: ω j k = ω j mod k k Lemma Assume n is a power of 2 . The n ’th roots of unity are a recursively collapsible set. Proof. ) 2 = ω n +2 j Let X n = { 1 , ω n , ω 2 n , . . . , ω n − 1 } . ( ω n / 2+ j = ω 2 j n , n n n Ruta (UIUC) CS473 32 Spring 2018 32 / 55

Roots of unity form a collapsible set Observation 1: ω j k = ω j mod k k Lemma Assume n is a power of 2 . The n ’th roots of unity are a recursively collapsible set. Proof. ) 2 = ω n +2 j Let X n = { 1 , ω n , ω 2 n , . . . , ω n − 1 } . ( ω n / 2+ j = ω 2 j n , for n n n each j < n / 2 . Ruta (UIUC) CS473 32 Spring 2018 32 / 55

Roots of unity form a collapsible set Observation 1: ω j k = ω j mod k k Lemma Assume n is a power of 2 . The n ’th roots of unity are a recursively collapsible set. Proof. ) 2 = ω n +2 j Let X n = { 1 , ω n , ω 2 n , . . . , ω n − 1 } . ( ω n / 2+ j = ω 2 j n , for n n n each j < n / 2 . X 1 = { 1 } , X 2 = { 1 , − 1 } X 4 = { 1 , − 1 , i , − i } 1 X 8 = { 1 , − 1 , i , − i , 2 ( ± 1 ± i ) } √ Ruta (UIUC) CS473 32 Spring 2018 32 / 55

Discrete Fourier Transform Definition Given vector a = ( a 0 , a 1 , . . . , a n − 1 ) the Discrete Fourier Transform (DFT) of a is the vector a ′ = ( a ′ 0 , a ′ 1 , . . . , a ′ n − 1 ) where a ′ j = a ( ω j n ) for 0 ≤ j < n . a ′ is a sample representation of polynomial with coefficient reprentation a at n ’th roots of unity. Ruta (UIUC) CS473 33 Spring 2018 33 / 55

Discrete Fourier Transform Definition Given vector a = ( a 0 , a 1 , . . . , a n − 1 ) the Discrete Fourier Transform (DFT) of a is the vector a ′ = ( a ′ 0 , a ′ 1 , . . . , a ′ n − 1 ) where a ′ j = a ( ω j n ) for 0 ≤ j < n . a ′ is a sample representation of polynomial with coefficient reprentation a at n ’th roots of unity. We have shown that a ′ can be computed from a in O ( n log n ) time. This divide and conquer algorithm is called the Fast Fourier Transform (FFT). Ruta (UIUC) CS473 33 Spring 2018 33 / 55

Back to Convolutions and Polynomial Multiplication Convolutions (products) Compute convolution c = ( c 0 , c 1 , . . . , c 2 n − 2 ) of a = ( a 0 , a 1 , . . . a n − 1 ) and b = ( b 0 , b 1 , . . . b n − 1 ) Evaluate a and b at some n sample points. 1 Compute sample representation of product. That is 2 c ′ = ( a ′ 0 b ′ 0 , a ′ 1 b ′ 1 , . . . , a ′ n − 1 b ′ n − 1 ) . Compute coefficients of unique polynomial associated with 3 sample representation of product. That is compute c from c ′ . Ruta (UIUC) CS473 34 Spring 2018 34 / 55

Back to Convolutions and Polynomial Multiplication Convolutions (products) Compute convolution c = ( c 0 , c 1 , . . . , c 2 n − 2 ) of a = ( a 0 , a 1 , . . . a n − 1 ) and b = ( b 0 , b 1 , . . . b n − 1 ) Evaluate a and b at the n th roots of unity. 1 Compute sample representation of product. That is 2 c ′ = ( a ′ 0 b ′ 0 , a ′ 1 b ′ 1 , . . . , a ′ n − 1 b ′ n − 1 ) . Compute coefficients of unique polynomial associated with 3 sample representation of product. That is compute c from c ′ . Can we really compute c from c ′ ? Ruta (UIUC) CS473 34 Spring 2018 34 / 55

Back to Convolutions and Polynomial Multiplication Convolutions (products) Compute convolution c = ( c 0 , c 1 , . . . , c 2 n − 2 ) of a = ( a 0 , a 1 , . . . a n − 1 ) and b = ( b 0 , b 1 , . . . b n − 1 ) Evaluate a and b at the n th roots of unity. 1 Compute sample representation of product. That is 2 c ′ = ( a ′ 0 b ′ 0 , a ′ 1 b ′ 1 , . . . , a ′ n − 1 b ′ n − 1 ) . Compute coefficients of unique polynomial associated with 3 sample representation of product. That is compute c from c ′ . Can we really compute c from c ′ ? We only have n sample points and c ′ has 2 n − 1 coefficients! Ruta (UIUC) CS473 34 Spring 2018 34 / 55

Convolutions and Polynomial Multiplication Convolutions Compute convolution c = ( c 0 , c 1 , . . . , c 2 n − 2 ) of a = ( a 0 , a 1 , . . . a n − 1 ) and b = ( b 0 , b 1 , . . . b n − 1 ) Pad a with n zeroes to make it a (2 n − 1) degree polynomial 1 a = ( a 0 , a 1 , . . . , a n − 1 , a n , a n +1 , . . . , a 2 n − 1 ) . Similarly for b . Ruta (UIUC) CS473 35 Spring 2018 35 / 55

Convolutions and Polynomial Multiplication Convolutions Compute convolution c = ( c 0 , c 1 , . . . , c 2 n − 2 ) of a = ( a 0 , a 1 , . . . a n − 1 ) and b = ( b 0 , b 1 , . . . b n − 1 ) Pad a with n zeroes to make it a (2 n − 1) degree polynomial 1 a = ( a 0 , a 1 , . . . , a n − 1 , a n , a n +1 , . . . , a 2 n − 1 ) . Similarly for b . Compute values of a and b at the 2 n th roots of unity. 2 Compute sample representation of product. That is 3 c ′ = ( a ′ 0 b ′ 0 , a ′ 1 b ′ 1 , . . . , a ′ n − 1 b ′ n − 1 , . . . , a ′ 2 n − 1 b ′ 2 n − 1 ) . Compute coefficients of unique polynomial associated with 4 sample representation of product. That is compute c from c ′ . Ruta (UIUC) CS473 35 Spring 2018 35 / 55

Convolutions and Polynomial Multiplication Convolutions Compute convolution c = ( c 0 , c 1 , . . . , c 2 n − 2 ) of a = ( a 0 , a 1 , . . . a n − 1 ) and b = ( b 0 , b 1 , . . . b n − 1 ) Pad a with n zeroes to make it a (2 n − 1) degree polynomial 1 a = ( a 0 , a 1 , . . . , a n − 1 , a n , a n +1 , . . . , a 2 n − 1 ) . Similarly for b . Compute values of a and b at the 2 n th roots of unity. 2 Compute sample representation of product. That is 3 c ′ = ( a ′ 0 b ′ 0 , a ′ 1 b ′ 1 , . . . , a ′ n − 1 b ′ n − 1 , . . . , a ′ 2 n − 1 b ′ 2 n − 1 ) . Compute coefficients of unique polynomial associated with 4 sample representation of product. That is compute c from c ′ . Step 2 takes O ( n log n ) using divide and conquer algorithm Step 3 takes O ( n ) time Step 4? Ruta (UIUC) CS473 35 Spring 2018 35 / 55

Part II Inverse Fourier Transform Ruta (UIUC) CS473 36 Spring 2018 36 / 55

Inverse Fourier Transform Input Given the evaluation of a n − 1 -degree polynomial a on the n th roots of unity specified by vector a ′ Goal Compute the coefficients of a We saw that a ′ can be computed from a in O ( n log n ) time. Can we compute a from a ′ in O ( n log n ) time? Ruta (UIUC) CS473 37 Spring 2018 37 / 55

A Matrix Point of View a ( x ) = a 0 + a 1 x + · · · + a n − 1 x n − 1 a ′ 0 = a ( x 0 ) , a ′ 1 = a ( x 1 ) , . . . , a ′ n − 1 = a ( x n − 1 ) . x n − 1  x 2  1 . . . a ′ x 0  a 0    0 0 0 x n − 1 x 2 a ′ 1 x 1 . . . a 1   1 1 1      . . . .  . ... . . . . .    .  .   . . . . . .       =      x n − 1  a ′ x 2 1 . . . a j x j       j j j     . .  . . . .  ... .    .  . . . . .   . . . . .       x n − 1 a ′ a n − 1 x 2 1 x n − 1 . . . n − 1 n − 1 n − 1 Ruta (UIUC) CS473 38 Spring 2018 38 / 55

A Matrix Point of View a ( x ) = a 0 + a 1 x + · · · + a n − 1 x n − 1 Denote ω = ω 1 n = e 2 π/ n . Let x j = ω j a ′ 0 = a (1) , a ′ 1 = a ( ω ) , . . . , a ′ n − 1 = a ( ω n − 1 ) . a ′  1 1 1 . . . 1   a 0    0 ω 2 ω n − 1 a ′ 1 ω . . . a 1 1       . . . . . . ...      .  . . . . . . . . . . .       =       ω j ω 2 j ω j ( n − 1) a ′ 1 . . . a j       j       . . . . . . ... . . . . . .       . . . . . .       ω n − 1 ω 2( n − 1) ω ( n − 1)( n − 1) a ′ 1 . . . a n − 1 n − 1 Ruta (UIUC) CS473 39 Spring 2018 39 / 55

Inverting the Matrix − 1  a ′ 1 1 1 . . . 1  a 0     0 ω 2 ω n − 1 a ′ a 1 1 ω . . . 1       . . . . . . ...       . . . . . . . . . . . .       =       ω 2 j ω j ( n − 1) a ′ ω j a j 1 . . .       j       . . . . . . ... . . . . . .       . . . . . .       ω n − 1 ω 2( n − 1) ω ( n − 1)( n − 1) a ′ a n − 1 1 . . . n − 1 Ruta (UIUC) CS473 40 Spring 2018 40 / 55

Inverting the Matrix − 1 1 1 1 1 1 1 1 1  . . .   . . .  ω 2 ω n − 1 ω − 1 ω − 2 ω − ( n − 1) 1 1 ω . . . . . .         . . . . . . . . . . . . . .     . . . . . . . .     . . . . 1 . . . .     =     ω j ω 2 j ω j ( n − 1) ω − j ω − 2 j ω − j ( n − 1)  1   1  n . . . . . .         . . . . . . . . . . . . . .     . . . . . . . .     . . . . . . . .     ω n − 1 ω 2( n − 1) ω ( n − 1)( n − 1) ω − ( n − 1) ω − 2( n − 1) ω − ( n − 1)( n − 1) 1 1 . . . . . . Replace ω by ω − 1 which is also a root of unity! Since ω j = ω j mod n , we get ω − j = e − j 2 π/ n = ω ( n − j )2 π/ n . Ruta (UIUC) CS473 41 Spring 2018 41 / 55