Chapter 23 Fast Fourier Transform CS 573: Algorithms, Fall 2013 - PDF document

Chapter 23 Fast Fourier Transform CS 573: Algorithms, Fall 2013 November 7, 2013 23.1 Introduction 23.1.0.1 Multiplying polynomials quickly j =0 a j x j = a 0 + x ( a 1 + x ( a 2 + Definition 23.1.1. polynomial p ( x ) of degree n :a function p ( x ) = ∑ n . . . + xa n )) . x 0 : p ( x 0 ) can be computed in O ( n ) time. “dual” (and equivalent) representation... { } ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . , ( x n − 1 , y n − 1 ) of n point-value pairs such that Theorem 23.1.2. For any set all the x k values are distinct, there is a unique polynomial p ( x ) of degree n − 1 , such that y k = p ( x k ) , for k = 0 , . . . , n − 1 . 23.1.0.2 Polynomial via point-value { } ( x 0 , y 0 ) , ( x 1 , y 1 ) , ( x 2 , y 2 ) : polynomial through points: ❳❳❳❳ ( x − x 0 )( x − x 1 )( x − x 2 ) ✘ p ( x ) = y 0 ✘✘✘✘ ❳ ❳❳❳❳❳ ( x 0 − x 0 )( x 0 − x 1 )( x 0 − x 2 ) ( x − x 0 ) ✘✘✘✘ ❳❳❳❳ ( x − x 1 )( x − x 2 ) ✘ + y 1 ❳ ( x 1 − x 0 ) ❳❳❳❳❳ ( x 1 − x 1 )( x 1 − x 2 ) ( x − x 0 )( x − x 1 ) ✘✘✘✘ ❳❳❳❳ ( x − x 2 ) ✘ + y 2 ❳ ( x 2 − x 0 )( x 2 − x 1 ) ❳❳❳❳❳ ( x 2 − x 2 ) ( x − x 1 )( x − x 2 ) p ( x ) = y 0 ( x 0 − x 1 )( x 0 − x 2 ) ( x − x 0 )( x − x 2 ) + y 1 ( x 1 − x 0 )( x 1 − x 2 ) ( x − x 0 )( x − x 1 ) + y 2 ( x 2 − x 0 )( x 2 − x 1 ) 1

23.1.0.3 Polynomial via point-value { } ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . , ( x n − 1 , y n − 1 ) : polynomial through points: ∏ n − 1 j ̸ = i ( x − x j ) ∑ p ( x ) = y i j ̸ = i ( x i − x j ) . ∏ i =0 i th is zero for x = x 1 , . . . , x i − 1 , x i +1 , . . . , x n − 1 , and is equal to y i for x = x i . 23.1.1 Polynomials: regular vs. point-value pair representation 23.1.1.1 Just because. (A) Given n point-value pairs. Can compute p ( x ) in O ( n 2 ) time. (B) Point-value pairs representation: Multiply polynomials quickly! (C) p , q polynomial of degree n − 1, both represented by 2 n point-value pairs { } ( x 0 , y 0 ) , ( x 1 , y 1 ) , . . . , ( x 2 n − 1 , y 2 n − 1 ) for p ( x ) , { } ( x 0 , y ′ 0 ) , ( x 1 , y ′ 1 ) , . . . , ( x 2 n − 1 , y ′ and 2 n − 1 ) for q ( x ) . (D) r ( x ) = p ( x ) q ( x ): product. (E) In point-value representation representation of r ( x ) is { } ( x 0 , r ( x 0 )) , . . . , ( x 2 n − 1 , r ( x 2 n − 1 )) {( )} ) ( = x 0 , p ( x 0 ) q ( x 0 ) , . . . , x 2 n − 1 , p ( x 2 n − 1 ) q ( x 2 n − 1 ) { } ( x 0 , y 0 y ′ 0 ) , . . . , ( x 2 n − 1 , y 2 n − 1 y ′ = 2 n − 1 ) . 23.1.1.2 Which implies... (A) p ( x ) and q ( x ): point-value pairs = ⇒ compute r ( x ) = p ( x ) q ( x ) in linear time! (B) ...but r ( x ) is in point-value representation. Bummer. (C) ...but we can compute r ( x ) from this representation. (D) Purpose: Translate quickly (i.e., O ( n log n ) time) from the standard r to point-value pairs repre- sentation of polynomials. (E) ...and back! (F) = ⇒ computing product of two polynomials in O ( n log n ) time. (G) Fast Fourier Transform is a way to do this. (H) choosing the x i values carefully, and using divide and conquer. 2

Part I Computing a polynomial quickly on n values 3

Computing a polynomial quickly on n values 23.2 Computing a polynomial quickly on n values 23.2.1 23.2.1.1 Lets just use some magic. (A) Assume: polynomials have degree n − 1, where n = 2 k . (B) .. pad polynomials with terms having zero coefficients. (C) Magic set of numbers: Ψ = { x 1 , . . . , x n } . { x 2 � } Property: | SQ (Ψ) | = n/ 2, where SQ (Ψ) = � � x ∈ Ψ . (D) | square(Ψ) | = | Ψ | / 2. (E) Easy to find such set... (F) Magic : Have this property repeatedly... SQ ( SQ (Ψ)) has n/ 4 distinct values. (G) SQ ( SQ ( SQ (Ψ))) has n/ 8 values. (H) SQ i (Ψ) has n/ 2 i distinct values. (I) Oops: No such set of real numbers. (J) NO SUCH SET. 23.2.2 Collapsible sets 23.2.2.1 Assume magic... Let us for the time being ignore this technicality, and fly, for a moment, into the land of fantasy, and assume that we do have such a set of numbers, so that | SQ i (Ψ) | = n/ 2 i numbers, for i = 0 , . . . , k . Let us call such a set of numbers collapsible . 23.2.2.2 Breaking the input polynomial into two polynomials of half the degree (A) For a set X = { x 0 , . . . , x n } and polynomial p ( x ), let ⟨( )⟩ ( ) ) ( p = x 0 , p ( x 0 ) , . . . , x n , p ( x n ) . X i =0 a i x i as p ( x ) = u ( x 2 ) + x · v ( x 2 ), where (B) p ( x ) = ∑ n − 1 n/ 2 − 1 n/ 2 − 1 ∑ ∑ a 2 i y i a 1+2 i y i . u ( y ) = and v ( y ) = i =0 i =0 (C) all even degree terms in u ( · ), all odd degree terms in v ( · ). (D) maximum degree of u ( y ), v ( y ) is n/ 2. 23.2.2.3 FFT: The dividing stage i =0 a i x i as p ( x ) = u ( x 2 ) + x · v ( x 2 ). (A) p ( x ) = ∑ n − 1 (B) Ψ: collapsible set of size n . (C) p (Ψ): compute polynomial of degree n − 1 on n values. (D) Decompose: n/ 2 − 1 n/ 2 − 1 ∑ ∑ a 2 i y i a 1+2 i y i . u ( y ) = and v ( y ) = i =0 i =0 5

(E) Need to compute u ( x 2 ), for all x ∈ Ψ. (F) Need to compute v ( x 2 ), for all x ∈ Ψ. { x 2 � } (G) SQ (Ψ) = � � x ∈ Ψ . (H) = ⇒ Need to compute u ( SQ (Ψ)) , v ( SQ (Ψ)). (I) u ( SQ (Ψ)) , v ( SQ (Ψ)): comp. poly. degree n 2 − 1 on n 2 values. 23.2.2.4 FFT: The conquering stage (A) Ψ: Collapsible set of size n . i =0 a i x i as p ( x ) = u ( x 2 ) + x · v ( x 2 ). (B) p ( x ) = ∑ n − 1 (C) u ( y ) = ∑ n/ 2 − 1 v ( y ) = ∑ n/ 2 − 1 a 2 i y i a 1+2 i y i . and i =0 i =0 (D) u ( SQ (Ψ)) , v ( SQ (Ψ)): Computed recursively. (E) Need to compute p (Ψ). (F) For x ∈ Ψ: Compute p ( x ) = u ( x 2 ) + x · v ( x 2 ). (G) Takes constant time per single element x ∈ Ψ. (H) Takes O ( n ) time overall. 23.2.2.5 FFT algorithm FFTAlg ( p , X ) p ( x ) = ∑ n − 1 i =0 a i x i input: p ( x ) : A polynomial of degree n : X : A collapsible set of n elements. output: p ( X ) u ( y ) = ∑ n/ 2 − 1 a 2 i y i i =0 v ( y ) = ∑ n/ 2 − 1 a 1+2 i y i . i =0 { x 2 � } � Y = SQ ( X ) = � x ∈ X . U = FFTAlg ( u, Y ) /* U = u ( Y ) */ V = FFTAlg ( v , Y ) /* V = v ( Y ) */ Out ← ∅ for x ∈ X do /* p ( x ) = u ( x 2 ) + x ∗ v ( x 2 ) , U [ x 2 ] is the value u ( x 2 ) */ ( ) x, U [ x 2 ] + x · V [ x 2 ] ( x, p ( x )) ← Out ← Out ∪ { ( x, p ( x )) } return Out 23.2.3 Running time analysis... 23.2.3.1 ...an old foe emerges once again to serve (A) T ( m, n ): Time of computing a polynomial of degree m on n values. (B) We have that: T ( n − 1 , n ) = 2 T ( n/ 2 − 1 , n/ 2) + O ( n ) . (C) The solution to this recurrence is O ( n log n ). 23.2.4 Generating Collapsible Sets 23.2.4.1 Generating Collapsible Sets (A) How to generate collapsible sets? 6

(B) Trick: Use complex numbers! 23.2.4.2 Complex numbers – a quick reminder Im τ = α + β i (A) Complex number: pair ( α, β ) of real num- β bers. Written as τ = α + i β . Re (B) α : real part, α Im β : imaginary part. (C) i is the root of − 1. (D) Geometrically: a point in the complex τ = α + β i plane: r β φ Re α (A) polar form : τ = r cos ϕ + i r sin ϕ = r (cos ϕ + i sin ϕ ) (B) r = √ α 2 + β 2 and ϕ = arcsin( β/α ). A useful formula: cos ϕ + i sin ϕ = e iϕ 23.2.4.3 (A) By Taylor’s expansion: sin x = x − x 3 3! + x 5 5! − x 7 7! + · · · , cos x = 1 − x 2 2! + x 4 4! − x 6 6! + · · · , 1! + x 2 2! + x 3 e x = 1 + x and 3! + · · · . (B) Since i 2 = − 1: 1! − x 2 2! − i x 3 3! + x 4 4! + i x 5 5! − x 6 e i x = 1 + i x 6! · · · = cos x + i sin x. 23.2.4.4 Back to polar form (A) polar form : τ = r cos ϕ + i r sin ϕ = r (cos ϕ + i sin ϕ ) = re iϕ , τ ′ = r ′ e i ϕ ′ : complex numbers. (B) τ = re i ϕ , (C) τ · τ ′ = re i ϕ · r ′ e i ϕ ′ = rr ′ e i ( ϕ + ϕ ′ ) . (D) e i ϕ is 2 π periodic (i.e., e i ϕ = e i ( ϕ +2 π ) ), and 1 = e i 0 . (E) n th root of 1: complex number τ – raise it to power n get 1. (F) τ = re i ϕ , such that τ n = r n e i nϕ = e i 0 . (G) = ⇒ r = 1, and there must be an integer j , such that nϕ = 0 + 2 πj = ⇒ ϕ = j (2 π/n ) . 7

23.2.5 Roots of unity 23.2.5.1 The desire to avoid war? For j = 0 , . . . , n − 1, we get the n distinct roots of unity . γ 1 (4) = β 3 (4) = i γ 2 (8) = β 6 (8) = i γ 3 (8) = β 5 (8) γ 1 (8) = β 7 (8) γ 2 (4) = β 2 (4) = − 1 γ 4 (8) = β 4 (8) = − 1 1 1 γ 5 (8) = β 3 (8) γ 7 (8) = β 1 (8) γ 6 (8) = β 2 (8) = − i γ 3 (4) = β 1 (4) = − i ( n = 4) ( n = 8) ( n = 16) 23.2.5.2 Back to collapsible sets (A) Can do all basic calculations on complex numbers in O (1) time. (B) Idea: Work over the complex numbers. (C) Use roots of unity! (D) γ : n th root of unity. There are n such roots, and let γ j ( n ) denote the j th root. γ j ( n ) = cos((2 πj ) /n ) + i sin((2 πj ) /n ) = γ j . Let A ( n ) = { γ 0 ( n ) , . . . , γ n − 1 ( n ) } . (E) | SQ ( A ( n )) | has n/ 2 entries. (F) SQ ( A ( n )) = A ( n/ 2) (G) n to be a power of 2, then A ( n ) is the required collapsible set. 23.2.5.3 The first result... Theorem 23.2.1. Given polynomial p ( x ) of degree n , where n is a power of two, then we can compute p ( X ) in O ( n log n ) time, where X = A ( n ) is the set of n different powers of the n th root of unity over the complex numbers. 23.2.6 Problem... 23.2.6.1 We can go, but can we come back? (A) Can multiply two polynomials quickly (B) by transforming them to the point-value pairs representation... (C) over the n th root of unity. (D) Q: How to transform this representation back to the regular representation. (E) A: Do some confusing math... 8