1 n +k -1 X n = k-1 (1-X) k n = 0 The Binomial Formula n n - PowerPoint PPT Presentation

15-251: Great Theoretical Ideas in Computer Science Fall 2016 Lecture 27 December 1, 2016 Generating Functions   1 n +k -1 X n = k-1 (1-X) k n = 0

The Binomial Formula n n    (1 x) n x k       k   k 0  The polynomial (1+x) n packages in convenient algebraic form information about the sequence n     k=0,1,…,n   k   Generating functions are a formal algebraic view for (infinite) sequences

(1+x) n is the “generating function” for the sequence n   k=0,1,…,n     k   Generating functions are a formal algebraic representation for (infinite) sequences Often, surprisingly powerful representation to understand the sequence!

X n – 1 1 + X 1 + X 2 + X 3 + … + X n-2 + X n-1 = X - 1 1 - X n = 1 - X Recall the Geometric Series

1 1 + X 1 + X 2 + X 3 + … + X n + … = 1 - X the Infinite Geometric Series Holds when we plug X = a with |a| < 1 But also makes sense if we view the infinite sum on the left as a formal power series in variable X

1 + X 1 + X 2 + X 3 + … + X n + … P(X) = - X 1 - X 2 - X 3 - … - X n - X n+1 - … -X * P(X) = (1- X) P(X) = 1 1  P(X) = 1 - X

What is a Generating Function? Just a particular representation of sequences… In general, when is a sequence…

Formal Power Series   a n X n P(X) = n = 0 There are no worries about convergence issues. This is a purely syntactic object.

Formal Power Series   a i X i P(X) = i = 0 If you want, think of as the infinite vector V = < a 0 , a 1 , a 2 , ..., a n , … > But, as you will see, thinking of as a “polynomial” is very natural and powerful.

…And why would I use one? They're fun and powerful ! Solving (impossible looking) counting problems Solving recurrences precisely Proving identities

In Graham-Knuth- Patashnik’s text “Concrete Mathematics: A Foundation for Computer Science”, generating functions are described as “the most important idea in this whole book.” Generating functions transform problems about sequences into problems about functions, allowing us to put the piles of machinery available for manipulating functions to work for understanding sequences

Operations on Generating Functions A(X) = a 0 + a 1 X + a 2 X 2 + … B(X) = b 0 + b 1 X + b 2 X 2 + … adding them together (A+B)(X) = (a 0 +b 0 ) + (a 1 +b 1 ) X + (a 2 +b 2 ) X 2 + … like adding the vectors position-wise <4,2,3,…> + <5,1,1,….> = <9,3,4,…>

Operations on Generating Functions A(X) = a 0 X 0 + a 1 X 1 + a 2 X 2 + … multiplying by X X * A(X) = 0 X 0 + a 0 X 1 + a 1 X 2 + a 2 X 3 + … like shifting the vector entries SHIFT<4,2,3,…> = <0,4,2,3,…>

Example Example: Store: V := <1,0,0,…>; V = <1,0,0,0,…> V = <1,1,0,0,…> Loop n times V = <1,2,1,0,…> V := V + SHIFT(V); V = <1,3,3,1,…> V = n’th row of Pascal’s triangle (binomial coefficients 𝑜 𝑙 )

Example Example: V := <1,0,0,…>; P V := 1; Loop n times P V := P V *(1+X); V := V + SHIFT(V); V = n th row of Pascal’s triangle (binomial coefficients 𝑜 𝑙 )

Example Example: V := <1,0,0,…>; P V = (1+ X) n Loop n times V := V + SHIFT(V); As expected, the coefficients of P V give the binomial coefficients 𝑜 𝑙

To repeat… Given a sequence V = < a 0 , a 1 , a 2 , ..., a n , … > associate a formal power series with it   a i X i P(X) = i = 0 This is the “generating function” for V

Fibonacci Numbers F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 i.e., the sequence <0,1,1,2,3,5,8,13…> is represented by the power series (generating function) 0 + 1X 1 + 1X 2 + 2 X 3 + 3 X 4 + 5 X 5 + 8 X 6 +…

Two Representations F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 A(X) = 0 + 1X 1 + 1X 2 + 2 X 3 + 3 X 4 + 5 X 5 + 8 X 6 +… Can we write A(X) more succinctly?

A(X) = F 0 + F 1 X 1 + F 2 X 2 + F 3 X 3 + … + F n X n +… = X 1 + (F 1 + F 0 )X 2 + (F 2 +F 1 ) X 3 + … + (F n-1 +F n-2 ) X n +… X A(X) = (1 – X – X 2 )

G.F for Fibonaccis F 0 = 0, F 1 = 1, F n = F n-1 + F n-2 has the generating function X A(X) = (1 – X – X 2 ) i.e., the coefficient of X n in A(X) is F n

+ X 2 + 2X 3 + 3X 4 + 5X 5 + 8X 6 X 1 – X – X 2 X -(X – X 2 – X 3 ) X 2 + X 3 -(X 2 – X 3 – X 4 ) 2X 3 + X 4 -(2X 3 – 2X 4 – 2X 5 ) 3X 4 + 2X 5 -(3X 4 – 3X 5 – 3X 6 ) 5X 5 + 3X 6 -(5X 5 – 5X 6 – 5X 7 ) 8X 6 + 5X 7 -(8X 6 – 8X 7 – 8X 8 )

Closed form expression for F n ? F 0 = 0, F 1 = 1, X A(X) = F n = F n-1 + F n-2 (1 – X – X 2 ) let’s factor (1 – X – X 2 ) (1 – X – X 2 ) = (1 – φ 1 X)(1 – φ 2 X) where φ 1 = 1 + √5 2 φ 2 = 1 - √5 2

Let’s simplify X F 0 = 0, F 1 = 1, A(X) = F n = F n-1 + F n-2 (1 – φ 1 X)(1 – φ 2 X) some elementary algebra omitted…* 1 1 1 -1 A(X) = + (1 – φ 2 X) √5 (1 – φ 1 X) √5 *you are not allowed to say this in your answers…

1 1 1 -1 A(X) = + (1 – φ 2 X) (1 – φ 1 X) √5 √5 1 = 1 + φ 1 X + φ 1 2 X 2 + … + φ 1 n X n + … (1 – φ 1 X) 1 = 1 + Y 1 + Y 2 + Y 3 + … + Y n + … 1 - Y the Infinite Geometric Series

1 1 1 -1 A(X) = + (1 – φ 2 X) (1 – φ 1 X) √5 √5 1 = 1 + φ 1 X + φ 1 2 X 2 + … + φ 1 n X n + … (1 – φ 1 X) 1 = 1 + φ 2 X + … + φ 2 n X n + … (1 – φ 2 X)  the coefficient of X n in A(X) is… 1 -1 φ 2 n φ 1 n + √5 √5

Closed form for Fibonaccis 1 -1 F n = φ n (-1/ φ ) n + √5 √5 where φ = 1 + √5 2 “golden ratio”

Closed form for Fibonaccis 1 -1 F n = φ n (-1/ φ ) n + √5 √5 1 φ n F n = closest integer to √5

To recap… Given a sequence V = < a 0 , a 1 , a 2 , ..., a n , … > associate a formal power series with it   a i X i P(X) = i = 0 This is the “generating function” for V We just used this for solving the Fibonacci recurrence…

Multiplication A(X) = a 0 + a 1 X + a 2 X 2 + … B(X) = b 0 + b 1 X + b 2 X 2 + … multiply them together (A*B)(X) = (a 0* b 0 ) + (a 0 b 1 + a 1 b 0 ) X + (a 0 b 2 + a 1 b 1 + a 2 b 0 ) X 2 + (a 0 b 3 + a 1 b 2 + a 2 b 1 + a 3 b 0 ) X 3 + … seems a bit less natural in the vector representation (it’s called a “convolution” there)

Multiplication: special case A(X) = a 0 + a 1 X + a 2 X 2 + … 1 Special case: B(X) = 1 + X + X 2 + … = 1-X multiply them together (A*B)(X) = a 0 + (a 0 + a 1 ) X + (a 0 + a 1 + a 2 ) X 2 + (a 0 + a 1 + a 2 + a 3 ) X 3 + … It gives us partial sums!

Poll time What’s a closed form for the generating function of the sequence of natural numbers 〈0,1,2,3,4, … 〉 , i.e., the sequence 𝑏 𝑜 = 𝑜 for 𝑜 ≥ 0 ? 𝑌 + 2 𝑌 2 + 3 𝑌 3 + ⋯ + 𝑜 𝑌 𝑜 + ⋯ equals 𝑌 1 − 𝑌 2

A(X) = a 0 + a 1 X + a 2 X 2 + … 1 B(X) = 1 + X + X 2 + … = 1-X (A*B)(X) = a 0 + (a 0 + a 1 ) X + (a 0 + a 1 + a 2 ) X 2 + (a 0 + a 1 + a 2 + a 3 ) X 3 + … It gives us partial sums. Apply with A(X)=B(X) 1 1 + 2𝑌 + 3𝑌 2 + 4𝑌 3 + ⋯ + 𝑜 𝑌 𝑜−1 + ⋯ = 1 − 𝑌 2 To get generating function for naturals 〈0,1,2,3, … 〉 , which is a shift of 1,2,3, … , multiply the G.F by 𝑌

What happens if we again take prefix sums? 1 Take 1 + 2X + 3X 2 + 4X 3 + … = (1-X) 2 multiplying through by 1/(1-X) 1 Δ 1 + Δ 2 𝑌 + Δ 3 𝑌 2 + ⋯ = (1-X) 3 𝑜+1 where Δ 𝑜 = is the 2 sequence of triangular numbers

What’s the pattern? 1 <1,1,1,1,…> = 1-X 1 <1,2,3,4,…> = (1-X) 2 1 <1,3,6,10,…> = (1-X) 3 1 ??? = (1-X) k

What’s the pattern? 0 1 2 3 1 , , , , … = 0 0 0 0 1-X 1 <1,2,3,4,…> = (1-X) 2 1 = <1,3,6,10,…> (1-X) 3 1 ??? = (1-X) n

What’s the pattern? 0 1 2 3 1 , , , , … = 0 0 0 0 1-X 1 1 2 3 4 = , , , , … (1-X) 2 1 1 1 1 1 = < 1,3,6,10,…> (1-X) 3 1 ??? = (1-X) n

What’s the pattern? 0 1 2 3 1 , , , , … = 0 0 0 0 1-X 1 1 2 3 4 = , , , , … (1-X) 2 1 1 1 1 1 2 3 4 5 = , , , , … 2 2 2 2 (1-X) 3 1 ??? = (1-X) k

What’s the pattern? 0 1 2 3 1 , , , , … = 0 0 0 0 1-X 1 1 2 3 4 = , , , , … (1-X) 2 1 1 1 1 1 2 3 4 5 = , , , , … 2 2 2 2 (1-X) 3   1 n+k-1 X n = (1-X) k k-1 n = 0

What is the coefficient of X n in Another the expansion of: way to see it… ( 1 + X + X 2 + X 3 + X 4 + . . . . ) k ? To get 𝑌 𝑜 we need to pick 𝑌 𝑓 𝑗 in 𝑗 ′ 𝑢ℎ factor, for 𝑗 = 1,2, … , 𝑙 with 𝑓 1 + 𝑓 2 + ⋯ + 𝑓 𝑙 = 𝑜. Each exponent can be any natural number. ∴ coefficient of X n is the number of non-negative solutions to: e 1 + e 2 + . . . + e k = n n + k - 1 which is k - 1