Solving Recurrence Relations Cunsheng Ding HKUST, Hong Kong - PowerPoint PPT Presentation

Solving Recurrence Relations Cunsheng Ding HKUST, Hong Kong October 10, 2015 Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 1 / 25

Contents Introduction 1 Linear Recurrence Relations 2 Solving Linear Homogeneous Recurrence Relations 3 4 Generating Functions and Linear Recursions Solving Nonlinear Recurrence Relations 5 Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 2 / 25

Objectives of this Lecture Recursions and linear recursions were introduced in the previous lecture. The objectives of this lecture are the following. Recall the definitions of linear recurrence relations. Introduce general techniques for solving linear recurrence relations. Solving a number of important types of linear recurrence relations. Solving nonlinear recurrence relations. These techniques will be fundamental in the design and analysis of computer algorithms. Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 3 / 25

Linear Recurrence Relations Definition 1 A linear recurrence relation with constant coefficients for a sequence ( s i ) ∞ i = 0 is a formula that relates each term s i to its predecessors s i − 1 , s i − 2 ,..., s i − ℓ in the form s i = c 1 s i − 1 + c 2 s i − 2 + ··· + c ℓ s i − ℓ + d for all i ≥ ℓ, (1) where ℓ is some fixed integer and d is a constant. Example 2 Let ( s i ) ∞ i = 0 be defined by s i = i for all integers i ≥ 0. Then s i = s i − 1 + 1 is a linear recurrence relation for the sequence with the initial condition that s 0 = 0. Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 4 / 25

Linear Homogeneous Recurrence Relations Definition 3 A linear homogeneous recurrence relation of degree ℓ with constant coefficients (in sort, LHRRCC) for a sequence ( s i ) ∞ i = 0 is a formula that relates each term s i to its predecessors s i − 1 , s i − 2 ,..., s i − ℓ in the form s i = c 1 s i − 1 + c 2 s i − 2 + ··· + c ℓ s i − ℓ for all i ≥ ℓ, (2) where ℓ is some fixed integer, and c i ’s are real constants with c ℓ � = 0. The equation x ℓ − c 1 x ℓ − 1 − c 2 x ℓ − 2 −···− c ℓ − 1 x − c ℓ = 0 (3) is called the characteristic equation of the linear recursion of (2), and its roots are referred to as the characteristic roots. The polynomial x ℓ − c 1 x ℓ − 1 − c 2 x ℓ − 2 −···− c ℓ − 1 x − c ℓ is called the characteristic polynomial of the sequence. Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 5 / 25

Solving Linear Homogeneous Recurrence Relations Question 1 Given a sequence ( s i ) ∞ i = i 0 defined by a linear homogeneous recurrence relation with constant coefficients, how do you solve the LHRRCC so that you are able to find a mathematical formula for each term of the sequence? Example 4 Let ( s i ) ∞ i = 0 be defined by the following linear homogeneous recurrence relation of degree 2: s i + 1 = 2 s i − s i − 1 for all i ≥ 1 with initial conditions s 0 = 1 and s 1 = 3. Find a mathematical formula in terms of i for each s i . Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 6 / 25

When the Characteristic Roots Have Multiplicity 1 Recurrence: s i = c 1 s i − 1 + c 2 s i − 2 + ··· + c ℓ s i − ℓ for all i ≥ ℓ . Characteristic equation: x ℓ − c 1 x ℓ − 1 − c 2 x ℓ − 2 −···− c ℓ − 1 x − c ℓ = 0. Theorem 5 If the characteristic equation has distinct roots r 1 , r 2 ,..., r ℓ , then a sequence ( s i ) ∞ i = 0 satisfies the linear recurrence relation if and only if s i = α 1 r i 1 + α 2 r i 2 + ... + α ℓ r i ℓ for integers i ≥ 0 , (4) where α 1 , α 2 ,..., α ℓ are constants. Remarks A proof of the necessity will be presented in a tutorial. The proof of the sufficiency will be left as an assignment problem. Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 7 / 25

When the Characteristic Roots Have Multiplicity 1 Recurrence: s i = c 1 s i − 1 + c 2 s i − 2 + ··· + c ℓ s i − ℓ for all i ≥ ℓ . Characteristic equation: x ℓ − c 1 x ℓ − 1 − c 2 x ℓ − 2 −···− c ℓ − 1 x − c ℓ = 0. Steps in solving the recurrence relation Solving the characteristic equation to find out all the distinct roots 1 r 1 , r 2 ,..., r ℓ . Use the initial conditions s 0 , s 1 ,..., s ℓ − 1 and the roots r i to solve the 2 following set of equations, s i = α 1 r i 1 + α 2 r i 2 + ... + α ℓ r i ℓ , i = 0 , 1 , 2 ,...,ℓ − 1 . This will determine α 1 , α 2 ,..., α ℓ . Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 8 / 25

Solving the First-order Linear Homogeneous Recurrence Relations Recurrence: s i = c 1 s i − 1 for all i ≥ 1. Characteristic equation: x − c 1 = 0. Steps in solving the recurrence relation Solving the characteristic equation to find out the unique root r 1 = c 1 . 1 Use the initial condition s 0 and the root r 1 to solve the following equation 2 s 0 = α 1 . This will determine α 1 = s 0 . Hence, s i = s 0 c i 1 for all integers i ≥ 0. This is the geometric sequence. Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 9 / 25

Solving the Second-order Linear Homogeneous Recurrence Relations Recurrence: s i = c 1 s i − 1 + c 2 s i − 2 for all i ≥ 2. Characteristic equation: x 2 − c 1 x − c 2 = 0. Steps in solving the recurrence relation Solving the characteristic equation to find out the two distinct roots r 1 , r 2 . 1 Use the initial conditions s 0 , s 1 and the roots r 1 , r 2 to solve the following 2 set of equations, s 0 = α 1 + α 2 , s 1 = α 1 r 1 + α 2 r 2 . This yields α 1 and α 2 . By Theorem 5, we have s i = s 1 − s 0 r 2 1 + s 0 r 1 − s 1 r i r i 1 r 1 − r 2 r 1 − r 2 Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 10 / 25

The Fibonacci Sequence Problem 6 The sequence ( F i ) ∞ i = 0 is defined by the linear homogeneous recursion F i = F i − 1 + F i − 2 for all i ≥ 2 , with initial condition F 0 = 0 and F 1 = 1 . Solve this linear recurrence relation. Solution 7 The characteristic equation x 2 − x − 1 = 0 has the following distinct roots √ √ r 1 = 1 + 5 , r 2 = 1 − 5 . 2 2 Hence, √ √ � i � i � � 1 1 + 5 − 1 1 − 5 F i = √ √ 2 2 5 5 Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 11 / 25

An Exercise Problem 8 Solve the following linear recurrence relation s i = 6 s i − 1 − 11 s i − 2 + 6 s i − 3 for all i ≥ 3 with initial conditions s 0 = 2 , s 1 = 5 and s 2 = 15 . Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 12 / 25

When the Characteristic Roots Have Multiplicity > 1 Recurrence: s i = c 1 s i − 1 + c 2 s i − 2 + ··· + c ℓ s i − ℓ for all i ≥ ℓ . Characteristic equation: x ℓ − c 1 x ℓ − 1 − c 2 x ℓ − 2 −···− c ℓ − 1 x − c ℓ = 0. Theorem 9 If the characteristic equation has distinct roots r 1 , r 2 ,..., r t with multiplicities m 1 , m 2 ,..., m t , respectively, so that all m i ’s are positive and ∑ t i = 1 m i = ℓ , then a sequence ( s i ) ∞ i = 0 satisfies the linear recurrence relation if and only if ( α 1 , 0 + α 1 , 1 i + ... + α 1 , m 1 − 1 i m 1 − 1 ) r i = 1 + s i ( α 2 , 0 + α 2 , 1 i + ... + α 2 , m 2 − 1 i m 2 − 1 ) r i 2 + ... + ( α t , 0 + α t , 1 i + ... + α t , m t − 1 i m t − 1 ) r i t for all i ≥ 0 , (5) where all α i , j ’s are constants. Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 13 / 25

When the Characteristic Roots Have Multiplicity > 1 Recurrence: s i = c 1 s i − 1 + c 2 s i − 2 + ··· + c ℓ s i − ℓ for all i ≥ ℓ . Characteristic equation: x ℓ − c 1 x ℓ − 1 − c 2 x ℓ − 2 −···− c ℓ − 1 x − c ℓ = 0. Steps in solving the recurrence relation Solving the characteristic equation to find out all the distinct roots 1 r 1 , r 2 ,..., r t and their multiplicities. Use the initial conditions s 0 , s 1 ,..., s ℓ − 1 and the roots r i ’s and their 2 multiplicities m i to solve the following set of equations, ( α 1 , 0 + α 1 , 1 i + ... + α 1 , m 1 − 1 i m 1 − 1 ) r i = 1 + s i ( α 2 , 0 + α 2 , 1 i + ... + α 2 , m 2 − 1 i m 2 − 1 ) r i = 2 + ... + ( α t , 0 + α t , 1 i + ... + α t , m t − 1 i m t − 1 ) r i = t , i = 0 , 1 ,...,ℓ − 1 . This will determine α i , j ’s. Remark: We will not present a proof for this theorem. Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 14 / 25

When the Characteristic Roots Have Multiplicity > 1 Recurrence: s i = 6 s i − 1 − 9 s i − 2 for all i ≥ 2 with s 0 = 1 and s 1 = 6. Characteristic equation: x 2 − 6 x + 9 = 0. Solution 10 Note that x 2 − 6 x + 9 = 0 has the only root x = 3 with multiplicity 2 . By Theorem 9, s i = α 1 3 i + α 2 i 3 i . Using the initial conditions, we obtain that α 1 = α 2 = 1 . Hence, s i = ( i + 1 ) 3 i . Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 15 / 25

Rational Functions Definition 11 A rational function is the quotient of two “polynomials” of finite degree over the set of real numbers. Example 12 x + x 2 1 − 3 x + 3 x 2 − x 3 . Cunsheng Ding (HKUST, Hong Kong) Solving Recurrence Relations October 10, 2015 16 / 25