Linearly independent functions Definition The set of functions { 1 - PowerPoint PPT Presentation

Linearly independent functions Definition The set of functions { φ 1 , . . . , φ n } is called linearly independent on [ a , b ] if c 1 φ 1 ( x ) + c 2 φ 2 ( x ) + · · · + c n φ n ( x ) = 0 , for all x ∈ [ a , b ] implies that c 1 = c 2 = · · · = c n = 0. Otherwise the set of functions is called linearly dependent . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 203

Linearly independent functions Example Suppose φ j ( x ) is a polynomial of degree j for j = 0 , 1 , . . . , n , then { φ 0 , . . . , φ n } is linearly independent on any interval [ a , b ]. Proof. Suppose there exist c 0 , . . . , c n such that c 0 φ 0 ( x ) + · · · + c n φ n ( x ) = 0 for all x ∈ [ a , b ]. If c n � = 0, then this is a polynomial of degree n and can have at most n roots, contradiction. Hence c n = 0. Repeat this to show that c 0 = · · · = c n = 0. Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 204

Linearly independent functions Example Suppose φ 0 ( x ) = 2 , φ 1 ( x ) = x − 3 , φ 2 ( x ) = x 2 + 2 x + 7, and Q ( x ) = a 0 + a 1 x + a 2 x 2 . Show that there exist constants c 0 , c 1 , c 2 such that Q ( x ) = c 0 φ 0 ( x ) + c 1 φ 1 ( x ) + c 2 φ 2 ( x ). Solution. Substitute φ j into Q ( x ), and solve for c 0 , c 1 , c 2 . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 205

Linearly independent functions We denote Π n = { a 0 + a 1 x + · · · + a n x n | a 0 , a 1 , . . . , a n ∈ R } , i.e., Π n is the set of polynomials of degree ≤ n . Theorem Suppose { φ 0 , . . . , φ n } is a collection of linearly independent polynomials in Π n , then any polynomial in Π n can be written uniquely as a linear combination of φ 0 ( x ) , . . . , φ n ( x ) . { φ 0 , . . . , φ n } is called a basis of Π n . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 206

Orthogonal functions Definition An integrable function w is called a weight function on the interval I if w ( x ) ≥ 0, for all x ∈ I , but w ( x ) �≡ 0 on any subinterval of I . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 207

Orthogonal functions Example 1 Define a weight function w ( x ) = 1 − x 2 on interval ( − 1 , 1). √ ( x ) 1 x � 1 1 Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 208

Orthogonal functions Suppose { φ 0 , . . . , φ n } is a set of linearly independent functions in C [ a , b ] and w is a weight function on [ a , b ]. Given f ( x ) ∈ C [ a , b ], we seek a linear combination n � a k φ k ( x ) k =0 to minimize the least squares error: � 2 � b n � � E ( a ) = w ( x ) f ( x ) − a k φ k ( x ) d x a k =0 where a = ( a 0 , . . . , a n ). Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 209

Orthogonal functions As before, we need to solve a ∗ from ∇ E ( a ) = 0: � b n ∂ E � � � = w ( x ) f ( x ) − a k φ k ( x ) φ j ( x ) d x = 0 ∂ a j a k =0 for all j = 0 , . . . , n . Then we obtain the normal equation �� b � b n � � w ( x ) φ k ( x ) φ j ( x ) d x a k = w ( x ) f ( x ) φ j ( x ) d x a a k =0 which is a linear system of n + 1 equations about a = ( a 0 , . . . , a n ) ⊤ . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 210

Orthogonal functions If we chose the basis { φ 0 , . . . , φ n } such that � b � 0 , when j � = k w ( x ) φ k ( x ) φ j ( x ) d x = when j = k α j , a for some α j > 0, then the LHS of the normal equation simplifies to α j a j . Hence we obtain closed form solution a j : � b a j = 1 w ( x ) f ( x ) φ j ( x ) d x α j a for j = 0 , . . . , n . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 211

Orthogonal functions Definition A set { φ 0 , . . . , φ n } is called orthogonal on the interval [ a , b ] with respect to weight function w if � b � 0 , when j � = k w ( x ) φ k ( x ) φ j ( x ) d x = when j = k α j , a for some α j > 0 for all j = 0 , . . . , n . If in addition α j = 1 for all j = 0 , . . . , n , then the set is called orthonormal with respect to w . The definition above applies to general functions, but for now we focus on orthogonal/orthonormal polynomials only. Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 212

Gram-Schmidt process Theorem A set of orthogonal polynomials { φ 0 , . . . , φ n } on [ a , b ] with respect to weight function w can be constructed in the recursive way ◮ First define � b a xw ( x ) d x φ 0 ( x ) = 1 , φ 1 ( x ) = x − � b a w ( x ) d x ◮ Then for every k ≥ 2 , define φ k ( x ) = ( x − B k ) φ k − 1 ( x ) − C k φ k − 2 ( x ) where � b � b a xw ( x )[ φ k − 1 ( x )] 2 d x a xw ( x ) φ k − 1 ( x ) φ k − 2 ( x ) d x B k = , C k = � b � b a w ( x )[ φ k − 1 ( x )] 2 d x a w ( x )[ φ k − 2 ( x )] 2 d x Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 213

Orthogonal polynomials Corollary Let { φ 0 , . . . , φ n } be constructed by the Gram-Schmidt process in the theorem above, then for any polynomial Q k ( x ) of degree k < n, there is � b w ( x ) φ n ( x ) Q k ( x ) d x = 0 a Proof. Q k ( x ) can be written as a linear combination of φ 0 ( x ) , . . . , φ k ( x ), which are all orthogonal to φ n with respect to w . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 214

Legendre polynomials Using weight function w ( x ) ≡ 1 on [ − 1 , 1], we can construct Legendre polynomials using the recursive process above to get P 0 ( x ) = 1 P 1 ( x ) = x P 2 ( x ) = x 2 − 1 3 P 3 ( x ) = x 3 − 3 5 x P 4 ( x ) = x 4 − 6 7 x 2 + 3 35 P 5 ( x ) = x 5 − 10 9 x 3 + 5 21 x . . . Use the Gram-Schmidt process to construct them by yourself. Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 215

Legendre polynomials The first few Legendre polynomials: y y = P 1 ( x ) 1 y = P 2 ( x ) 0.5 y = P 3 ( x ) y = P 4 ( x ) y = P 5 ( x ) x 1 � 1 � 0.5 � 1 Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 216

Chebyshev polynomials 1 Using weight function w ( x ) = 1 − x 2 on ( − 1 , 1), we can construct √ Chebyshev polynomials using the recursive process above to get T 0 ( x ) = 1 T 1 ( x ) = x T 2 ( x ) = 2 x 2 − 1 T 3 ( x ) = 4 x 3 − 3 x T 4 ( x ) = 8 x 4 − 8 x 2 + 1 . . . It can be shown that T n ( x ) = cos( n arccos x ) for n = 0 , 1 , . . . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 217

Chebyshev polynomials The first few Chebyshev polynomials: y y = T 1 ( x ) 1 y = T 3 ( x ) y = T 4 ( x ) x � 1 1 y = T 2 ( x ) � 1 Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 218

Chebyshev polynomials The Chebyshev polynomials T n ( x ) of degree n ≥ 1 has n simple zeros in [ − 1 , 1] (from right to left) at � 2 k − 1 � x k = cos ¯ for each k = 1 , 2 , . . . , n π , 2 n Moreover, T n has maximum/minimum (from right to left) at � k π � k ) = ( − 1) k for each k = 0 , 1 , 2 , . . . , n x ′ x ′ ¯ k = cos where T n (¯ n Therefore T n ( x ) has n distinct roots and n + 1 extreme points on [ − 1 , 1]. These 2 n + 1 points, from right to left, are max, zero, min, zero, max ... Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 219

Monic Chebyshev polynomials The monic Chebyshev polynomials ˜ T n ( x ) are given by ˜ T 0 = 1 and 1 ˜ T n = 2 n − 1 T n ( x ) for n ≥ 1. y y = T 1 ( x ) � 1 � y = T 2 ( x ) � y = T 3 ( x ) � � y = T 5 ( x ) y = T 4 ( x ) x � 1 1 � 1 Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 220

Monic Chebyshev polynomials The monic Chebyshev polynomials are ˜ T 0 ( x ) = 1 ˜ T 1 ( x ) = x T 2 ( x ) = x 2 − 1 ˜ 2 T 3 ( x ) = x 3 − 3 ˜ 4 x T 4 ( x ) = x 4 − x 2 + 1 ˜ 8 . . . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 221

Monic Chebyshev polynomials The monic Chebyshev polynomials ˜ T n ( x ) of degree n ≥ 1 has n simple zeros in [ − 1 , 1] at � 2 k − 1 � x k = cos ¯ π , for each k = 1 , 2 , . . . , n 2 n Moreover, T n has maximum/minimum at k ) = ( − 1) k � k π � x ′ x ′ ¯ k = cos where T n (¯ 2 n − 1 , for each k = 0 , 1 , . . . , n n Therefore ˜ T n ( x ) also has n distinct roots and n + 1 extreme points on [ − 1 , 1]. Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 222

Monic Chebyshev polynomials Denote ˜ Π n be the set of monic polynomials of degree n . Theorem For any P n ∈ ˜ Π n , there is 1 x ∈ [ − 1 , 1] | ˜ 2 n − 1 = max T n ( x ) | ≤ x ∈ [ − 1 , 1] | P n ( x ) | max The “=” holds only if P n ≡ ˜ T n . Numerical Analysis I – Xiaojing Ye, Math & Stat, Georgia State University 223