Linear Algebra Chapter 6: Orthogonality Section 6.2. The - PowerPoint PPT Presentation

Linear Algebra Chapter 6: Orthogonality Section 6.2. The Gram-Schmidt Process—Proofs of Theorems May 5, 2020 () Linear Algebra May 5, 2020 1 / 32

Table of contents Theorem 6.2. Orthogonal Bases 1 Theorem 6.3. Projection Using an Orthogonal Basis 2 Page 347 Number 4 3 Theorem 6.4. Orthonormal Basis (Gram-Schmidt) Theorem 4 Page 348 Number 10 5 Corollary 1. QR -Factorization 6 Page 348 Number 26 7 Corollary 2. Expansion of an Orthogonal Set to an Orthogonal Basis 8 Page 348 Number 20 9 10 Page 349 Number 30 11 Page 349 Number 32 12 Page 349 Number 34 () Linear Algebra May 5, 2020 2 / 32

Theorem 6.2. Orthogonal Bases Theorem 6.2 Theorem 6.2. Orthogonal Bases. v k } be an orthogonal set of nonzero vectors in R n . Then Let { � v 1 , � v 2 , . . . , � this set is independent and consequently is a basis for the subspace sp( � v 1 , � v 2 , . . . , � v k ) . Proof. Let j be an integer between 2 and k . Consider v j = s 1 � � v 1 + s 2 � v 2 + · · · + s j − 1 � v j − 1 . () Linear Algebra May 5, 2020 3 / 32

Theorem 6.2. Orthogonal Bases Theorem 6.2 Theorem 6.2. Orthogonal Bases. v k } be an orthogonal set of nonzero vectors in R n . Then Let { � v 1 , � v 2 , . . . , � this set is independent and consequently is a basis for the subspace sp( � v 1 , � v 2 , . . . , � v k ) . Proof. Let j be an integer between 2 and k . Consider v j = s 1 � � v 1 + s 2 � v 2 + · · · + s j − 1 � v j − 1 . If we take the dot product of each side of this equation with � v j then, since the set of vectors is orthogonal, we get � v j · � v j = 0, which contradicts the v j � = � hypothesis that � 0. Therefore no � v j is a linear combination of its predecessors and by Page 203 Number 37, the set is independent. Therefore the set is a basis for its span. () Linear Algebra May 5, 2020 3 / 32

Theorem 6.2. Orthogonal Bases Theorem 6.2 Theorem 6.2. Orthogonal Bases. v k } be an orthogonal set of nonzero vectors in R n . Then Let { � v 1 , � v 2 , . . . , � this set is independent and consequently is a basis for the subspace sp( � v 1 , � v 2 , . . . , � v k ) . Proof. Let j be an integer between 2 and k . Consider v j = s 1 � � v 1 + s 2 � v 2 + · · · + s j − 1 � v j − 1 . If we take the dot product of each side of this equation with � v j then, since the set of vectors is orthogonal, we get � v j · � v j = 0, which contradicts the v j � = � hypothesis that � 0. Therefore no � v j is a linear combination of its predecessors and by Page 203 Number 37, the set is independent. Therefore the set is a basis for its span. () Linear Algebra May 5, 2020 3 / 32

Theorem 6.3. Projection Using an Orthogonal Basis Theorem 6.3 Theorem 6.3. Projection Using an Orthogonal Basis. v k } be an orthogonal basis for a subspace W of R n , and Let { � v 1 , � v 2 , . . . , � let � b ∈ R n . The projection of � b on W is � � � b · � v 1 b · � v 2 b · � v k � b W = proj W ( � b ) = v 1 + � v 2 + · · · + � v k . � v 1 · � � v 1 v 2 · � � v 2 v k · � � v k Proof. We know from Theorem 6.1 that � b = � b W + � b W ⊥ where � b W is the projection of � b on W and � b W ⊥ is the projection of � b on W ⊥ . Since � b W ∈ W and { � v 1 , � v 2 , . . . , � v k } is a basis of W , then � b W = r 1 � v 1 + r 2 � v 2 + · · · + r k � v k for some scalars r 1 , r 2 , . . . , r k . We now find these r i ’s. () Linear Algebra May 5, 2020 4 / 32

Theorem 6.3. Projection Using an Orthogonal Basis Theorem 6.3 Theorem 6.3. Projection Using an Orthogonal Basis. v k } be an orthogonal basis for a subspace W of R n , and Let { � v 1 , � v 2 , . . . , � let � b ∈ R n . The projection of � b on W is � � � b · � v 1 b · � v 2 b · � v k � b W = proj W ( � b ) = v 1 + � v 2 + · · · + � v k . � v 1 · � � v 1 v 2 · � � v 2 v k · � � v k Proof. We know from Theorem 6.1 that � b = � b W + � b W ⊥ where � b W is the projection of � b on W and � b W ⊥ is the projection of � b on W ⊥ . Since � b W ∈ W and { � v 1 , � v 2 , . . . , � v k } is a basis of W , then � b W = r 1 � v 1 + r 2 � v 2 + · · · + r k � v k for some scalars r 1 , r 2 , . . . , r k . We now find these r i ’s. () Linear Algebra May 5, 2020 4 / 32

Theorem 6.3. Projection Using an Orthogonal Basis Theorem 6.3 (continued) Proof (continued). Taking the dot product of � b with � v i we have � ( � b W + � v i = ( � v i ) + ( � b · � v i = b W ⊥ ) · � b W · � b W ⊥ · � v i ) ( r 1 � v 1 · � v i + r 2 � v 2 · � v i + · · · + r k � v k · � = v i ) + 0 = r i � v i · � v i . Therefore r i = ( � b · � v i ) / ( � v i · � v i ) and so � b · � v i r i � v i = � v i . v i · � � v i Substituting these values of the r i ’s into the expression for � b W yields the theorem. () Linear Algebra May 5, 2020 5 / 32

Theorem 6.3. Projection Using an Orthogonal Basis Theorem 6.3 (continued) Proof (continued). Taking the dot product of � b with � v i we have � ( � b W + � v i = ( � v i ) + ( � b · � v i = b W ⊥ ) · � b W · � b W ⊥ · � v i ) ( r 1 � v 1 · � v i + r 2 � v 2 · � v i + · · · + r k � v k · � = v i ) + 0 = r i � v i · � v i . Therefore r i = ( � b · � v i ) / ( � v i · � v i ) and so � b · � v i r i � v i = � v i . v i · � � v i Substituting these values of the r i ’s into the expression for � b W yields the theorem. () Linear Algebra May 5, 2020 5 / 32

Page 347 Number 4 Page 347 Number 4 Page 347 Number 4. Consider W = sp([1 , − 1 , 1 , 1] , [ − 1 , 1 , 1 , 1] , [1 , 1 , − 1 , 1]). Verify that the generating set of W is orthogonal and find the projection of � b = [1 , 4 , 1 , 2] on W . Solution. We check pairwise for orthogonality of the three generating vectors: [1 , − 1 , 1 , 1] · [ − 1 , 1 , 1 , 1] = (1)( − 1) + ( − 1)(1) + (1)(1) + (1)(1) = − 1 − 1 + 1 + 1 = 0 , [1 , − 1 , 1 , 1] · [1 , 1 , − 1 , 1] = (1)(1) + ( − 1)(1) + (1)( − 1) + (1)(1) = 1 − 1 − 1 + 1 = 0 , [ − 1 , 1 , 1 , 1] · [1 , 1 , − 1 , 1] = ( − 1)(1) + (1)(1) + (1)( − 1) + (1)(1) = − 1 + 1 − 1 + 1 = 0 . Since each dot product is 0 then the vectors form an orthogonal set (in fact, an orthogonal basis for W , by Theorem 6.2, “Orthogonal Bases”). () Linear Algebra May 5, 2020 6 / 32

Page 347 Number 4 Page 347 Number 4 Page 347 Number 4. Consider W = sp([1 , − 1 , 1 , 1] , [ − 1 , 1 , 1 , 1] , [1 , 1 , − 1 , 1]). Verify that the generating set of W is orthogonal and find the projection of � b = [1 , 4 , 1 , 2] on W . Solution. We check pairwise for orthogonality of the three generating vectors: [1 , − 1 , 1 , 1] · [ − 1 , 1 , 1 , 1] = (1)( − 1) + ( − 1)(1) + (1)(1) + (1)(1) = − 1 − 1 + 1 + 1 = 0 , [1 , − 1 , 1 , 1] · [1 , 1 , − 1 , 1] = (1)(1) + ( − 1)(1) + (1)( − 1) + (1)(1) = 1 − 1 − 1 + 1 = 0 , [ − 1 , 1 , 1 , 1] · [1 , 1 , − 1 , 1] = ( − 1)(1) + (1)(1) + (1)( − 1) + (1)(1) = − 1 + 1 − 1 + 1 = 0 . Since each dot product is 0 then the vectors form an orthogonal set (in fact, an orthogonal basis for W , by Theorem 6.2, “Orthogonal Bases”). () Linear Algebra May 5, 2020 6 / 32

Page 347 Number 4 Page 347 Number 4 (continued) Solution (continued). By Theorem 6.3, “Projection Using an Orthogonal Basis,” we have the projection of � b on W is � � � b · � v 1 b · � v 2 b · � v 3 � b W = proj W ( � � � � b ) = v 1 + v 2 + v 3 � v 1 · � v 1 v 2 · � � v 2 � v 3 · � v 3 where � v 1 ,� v 2 ,� v 3 are the three orthogonal generating vectors, so b W = [1 , 4 , 1 , 2] · [1 , − 1 , 1 , 1] � [1 , − 1 , 1 , 1] · [1 , − 1 , 1 , 1][1 , − 1 , 1 , 1] + [1 , 4 , 1 , 2] · [ − 1 , 1 , 1 , 1] [ − 1 , 1 , 1 , 1] · [ − 1 , 1 , 1 , 1][ − 1 , 1 , 1 , 1] + [1 , 4 , 1 , 2] · [1 , 1 , − 1 , 1] [1 , 1 , − 1 , 1] · [1 , 1 , − 1 , 1][1 , 1 , − 1 , 1] = 0 4[1 , − 1 , 1 , 1] + 6 4[ − 1 , 1 , 1 , ] + 6 4[1 , 1 , − 1 , 1] = 0[1 , − 1 , 1 , 1] + (3 / 2)[ − 1 , 1 , 1 , 1] + (3 / 2)[1 , 1 , − 1 , 1] = [0 , 3 , 0 , 3] . � () Linear Algebra May 5, 2020 7 / 32

Page 347 Number 4 Page 347 Number 4 (continued) Solution (continued). By Theorem 6.3, “Projection Using an Orthogonal Basis,” we have the projection of � b on W is � � � b · � v 1 b · � v 2 b · � v 3 � b W = proj W ( � � � � b ) = v 1 + v 2 + v 3 � v 1 · � v 1 v 2 · � � v 2 � v 3 · � v 3 where � v 1 ,� v 2 ,� v 3 are the three orthogonal generating vectors, so b W = [1 , 4 , 1 , 2] · [1 , − 1 , 1 , 1] � [1 , − 1 , 1 , 1] · [1 , − 1 , 1 , 1][1 , − 1 , 1 , 1] + [1 , 4 , 1 , 2] · [ − 1 , 1 , 1 , 1] [ − 1 , 1 , 1 , 1] · [ − 1 , 1 , 1 , 1][ − 1 , 1 , 1 , 1] + [1 , 4 , 1 , 2] · [1 , 1 , − 1 , 1] [1 , 1 , − 1 , 1] · [1 , 1 , − 1 , 1][1 , 1 , − 1 , 1] = 0 4[1 , − 1 , 1 , 1] + 6 4[ − 1 , 1 , 1 , ] + 6 4[1 , 1 , − 1 , 1] = 0[1 , − 1 , 1 , 1] + (3 / 2)[ − 1 , 1 , 1 , 1] + (3 / 2)[1 , 1 , − 1 , 1] = [0 , 3 , 0 , 3] . � () Linear Algebra May 5, 2020 7 / 32