Overview Last time we discussed orthogonal projection. Well review - PowerPoint PPT Presentation

Overview Last time we discussed orthogonal projection. We’ll review this today before discussing the question of how to find an orthonormal basis for a given subspace. From Lay, §6.4 Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 1 / 24

Orthogonal projection Given a subspace W of R n , you can write any vector y ∈ R n as y = ˆ y + z = proj W y + proj W ⊥ y , y ∈ W is the closest vector in W to y and z ∈ W ⊥ . We call ˆ where ˆ y the orthogonal projection of y onto W . Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 2 / 24

Orthogonal projection Given a subspace W of R n , you can write any vector y ∈ R n as y = ˆ y + z = proj W y + proj W ⊥ y , y ∈ W is the closest vector in W to y and z ∈ W ⊥ . We call ˆ where ˆ y the orthogonal projection of y onto W . Given an orthogonal basis { u 1 , . . . , u p } for W , we have a formula to compute ˆ y : y = y · u 1 u 1 + · · · + y · u p ˆ u p . u 1 · u 1 u p · u p Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 2 / 24

Orthogonal projection Given a subspace W of R n , you can write any vector y ∈ R n as y = ˆ y + z = proj W y + proj W ⊥ y , y ∈ W is the closest vector in W to y and z ∈ W ⊥ . We call ˆ where ˆ y the orthogonal projection of y onto W . Given an orthogonal basis { u 1 , . . . , u p } for W , we have a formula to compute ˆ y : y = y · u 1 u 1 + · · · + y · u p ˆ u p . u 1 · u 1 u p · u p If we also had an orthogonal basis { u p +1 , . . . , u n } for W ⊥ , we could find z by projecting y onto W ⊥ : y · u p +1 u p +1 + · · · + y · u n z = u n . u p +1 · u p +1 u n · u n However, once we subtract off the projection of y to W , we’re left with z ∈ W ⊥ . We’ll make heavy use of this observation today. Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 2 / 24

Orthonormal bases In the case where we have an orthonormal basis { u 1 , . . . , u p } for W , the computations are made even simpler: ˆ y = ( y · u 1 ) u 1 + ( y · u 2 ) u 2 + · · · + ( y · u p ) u p . Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 3 / 24

Orthonormal bases In the case where we have an orthonormal basis { u 1 , . . . , u p } for W , the computations are made even simpler: ˆ y = ( y · u 1 ) u 1 + ( y · u 2 ) u 2 + · · · + ( y · u p ) u p . If U = { u 1 , . . . , u p } is an orthonormal basis for W and U is the matrix whose columns are the u i , then UU T y = ˆ y U T U = I p Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 3 / 24

The Gram Schmidt Process The aim of this section is to find an orthogonal basis { v 1 , . . . , v n } for a subspace W when we start with a basis { x 1 , . . . , x n } that is not orthogonal. Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 4 / 24

The Gram Schmidt Process The aim of this section is to find an orthogonal basis { v 1 , . . . , v n } for a subspace W when we start with a basis { x 1 , . . . , x n } that is not orthogonal. Start with v 1 = x 1 . Now consider x 2 . If v 1 and x 2 are not orthogonal, we’ll modify x 2 so that we get an orthogonal pair v 1 , v 2 satisfying Span { x 1 , x 2 } = Span { v 1 , v 2 } . Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 4 / 24

The Gram Schmidt Process The aim of this section is to find an orthogonal basis { v 1 , . . . , v n } for a subspace W when we start with a basis { x 1 , . . . , x n } that is not orthogonal. Start with v 1 = x 1 . Now consider x 2 . If v 1 and x 2 are not orthogonal, we’ll modify x 2 so that we get an orthogonal pair v 1 , v 2 satisfying Span { x 1 , x 2 } = Span { v 1 , v 2 } . Then we modify x 3 so get v 3 satisfying v 1 · v 3 = v 2 · v 3 = 0 and Span { x 1 , x 2 , x 3 } = Span { v 1 , v 2 , v 3 } . Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 4 / 24

The Gram Schmidt Process The aim of this section is to find an orthogonal basis { v 1 , . . . , v n } for a subspace W when we start with a basis { x 1 , . . . , x n } that is not orthogonal. Start with v 1 = x 1 . Now consider x 2 . If v 1 and x 2 are not orthogonal, we’ll modify x 2 so that we get an orthogonal pair v 1 , v 2 satisfying Span { x 1 , x 2 } = Span { v 1 , v 2 } . Then we modify x 3 so get v 3 satisfying v 1 · v 3 = v 2 · v 3 = 0 and Span { x 1 , x 2 , x 3 } = Span { v 1 , v 2 , v 3 } . We continue this process until we’ve built a new orthogonal basis for W . Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 4 / 24

Example 1     1 2 Suppose that W = Span { x 1 , x 2 } where x 1 = 1  and x 2 = 2  . Find an       0 3 orthogonal basis { v 1 , v 2 } for W . To start the process we put v 1 = x 1 . We then find     1 2 y = proj v 1 x 2 = x 2 · v 1 v 1 = 4 ˆ 1  = 2  .     v 1 · v 1 2   0 0 Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 5 / 24

Now we define v 2 = x 2 − ˆ y ; this is orthogonal to x 1 = v 1 :       2 2 0 v 2 = x 2 − x 2 · v 1 v 1 = x 2 − ˆ y = 2  − 2  = 0  .       v 1 · v 1    3 0 3 So v 2 is the component of x 2 orthogonal to x 1 . Note that v 2 is in W = Span { x 1 , x 2 } because it is a linear combination of v 1 = x 1 and x 2 . So we have that       1 0     v 1 = 1  , v 2 = 0         0 3    is an orthogonal basis for W . Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 6 / 24

Example 2 Suppose that { x 1 , x 2 , x 3 } is a basis for a subspace W of R 4 . Describe an orthogonal basis for W . • As in the previous example, we put v 2 = x 2 − x 2 · v 1 v 1 = x 1 and v 1 . v 1 · v 1 Then { v 1 , v 2 } is an orthogonal basis for W 2 =Span { x 1 , x 2 } = Span { v 1 , v 2 } . • Now proj W 2 x 3 = x 3 · v 1 v 1 + x 3 · v 2 v 2 and v 1 · v 1 v 2 · v 2 v 3 = x 3 − proj W 2 x 3 = x 3 − x 3 · v 1 v 1 − x 3 · v 2 v 2 v 1 · v 1 v 2 · v 2 is the component of x 3 orthogonal to W 2 . Furthermore, v 3 is in W because it is a linear combination of vectors in W . • Thus we obtain that { v 1 , v 2 , v 3 } is an orthogonal basis for W . Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 7 / 24

Theorem (The Gram Schmidt Process) Given a basis { x 1 , x 2 , . . . , x p } for a subspace W of R n , define v 1 = x 1 x 2 − x 2 · v 1 v 2 = v 1 v 1 · v 1 x 3 − x 3 · v 1 v 1 − x 3 · v 2 v 3 = v 2 v 1 · v 1 v 2 · v 2 . . . x p − x p · v 1 x p · v p − 1 = v 1 − . . . − v p v p − 1 v 1 · v 1 v p − 1 · v p − 1 Then { v 1 , . . . , v p } is an orthogonal basis for W . Also Span { v 1 , . . . , v k } = Span { x 1 , . . . , x k } for 1 ≤ k ≤ p . Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 8 / 24

Example 3 The vectors     3 − 3 x 1 = − 4  , x 2 = 14        5 − 7 form a basis for a subspace W . Use the Gram-Schmidt process to produce an orthogonal basis for W . Step 1 Put v 1 = x 1 . Step 2 x 2 − x 2 · v 1 = v 2 v 1 v 1 · v 1       − 3 3 3  − ( − 100) = 14 − 4  = 6  .       50    − 7 5 3 Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 9 / 24

Then { v 1 , v 2 } is an orthogonal basis for W . To construct an orthonormal basis for W we normalise the basis { v 1 , v 2 } :   3 1 1 √ u 1 = � v 1 � v 1 = − 4   50   5     3 1 1 1 1 √ √ u 2 = � v 2 � v 2 = 6  = 2     54  6   3 1 Then { u 1 , u 2 } is an orthonormal basis for W . Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 10 / 24

Example 4   − 1 6 6 3 − 8 3   Let A =  . Use the Gram-Schmidt process to find an   1 − 2 6    1 − 4 3 orthogonal basis for the column space of A . Let x 1 , x 2 , x 3 be the three columns of A .   − 1 3   Step 1 Put v 1 = x 1 =  .   1    1 Step 2       6 − 1 3 x 2 − x 2 · v 1 − 8  − ( − 36) 3 1       = v 1 =  =  . v 2       − 2 1 1 v 1 · v 1   12        − 4 1 − 1 Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 11 / 24

Step 3 x 3 − x 3 · v 1 v 1 − x 3 · v 2 = v 3 v 2 v 1 · v 1 v 2 · v 2       6 − 1 3  − 12  − 24 3 3 1       =       6 1 1   12   12       3 1 − 1   1 − 2   =  .   3    4 Thus an orthogonal basis for the column space of A is given by         − 1 3 1      3 1 − 2           ,  , .       1 1 3                 1 − 1 4   Dr Scott Morrison (ANU) MATH1014 Notes Second Semester 2015 12 / 24