Final exam: 68:50pm in Clough 152 Cumulative final covers the whole - PowerPoint PPT Presentation

Announcements Monday, December 04 ◮ Final exam: 6–8:50pm in Clough 152 ◮ Cumulative final covers the whole class pretty evenly. ◮ About twice as long as a midterm. ◮ Common for all 1553 sections; written collaboratively. ◮ Studying resources for the final: ◮ Practice final. ◮ Extra general practice problems posted on the website. ◮ Problems on midterms and practice midterms. ◮ Reference sheet. ◮ Early draft of Dan’s and my textbook. ◮ Problems in Lay. ◮ Reading day: is 1–3pm on December 6 in Clough 144 and 152. ◮ Double Rabinoffice hours: Monday, 12–2pm; Tuesday, 9–11am; Thursday, 10–12pm; Friday, 2–4pm. ◮ Please fill out your CIOS survey! ◮ 80% response rate by 11:59pm on Thursday extra dropped quiz

Review for the Final Exam Selected Topics

Orthogonal Sets Definition A set of nonzero vectors is orthogonal if each pair of vectors is orthogonal. It is orthonormal if, in addition, each vector is a unit vector.         1 1 1    ,  ,     Example: B 1 = 1 − 2 0  is not orthogonal.  1 1 1         1 1 1     ,   ,   1 − 2 0 Example: B 2 =  is orthogonal but not orthonormal.  1 1 − 1         1 1 1   1 1 1  ,  ,     Example: B 3 = √ 1 √ − 2 √ 0  is orthonormal.  3 6 2 1 1 − 1 To go from an orthogonal set { u 1 , u 2 , . . . , u m } to an orthonormal set, replace each u i with u i / � u i � . Theorem An orthogonal set is linearly independent. In particular, it is a basis for its span.

Orthogonal Projection Let W be a subspace of R n , and let B = { u 1 , u 2 , . . . , u m } be an orthogonal basis for W . The orthogonal projection of a vector x onto W is � m u i · u i u i = x · u 1 x · u i u 1 · u 1 u 1 + x · u 2 u 2 · u 2 u 2 + · · · + x · u m def proj W ( x ) = u m · u m u m . i =1 This is the closest vector to x that lies on W . In other words, the difference x − proj W ( x ) is perpendicular to W : it is in W ⊥ . Notation: x W = proj W ( x ) x W ⊥ = x − proj W ( x ) . So x W is in W , x W ⊥ is in W ⊥ , and x = x W + x W ⊥ . x x x W ⊥ x − proj W ( x ) x W W W proj W ( x )

Orthogonal Projection Special cases Special case: If x is in W , then x = proj W ( x ), so x = x · u 1 u 1 · u 1 u 1 + x · u 2 u 2 · u 2 u 2 + · · · + x · u m u m · u m u m . In other words, the B -coordinates of x are � x · u 1 � u 1 · u 1 , x · u 2 u 1 · u 2 , . . . , x · u m , u 1 · u m where B = { u 1 , u 2 , . . . , u m } , an orthogonal basis for W . Special case: If W = L is a line, then L = Span { u } for some nonzero vector u , and proj L ( x ) = x · u u · u u x x L ⊥ u x L = proj L ( x ) L

Orthogonal Projection And matrices Let W be a subspace of R n . Theorem The orthogonal projection proj W is a linear transformation from R n to R n . Its range is W . If A is the matrix for proj W , then A 2 = A because projecting twice is the same as projecting once: proj W ◦ proj W = proj W . Theorem The only eigenvalues of A are 1 and 0. Why? ⇒ A 2 v = A ( Av ) = A ( λ v ) = λ ( Av ) = λ 2 v . Av = λ v = So if λ is an eigenvalue of A , then λ 2 is an eigenvalue of A 2 . But A 2 = A , so λ 2 = λ , and hence λ = 0 or 1. The 1-eigenspace of A is W , and the 0-eigenspace is W ⊥ .

The Gram–Schmidt Process The Gram–Schmidt Process Let { v 1 , v 2 , . . . , v m } be a basis for a subspace W of R n . Define: 1. u 1 = v 1 = v 2 − v 2 · u 1 2. u 2 = v 2 − proj Span { u 1 } ( v 2 ) u 1 · u 1 u 1 = v 3 − v 3 · u 1 u 1 · u 1 u 1 − v 3 · u 2 3. u 3 = v 3 − proj Span { u 1 , u 2 } ( v 3 ) u 2 · u 2 u 2 . . . m − 1 � v m · u i m. u m = v m − proj Span { u 1 , u 2 ,..., u m − 1 } ( v m ) = v m − u i · u i u i i =1 Then { u 1 , u 2 , . . . , u m } is an orthogonal basis for the same subspace W . In fact, for each i , Span { u 1 , u 2 , . . . , u i } = Span { v 1 , v 2 , . . . , v i } . Note if v i is in Span { v 1 , v 2 , . . . , v i − 1 } = Span { u 1 , u 2 , . . . , u i − 1 } , then v i = proj Span { u 1 , u 2 ,..., u i − 1 } ( v i ), so u i = 0. So this also detects linear dependence.

Subspaces Definition A subspace of R n is a subset V of R n satisfying: 1. The zero vector is in V . “not empty” 2. If u and v are in V , then u + v is also in V . “closed under addition” 3. If u is in V and c is in R , then cu is in V . “closed under × scalars” Examples: ◮ Any Span { v 1 , v 2 , . . . , v m } . ◮ The column space of a matrix: Col A = Span { columns of A } . ◮ The range of a linear transformation (same as above). � � ◮ The null space of a matrix: Nul A = x | Ax = 0 . ◮ The row space of a matrix: Row A = Span { rows of A } . ◮ The λ -eigenspace of a matrix, where λ is an eigenvalue. ◮ The orthogonal complement W ⊥ of a subspace W . ◮ The zero subspace { 0 } . ◮ All of R n .

Subspaces and Bases Definition Let V be a subspace of R n . A basis of V is a set of vectors { v 1 , v 2 , . . . , v m } in R n such that: 1. V = Span { v 1 , v 2 , . . . , v m } , and 2. { v 1 , v 2 , . . . , v m } is linearly independent. The number of vectors in a basis is the dimension of V , and is written dim V . Every subspace has a basis, so every subspace is a span. But subspaces have many different bases, and some might be better than others. For instance, Gram–Schmidt takes a basis and produces an orthogonal basis. Or, diagonalization produces a basis of eigenvectors of a matrix. How do I know if a subset V is a subspace or not? ◮ Can you write V as one of the examples on the previous slide? ◮ If not, does it satisfy the three defining properties? Note on subspaces versus subsets: A subset of R n is any collection of vectors whatsoever. Like, the unit circle in R 2 , or all vectors with whole-number coefficients. A subspace is a subset that satisfies three additional properties. Most subsets are not subspaces.

Similarity Definition Two n × n matrices A and B are similar if there is an invertible n × n matrix P such that A = PBP − 1 . Important Facts: 1. Similar matrices have the same characteristic polynomial. 2. It follows that similar matrices have the same eigenvalues. 3. If A is similar to B and B is similar to C , then A is similar to C . Caveats: 1. Matrices with the same characteristic polynomial need not be similar. 2. Similarity has nothing to do with row equivalence. 3. Similar matrices usually do not have the same eigenvectors.

Similarity Geometric meaning Let A = PBP − 1 , and let v 1 , v 2 , . . . , v n be the columns of P . These form a basis B for R n because P is invertible. Key relation: for any vector x in R n , [ Ax ] B = B [ x ] B . This says: A acts on the usual coordinates of x in the same way that B acts on the B -coordinates of x . Example: � 5 � � 2 � � 1 � A = 1 3 0 1 B = P = . 4 3 5 0 1 / 2 1 − 1 Then A = PBP − 1 . B acts on the usual coordinates by scaling the first coordinate by 2, and the second by 1 / 2: � 2 x 1 � x 1 � � B = . x 2 / 2 x 2 The unit coordinate vectors are eigenvectors: e 1 has eigenvalue 2, and e 2 has eigenvalue 1 / 2.

Similarity Example � 5 � 2 � 1 � � � A = 1 3 0 1 B = P = [ Ax ] B = B [ x ] B . 3 5 0 1 / 2 1 − 1 4 �� 1 � � 1 �� � 1 � � 1 � In this case, B = , . Let v 1 = and v 2 = . 1 − 1 1 − 1 � 2 � To compute y = Ax : Say x = . 0 � 1 � 1. Find [ x ] B . 1. x = v 1 + v 2 so [ x ] B = . 1 � 1 � � 2 � 2. [ y ] B = B [ x ] B . 2. [ y ] B = B = . 1 1 / 2 � 5 / 2 � 3. Compute y from [ y ] B . 3. y = 2 v 1 + 1 2 v 2 = . 3 / 2 Picture: Av 1 v 1 A Ax x A scales the v 1 - coordinate by Av 2 v 2 2, and the v 2 - coordinate by 1 2 .

Consistent and Inconsistent Systems Definition A matrix equation Ax = b is consistent if it has a solution, and inconsistent otherwise. If A has columns v 1 , v 2 , . . . , v n , then     x 1 | | | .     . b = Ax = v 1 v 2 · · · v m  = x 1 v 1 + x 2 v 2 + · · · + x n v n .  . | | | x n So if Ax = b has a solution, then b is a linear combination of v 1 , v 2 , . . . , v n , and conversely. Equivalently, b is in Span { v 1 , v 2 , . . . , v n } = Col A . Important Ax = b is consistent if and only if b is in Col A .

Least-Squares Solutions Suppose that Ax = b is in consistent. Let � b = proj Col A ( b ) be the closest vector x = � for which A � b does have a solution. Definition x = � A solution to A � b is a least squares solution to Ax = b . This is the solution � x for which A � x is closest to b (with respect to the usual notion of distance in R n ). Theorem The least-squares solutions to Ax = b are the solutions to A T A � x = A T b . If A has orthogonal columns u 1 , u 2 , . . . , u n , then the least-squares solution is � x · u 1 � u 1 · u 1 , x · u 2 x · u m � x = u 2 · u 2 , · · · , u m · u m because b = x · u 1 u 1 · u 1 u 1 + x · u 2 u 2 · u 2 u 2 + · · · + x · u m x = � u m · u m u m . A �