Math 221: LINEAR ALGEBRA §8-6. Singular Value Decomposition Le Chen 1 Emory University, 2020 Fall (last updated on 08/27/2020) Creative Commons License (CC BY-NC-SA) 1 Slides are adapted from those by Karen Seyffarth from University of Calgary.
Singular Value Decomposition (SVD) Definition Let A be an m × n matrix. The singular values of A are the square roots of the nonzero eigenvalues of A T A.
Singular Value Decomposition (SVD) Definition Let A be an m × n matrix. The singular values of A are the square roots of the nonzero eigenvalues of A T A. Singular Value Decomposition (SVD) can be thought of as a generalization of orthogonal diagonalization of a symmetric matrix to an arbitrary m × n matrix.
Singular Value Decomposition (SVD) Definition Let A be an m × n matrix. The singular values of A are the square roots of the nonzero eigenvalues of A T A. Singular Value Decomposition (SVD) can be thought of as a generalization of orthogonal diagonalization of a symmetric matrix to an arbitrary m × n matrix. Given an m × n matrix A, we will see how to express A as a product A = U Σ V T where
Singular Value Decomposition (SVD) Definition Let A be an m × n matrix. The singular values of A are the square roots of the nonzero eigenvalues of A T A. Singular Value Decomposition (SVD) can be thought of as a generalization of orthogonal diagonalization of a symmetric matrix to an arbitrary m × n matrix. Given an m × n matrix A, we will see how to express A as a product A = U Σ V T where ◮ U is an m × m orthogonal matrix whose columns are eigenvectors of AA T .
Singular Value Decomposition (SVD) Definition Let A be an m × n matrix. The singular values of A are the square roots of the nonzero eigenvalues of A T A. Singular Value Decomposition (SVD) can be thought of as a generalization of orthogonal diagonalization of a symmetric matrix to an arbitrary m × n matrix. Given an m × n matrix A, we will see how to express A as a product A = U Σ V T where ◮ U is an m × m orthogonal matrix whose columns are eigenvectors of AA T . ◮ V is an n × n orthogonal matrix whose columns are eigenvectors of A T A.
Singular Value Decomposition (SVD) Definition Let A be an m × n matrix. The singular values of A are the square roots of the nonzero eigenvalues of A T A. Singular Value Decomposition (SVD) can be thought of as a generalization of orthogonal diagonalization of a symmetric matrix to an arbitrary m × n matrix. Given an m × n matrix A, we will see how to express A as a product A = U Σ V T where ◮ U is an m × m orthogonal matrix whose columns are eigenvectors of AA T . ◮ V is an n × n orthogonal matrix whose columns are eigenvectors of A T A. ◮ Σ is an m × n matrix whose only nonzero values lie on its main diagonal, and are the square roots of the eigenvalues of both AA T and A T A.
Singular Value Decomposition (SVD) Definition Let A be an m × n matrix. The singular values of A are the square roots of the nonzero eigenvalues of A T A. Singular Value Decomposition (SVD) can be thought of as a generalization of orthogonal diagonalization of a symmetric matrix to an arbitrary m × n matrix. Given an m × n matrix A, we will see how to express A as a product A = U Σ V T where ◮ U is an m × m orthogonal matrix whose columns are eigenvectors of AA T . ◮ V is an n × n orthogonal matrix whose columns are eigenvectors of A T A. ◮ Σ is an m × n matrix whose only nonzero values lie on its main diagonal, and are the square roots of the eigenvalues of both AA T and A T A. Remark Although we haven’t proved it, A T A and AA T have the same nonzero eigenvalues.
Example � 1 � − 1 3 Let A = . Then 3 1 1 � 1 � 11 � 1 3 − 1 3 5 � AA T = = − 1 1 . 3 1 1 5 11 3 1 � 1 1 3 10 2 6 � − 1 3 A T A = . − 1 1 = 2 2 − 2 3 1 1 3 1 6 − 2 10
Example � 1 � − 1 3 Let A = . Then 3 1 1 � 1 � 11 � 1 3 − 1 3 5 � AA T = = − 1 1 . 3 1 1 5 11 3 1 � 1 1 3 10 2 6 � − 1 3 A T A = . − 1 1 = 2 2 − 2 3 1 1 3 1 6 − 2 10
Example (continued) Since AA T is 2 × 2 while A T A is 3 × 3 , and AA T and A T A have the same nonzero eigenvalues, compute c AA T ( x ) (because it’s easier to compute than c A T A ( x ) ). � � x − 11 − 5 det ( xI − AA T ) = � � c AA T ( x ) = � � − 5 x − 11 � � ( x − 11) 2 − 25 = x 2 − 22 x + 121 − 25 = x 2 − 22 x + 96 = = ( x − 16)( x − 6) . Therefore, the eigenvalues of AA T are λ 1 = 16 and λ 2 = 6 .
so . Since the eigenvalues of . : solve so . : solve need only normalize them. are distinct, the corresponding eigenvectors are orthogonal, and we , fjnd eigenvectors for from largest to smallest). eigenvalues (and corresponding singular values) in nonincreasing order (i.e., Example (continued) The eigenvalues of A T A are λ 1 = 16 , λ 2 = 6 , and λ 3 = 0 , and the singular √ √ values of A are σ 1 = 16 = 4 and σ 2 = 6 . By convention, we list the
so need only normalize them. . : solve so . : solve eigenvalues (and corresponding singular values) in nonincreasing order (i.e., from largest to smallest). Example (continued) The eigenvalues of A T A are λ 1 = 16 , λ 2 = 6 , and λ 3 = 0 , and the singular √ √ values of A are σ 1 = 16 = 4 and σ 2 = 6 . By convention, we list the To find the matrix V , fjnd eigenvectors for A T A . Since the eigenvalues of AA T are distinct, the corresponding eigenvectors are orthogonal, and we
so eigenvalues (and corresponding singular values) in nonincreasing order (i.e., . : solve need only normalize them. from largest to smallest). Example (continued) The eigenvalues of A T A are λ 1 = 16 , λ 2 = 6 , and λ 3 = 0 , and the singular √ √ values of A are σ 1 = 16 = 4 and σ 2 = 6 . By convention, we list the To find the matrix V , fjnd eigenvectors for A T A . Since the eigenvalues of AA T are distinct, the corresponding eigenvectors are orthogonal, and we λ 1 = 16 : solve (16 I − A T A ) � y 1 = � 0 . t 6 − 2 − 6 0 1 0 − 1 0 1 , so � , t ∈ R . → = t − 2 14 2 0 0 1 0 0 y 1 = 0 0 t − 6 2 6 0 0 0 0 0 1
need only normalize them. eigenvalues (and corresponding singular values) in nonincreasing order (i.e., from largest to smallest). Example (continued) The eigenvalues of A T A are λ 1 = 16 , λ 2 = 6 , and λ 3 = 0 , and the singular √ √ values of A are σ 1 = 16 = 4 and σ 2 = 6 . By convention, we list the To find the matrix V , fjnd eigenvectors for A T A . Since the eigenvalues of AA T are distinct, the corresponding eigenvectors are orthogonal, and we λ 1 = 16 : solve (16 I − A T A ) � y 1 = � 0 . t 6 − 2 − 6 0 1 0 − 1 0 1 , so � , t ∈ R . → = t − 2 14 2 0 0 1 0 0 y 1 = 0 0 t − 6 2 6 0 0 0 0 0 1 λ 2 = 6 : solve (6 I − A T A ) � y 2 = � 0 . − 4 − 2 − 6 0 1 0 1 0 − s − 1 , so � = s , s ∈ R . − 2 4 2 0 → 0 1 1 0 y 2 = − s − 1 − 6 2 − 4 0 0 0 0 0 s 1
. Let to fjnd , and , and we use Also, Then Example (continued) λ 3 = 0 : solve ( − A T A ) � y 3 = � 0 . − 10 − 2 − 6 0 1 0 1 0 − r − 1 , so � , r ∈ R . → = r − 2 − 2 2 0 0 1 − 2 0 y 3 = 2 r 2 − 6 2 − 10 0 0 0 0 0 r 1
. Let to fjnd , and , and we use Also, Then Example (continued) λ 3 = 0 : solve ( − A T A ) � y 3 = � 0 . − 10 − 2 − 6 0 1 0 1 0 − r − 1 , so � , r ∈ R . → = r − 2 − 2 2 0 0 1 − 2 0 y 3 = 2 r 2 − 6 2 − 10 0 0 0 0 0 r 1 1 − 1 − 1 1 1 1 ,� ,� . v 1 = � 0 v 2 = − 1 v 3 = 2 √ √ √ 2 3 6 1 1 1 √ √ 3 2 − 1 − 1 √ . V = 0 2 2 √ − 6 √ √ 3 2 1
Also, Then Let Example (continued) λ 3 = 0 : solve ( − A T A ) � y 3 = � 0 . − 10 − 2 − 6 0 1 0 1 0 − r − 1 , so � , r ∈ R . → = r − 2 − 2 2 0 0 1 − 2 0 y 3 = 2 r 2 − 6 2 − 10 0 0 0 0 0 r 1 1 − 1 − 1 1 1 1 ,� ,� . v 1 = � 0 v 2 = − 1 v 3 = 2 √ √ √ 2 3 6 1 1 1 √ √ 3 2 − 1 − 1 √ . V = 0 2 2 √ − 6 √ √ 3 2 1 � 4 � 0 0 Σ = , √ 0 6 0 and we use A , V T , and Σ to fjnd U .
and Then we have . Thus, and which implies that Example (continued) Since V is orthogonal and A = U Σ V T , it follows that AV = U Σ . Let � � � � V = � v 1 � v 2 � v 3 , and let U = � u 1 � u 2 , where � u 1 and � u 2 are the two columns of U .
Recommend
More recommend
