Lecture 12: SVD, Procrustes Analysis COMPSCI/MATH 290-04 Chris Tralie, Duke University 2/23/2016 COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Announcements ⊲ Group Assignment 1 Full Submission Due next Tuesday 11:55 PM ⊲ Hackathon Saturday 2/27 4:00 PM - 10:00 PM Gross Hall 330 ⊲ Rank Top 3 Final Project Choices By Next Wednesday 3/2 COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Table of Contents ◮ Ray Tracing Special Case ⊲ PCA Review ⊲ Singular Value Decomposition ⊲ Procrustes Distance ⊲ Final Projects COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Ray Tracing Special Case COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Ray Tracing Special Case COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Ray Tracing Special Case COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Ray Tracing Special Case s Face 1 s2 s1 Face 2 r COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Table of Contents ⊲ Ray Tracing Special Case ◮ PCA Review ⊲ Singular Value Decomposition ⊲ Procrustes Distance ⊲ Final Projects COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
PCA Review Organize point cloud into N × d matrix, each point along a column | | . . . | . . X = � � � v 1 v 2 . v N | | | . . . Choose a unit column vector direction u ∈ R d × 1 Then d = u T X gives projections onto u COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
PCA Review Organize point cloud into N × d matrix, each point along a column | | . . . | . . X = � � � v 1 v 2 . v N | | | . . . Choose a unit column vector direction u ∈ R d × 1 Then d = u T X gives projections onto u ⊲ More consistent with what we’ve done; points in columns COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
PCA New Convention d = u T X COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
PCA New Convention d = u T X ⊲ How to express the sum of the squares of the dot products? COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
PCA New Convention d = u T X ⊲ How to express the sum of the squares of the dot products? dd T COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
PCA New Convention d = u T X ⊲ How to express the sum of the squares of the dot products? dd T dd T = ( u T X )( u T X ) T = u T XX T u Want to find u that maximizes the above quadratic form COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
PCA New Convention Use eigenvectors of A = XX T to find principal directions maximizing u T Au λ 1 = 422 COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
PCA Review Use eigenvectors of A = XX T to find principal directions maximizing u T Au λ 2 = 21 . 6 COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Table of Contents ⊲ Ray Tracing Special Case ⊲ PCA Review ◮ Singular Value Decomposition ⊲ Procrustes Distance ⊲ Final Projects COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Orthogonal Matrices / Rotation Review � cos ( θ ) � � x − sin ( θ ) � sin ( θ ) cos ( θ ) y Y cos( θ ) sin( θ ) θ θ X cos( θ ) -sin( θ ) COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Orthogonal Matrices / Rotation Review Inverse rotation: dot product interpretation � cos ( θ ) � � x � sin ( θ ) − sin ( θ ) cos ( θ ) y Y cos( θ ) sin( θ ) θ θ X cos( θ ) -sin( θ ) COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Orthogonal Matrices / Rotation Review In general . . | | . | � � � R = u 1 u 2 . . . u N . . | | . | u i · � � u j = 1 , i = j u i · � � u j = 0 , i � = j In 3D, u 1 × � � u 2 = � u 3 for a pure rotation COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Orthogonal Matrices / Rotation Review . . | | . | � � � R = u 1 u 2 . . . u N . . | | . | − u 1 � − � − u 2 − R T = . . . . . . . . . − � − u N u i · � � u j = 1 , i = j u i · � � u j = 0 , i � = j R T R = RR T = I COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition Given an m × n matrix A , the SVD of A is A = USV T ⊲ U is an M × M rotation matrix ⊲ S is an M × N matrix, where S ij = 0 i � = j ⊲ V is an N × N rotation matrix s 1 0 0 . . . 0 � − v 1 − . . 0 s 2 0 0 . . . | | . | � − v 2 − 0 0 s 3 . . . 0 � � � A = u 1 u 2 . . . u M . . . . . . . . . . . . . . . . . . . . . . . . . . | | . | � − v N − 0 0 . . . s M . . . 0 COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition A = USV T s 1 0 0 . . . 0 − � − v 1 . . 0 s 2 0 0 . . . | | . | � − v 2 − 0 0 s 3 . . . 0 � � � A = u 1 u 2 . . . u M . . . . . . . . . . . . . . . . . . . . . . . . . . | | . | � − v N − 0 0 . . . s M . . . 0 COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition A = USV T s 1 0 0 . . . 0 − � − v 1 . . 0 s 2 0 0 . . . | | . | � − v 2 − 0 0 s 3 . . . 0 � � � A = u 1 u 2 . . . u M . . . . . . . . . . . . . . . . . . . . . . . . . . | | . | � − v N − 0 0 . . . s M . . . 0 ⊲ s 1 > s 2 > s 3 > ... > s M COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition A = USV T s 1 0 0 . . . 0 − � − v 1 . . 0 s 2 0 0 . . . | | . | � − v 2 − 0 0 s 3 . . . 0 � � � A = u 1 u 2 . . . u M . . . . . . . . . . . . . . . . . . . . . . . . . . | | . | � − v N − 0 0 . . . s M . . . 0 ⊲ s 1 > s 2 > s 3 > ... > s M ⊲ U holds the eigenvectors of AA T COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition A = USV T s 1 0 0 . . . 0 − � − v 1 . . 0 s 2 0 0 . . . | | . | � − v 2 − 0 0 s 3 . . . 0 � � � A = u 1 u 2 . . . u M . . . . . . . . . . . . . . . . . . . . . . . . . . | | . | � − v N − 0 0 . . . s M . . . 0 ⊲ s 1 > s 2 > s 3 > ... > s M ⊲ U holds the eigenvectors of AA T ⊲ V holds the eigenvectors of A T A COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition A = USV T s 1 0 0 . . . 0 − � − v 1 . . 0 s 2 0 0 . . . | | . | � − v 2 − 0 0 s 3 . . . 0 � � � A = u 1 u 2 . . . u M . . . . . . . . . . . . . . . . . . . . . . . . . . | | . | � − v N − 0 0 . . . s M . . . 0 ⊲ s 1 > s 2 > s 3 > ... > s M ⊲ U holds the eigenvectors of AA T ⊲ V holds the eigenvectors of A T A ⊲ Each s is the square root of corresponding eigenvalue of AA T and A T A (they’re the same!) COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition: Example Ax T V x T U SV x T S V x COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition: Example Ax T V x T U SV x T S V x COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition: Example Ax T T V x U SV x T S V x COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition → PCA A = USV T ⊲ s 1 > s 2 > s 3 > ... > s M ⊲ U holds the eigenvectors of AA T ⊲ V holds the eigenvectors of A T A ⊲ Each s is the square root of corresponding eigenvalue of AA T and A T A Let X be a 3 × N matrix of points along columns. Can we use SVD( X ) to do PCA? COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Singular Value Decomposition → PCA X = USV T ⊲ Columns of U give principal components ⊲ Squares of corresponding S gives sum of squared magnitudes along directions of U ⊲ Coordinates along U directions? COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Table of Contents ⊲ Ray Tracing Special Case ⊲ PCA Review ⊲ Singular Value Decomposition ◮ Procrustes Distance ⊲ Final Projects COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Procrustes Distance http://www.procrustes.nl/gif/illustr.gif COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Procrustes Alignment COMPSCI/MATH 290-04 Lecture 12: SVD, Procrustes Analysis
Recommend
More recommend
