Lecture 10: Point Clouds, Eigenvectors, PCA COMPSCI/MATH 290-04 Chris Tralie, Duke University 2/16/2016 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Announcements ◮ Group Assignment 1 Part 1, Due This Sunday 11:55 PM ◮ Note on Assignment Difficulty ◮ Hackathon 2/26, 2/27, or 2/28 (upcoming poll) ◮ Choosing final projects before spring break ◮ Visitors Coming... COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Table of Contents ◮ Point Clouds Unit Overview ⊲ Eigenvalues / Eigenvectors ⊲ Principal Component Analysis COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
What Is A Point Cloud? Interactive Demo COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
What’s In This Unit? Point Cloud Registration (Alignment) Courtesy of http://www.pointclouds.org COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
What’s In This Unit? “Shape Google” (Point Cloud Statistics) (my undergrad senior thesis) COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
What’s In This Unit? Symmetries COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
What’s In This Unit? Symmetries COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
What’s In This Unit? Symmetries COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
What’s In This Unit? Surface Reconstruction COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
What’s In This Unit? Surface Reconstruction COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Table of Contents ⊲ Point Clouds Unit Overview ◮ Eigenvalues / Eigenvectors ⊲ Principal Component Analysis COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Ax = λ x A is N × N matrix x is N × 1 column matrix (vector) λ is a scalar called the eigenvalue of x COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Ax = λ x A is N × N matrix x is N × 1 column matrix (vector) λ is a scalar called the eigenvalue of x ⊲ A does not change the direction of x COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: X Scale � 2 � � 0 0 � = 0 1 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: X Scale � 2 � � 0 � 0 0 � � = 0 1 1 1 Yes! λ = 1 Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: X Scale � 2 � � 1 0 � = 0 1 0 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: X Scale � 2 � � 1 � 2 0 � � = 0 1 0 0 Yes! λ = 2 Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: X Scale � 2 � � 1 0 � = 0 1 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: X Scale � 2 � � 1 � 2 0 � � = 0 1 1 1 No, changes direction Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Shear � 1 � � 1 1 � = 0 1 0 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Shear � 1 � � 1 � 1 1 � � = 0 1 0 0 Yes! λ = 1 Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Shear � 1 � � 0 1 � = 0 1 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Shear � 1 � � 0 � 1 1 � � = 0 1 1 1 No, changes direction Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 1 0 � = 0 2 0 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 1 � − 1 0 � � = 0 2 0 0 Yes! λ = − 1 (tricky) Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 0 0 � = 0 2 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 0 � 0 0 � � = 0 2 1 2 Yes! λ = 2 Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 1 0 � = 0 2 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 1 � − 1 0 � � = 0 2 1 2 No, changes direction Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Rotation Matrix What are the eigenvectors/eigenvalues of � cos ( θ ) � − sin ( θ ) A = sin ( θ ) cos ( θ ) ? COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Linear Eigen-Decomposition Eigenvectors/eigenvalues ( λ 1 , v 1 ) , ( λ 2 , v 2 ) Let v = av 1 + bv 2 Then Av = A ( av 1 + bv 2 ) = aAv 1 + bAv 2 Av = λ 1 av 1 + λ 2 bv 2 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Linear Eigen-Decomposition: Example � � 4 − 1 − 1 4 � − 1 � v 1 = , λ 1 = 3 − 1 � − 1 � v 2 = , λ 2 = 5 1 � 0 � Let v = 2 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Spectral Theorem If A is symmetric that is A = A T then all of its eigenvectors are orthogonal COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Spectral Theorem: Example � 1 � 2 2 1 � − 1 � v 1 = , λ 1 = − 1 1 � 1 � v 2 = , λ 2 = 3 1 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Solving for Eigenvalues/Eigenvectors COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Table of Contents ⊲ Point Clouds Unit Overview ⊲ Eigenvalues / Eigenvectors ◮ Principal Component Analysis COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Point Cloud Centroid x i }| N Given a collection of points X = { � i = 1 Centroid � c is the “mean point” That is N c [ k ] = 1 � � x i [ k ] N i = 1 Average each coordinate independently COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Point Cloud Centroid COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Point Cloud Centroid COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance Organize point cloud into N × d matrix, each point along a row � − x 1 − − x 2 � − � − x 3 − X = . . . . . . . . . � − x N − Choose a unit column vector direction u ∈ R d × 1 Then d = Xu gives projections onto u COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance d = Xu gives projections onto u d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u Gives the sum of squared projections onto u COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Example 1000 point example in 2D, centroid is origin COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Some Math d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u = u T Au COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Some Math d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u = u T Au ⊲ Note that A = X T X is symmetric. COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Some Math d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u = u T Au ⊲ Note that A = X T X is symmetric. ⊲ It is also positive semi-definite . All eigenvalues are real and positive COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Directions of Variance: Some Math d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u = u T Au ⊲ Note that A = X T X is symmetric. ⊲ It is also positive semi-definite . All eigenvalues are real and positive ⊲ Let ( v 1 , λ 1 ) , ( v 2 , λ 2 ) , ..., ( v d , λ d ) be the d eigenvalue/eigenvector pairs. Then a direction u can be written as u = a 1 v 1 + a 2 v 2 + . . . + a d v d COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA
Recommend
More recommend