lecture 10 point clouds eigenvectors pca
play

Lecture 10: Point Clouds, Eigenvectors, PCA COMPSCI/MATH 290-04 - PowerPoint PPT Presentation

Lecture 10: Point Clouds, Eigenvectors, PCA COMPSCI/MATH 290-04 Chris Tralie, Duke University 2/16/2016 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA Announcements Group Assignment 1 Part 1, Due This Sunday 11:55 PM


  1. Lecture 10: Point Clouds, Eigenvectors, PCA COMPSCI/MATH 290-04 Chris Tralie, Duke University 2/16/2016 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  2. Announcements ◮ Group Assignment 1 Part 1, Due This Sunday 11:55 PM ◮ Note on Assignment Difficulty ◮ Hackathon 2/26, 2/27, or 2/28 (upcoming poll) ◮ Choosing final projects before spring break ◮ Visitors Coming... COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  3. Table of Contents ◮ Point Clouds Unit Overview ⊲ Eigenvalues / Eigenvectors ⊲ Principal Component Analysis COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  4. What Is A Point Cloud? Interactive Demo COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  5. What’s In This Unit? Point Cloud Registration (Alignment) Courtesy of http://www.pointclouds.org COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  6. What’s In This Unit? “Shape Google” (Point Cloud Statistics) (my undergrad senior thesis) COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  7. What’s In This Unit? Symmetries COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  8. What’s In This Unit? Symmetries COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  9. What’s In This Unit? Symmetries COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  10. What’s In This Unit? Surface Reconstruction COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  11. What’s In This Unit? Surface Reconstruction COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  12. Table of Contents ⊲ Point Clouds Unit Overview ◮ Eigenvalues / Eigenvectors ⊲ Principal Component Analysis COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  13. Eigenvalues/Eigenvectors Ax = λ x A is N × N matrix x is N × 1 column matrix (vector) λ is a scalar called the eigenvalue of x COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  14. Eigenvalues/Eigenvectors Ax = λ x A is N × N matrix x is N × 1 column matrix (vector) λ is a scalar called the eigenvalue of x ⊲ A does not change the direction of x COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  15. Eigenvalues/Eigenvectors Example: X Scale � 2 � � 0 0 � = 0 1 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  16. Eigenvalues/Eigenvectors Example: X Scale � 2 � � 0 � 0 0 � � = 0 1 1 1 Yes! λ = 1 Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  17. Eigenvalues/Eigenvectors Example: X Scale � 2 � � 1 0 � = 0 1 0 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  18. Eigenvalues/Eigenvectors Example: X Scale � 2 � � 1 � 2 0 � � = 0 1 0 0 Yes! λ = 2 Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  19. Eigenvalues/Eigenvectors Example: X Scale � 2 � � 1 0 � = 0 1 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  20. Eigenvalues/Eigenvectors Example: X Scale � 2 � � 1 � 2 0 � � = 0 1 1 1 No, changes direction Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  21. Eigenvalues/Eigenvectors Example: Shear � 1 � � 1 1 � = 0 1 0 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  22. Eigenvalues/Eigenvectors Example: Shear � 1 � � 1 � 1 1 � � = 0 1 0 0 Yes! λ = 1 Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  23. Eigenvalues/Eigenvectors Example: Shear � 1 � � 0 1 � = 0 1 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  24. Eigenvalues/Eigenvectors Example: Shear � 1 � � 0 � 1 1 � � = 0 1 1 1 No, changes direction Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  25. Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 1 0 � = 0 2 0 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  26. Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 1 � − 1 0 � � = 0 2 0 0 Yes! λ = − 1 (tricky) Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  27. Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 0 0 � = 0 2 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  28. Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 0 � 0 0 � � = 0 2 1 2 Yes! λ = 2 Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  29. Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 1 0 � = 0 2 1 Before COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  30. Eigenvalues/Eigenvectors Example: Flip X, Scale Y � − 1 � � 1 � − 1 0 � � = 0 2 1 2 No, changes direction Before After COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  31. Rotation Matrix What are the eigenvectors/eigenvalues of � cos ( θ ) � − sin ( θ ) A = sin ( θ ) cos ( θ ) ? COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  32. Linear Eigen-Decomposition Eigenvectors/eigenvalues ( λ 1 , v 1 ) , ( λ 2 , v 2 ) Let v = av 1 + bv 2 Then Av = A ( av 1 + bv 2 ) = aAv 1 + bAv 2 Av = λ 1 av 1 + λ 2 bv 2 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  33. Linear Eigen-Decomposition: Example � � 4 − 1 − 1 4 � − 1 � v 1 = , λ 1 = 3 − 1 � − 1 � v 2 = , λ 2 = 5 1 � 0 � Let v = 2 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  34. Spectral Theorem If A is symmetric that is A = A T then all of its eigenvectors are orthogonal COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  35. Spectral Theorem: Example � 1 � 2 2 1 � − 1 � v 1 = , λ 1 = − 1 1 � 1 � v 2 = , λ 2 = 3 1 COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  36. Solving for Eigenvalues/Eigenvectors COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  37. Table of Contents ⊲ Point Clouds Unit Overview ⊲ Eigenvalues / Eigenvectors ◮ Principal Component Analysis COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  38. Point Cloud Centroid x i }| N Given a collection of points X = { � i = 1 Centroid � c is the “mean point” That is N c [ k ] = 1 � � x i [ k ] N i = 1 Average each coordinate independently COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  39. Point Cloud Centroid COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  40. Point Cloud Centroid COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  41. Directions of Variance Organize point cloud into N × d matrix, each point along a row  �  − x 1 − − x 2 � −     � − x 3 − X =    .  .   . . . . . . .   � − x N − Choose a unit column vector direction u ∈ R d × 1 Then d = Xu gives projections onto u COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  42. Directions of Variance d = Xu gives projections onto u d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u Gives the sum of squared projections onto u COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  43. Directions of Variance: Example 1000 point example in 2D, centroid is origin COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  44. Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  45. Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  46. Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  47. Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  48. Directions of Variance: Example COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  49. Directions of Variance: Some Math d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u = u T Au COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  50. Directions of Variance: Some Math d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u = u T Au ⊲ Note that A = X T X is symmetric. COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  51. Directions of Variance: Some Math d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u = u T Au ⊲ Note that A = X T X is symmetric. ⊲ It is also positive semi-definite . All eigenvalues are real and positive COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

  52. Directions of Variance: Some Math d T d = ( Xu ) T ( Xu ) = u T ( X T X ) u = u T Au ⊲ Note that A = X T X is symmetric. ⊲ It is also positive semi-definite . All eigenvalues are real and positive ⊲ Let ( v 1 , λ 1 ) , ( v 2 , λ 2 ) , ..., ( v d , λ d ) be the d eigenvalue/eigenvector pairs. Then a direction u can be written as u = a 1 v 1 + a 2 v 2 + . . . + a d v d COMPSCI/MATH 290-04 Lecture 10: Point Clouds, Eigenvectors, PCA

Recommend


More recommend