CS475/CS675 Lecture 23: July 19, 2016 Principal Component Analysis, - PowerPoint PPT Presentation

CS475/CS675 Lecture 23: July 19, 2016 Principal Component Analysis, Eigenfaces CS475/CS675 (c) 2016 P. Poupart 1

Principal Component Analysis (PCA) • Data exploration technique: – Dimensionality reduction – Principal components are axes that preserve most of the variance in the data • Picture CS475/CS675 (c) 2016 P. Poupart 2

Empirical Variance • Data: � � • Empirical mean: � ��� � • Empirical covariance: � � � � � ��� ��� CS475/CS675 (c) 2016 P. Poupart 3

Principal Component • Axis that preserves the most variance � � � � � � � ��� � � ��� � � � � � ��� � � ��� � � � � � ��� � � ��� � � � When , then � is � is CS475/CS675 (c) 2016 P. Poupart 4

Principal Component Analysis • Eigendecomposition of the empirical covariance matrix � � �Λ� � • Eigenvector: dimension (or basis function) • Eigenvalue: amount of variance preserved in that dimension CS475/CS675 (c) 2016 P. Poupart 5

Dimensionality Reduction • Problem: what is the smallest linear subspace (i.e., fewest dimensions) that captures 95% of the variance? • Solution: retain eigenvectors of the � largest eigenvalues such that � is the smallest integer that satisfies � ∑ � � ��� � 0.95 � ∑ � � ��� • Picture: CS475/CS675 (c) 2016 P. Poupart 6

Example: Eigenfaces • Turk and Pentland (1991): – Image compression – Face detection • Solution: – embed images in low dimensional eigenspace – Face detection: nearest neighbour in eigenspace CS475/CS675 (c) 2016 P. Poupart 7

Principal Component Analysis • Data: ( pixels images) • Covariance matrix: ( ) � • Eigendecomposition: CS475/CS675 (c) 2016 P. Poupart 8

Eigenfaces Dataset Mean Eigenfaces image CS475/CS675 (c) 2016 P. Poupart 9

Face Detection • Project each image in the space if eigenfaces – I.e., approximate each image as a linear combination of the eigenfaces � � � • Face detection: find matching image in a database – Nearest neighbour in space of eigenfaces � ∗ � ������ � � � � � � � � CS475/CS675 (c) 2016 P. Poupart 10