Face identification Lecture: Face Recognition Juan Carlos Niebles and Ranjay Krishna Stanford Vision and Learning Lab 05-Nov-2019 1 St Stanfor ord University
CS 131 Roadmap Face identification Pixels Segments Images Videos Web Recognition Neural networks Convolutions Resizing Motion Detection Convolutional Edges Segmentation Tracking Machine learning neural networks Descriptors Clustering 05-Nov-2019 2 St Stanfor ord University
Let’s recap • A simple object recognition pipeline with kNN • PCA Face identification 05-Nov-2019 3 St Stanfor ord University
Object recognition: a classification framework • Apply a prediction function to a feature representation of Face identification the image to get the desired output: f( ) = “ apple ” f( ) = “ tomato ” 05-Nov-2019 f( ) = “ cow ” 4 Dataset: ETH-80, by B. Leibe Slide credit: L. Lazebnik St Stanfor ord University
A simple pipeline - Training Training Face identification Images Image Features 05-Nov-2019 5 St Stanfor ord University
A simple pipeline - Training Training Labels Training Images Face identification Image Training Features 05-Nov-2019 6 St Stanfor ord University
A simple pipeline - Training Training Labels Training Images Face identification Image Learned Training Features Classifier 05-Nov-2019 7 St Stanfor ord University
A simple pipeline - Training Training Labels Training Images Face identification Image Learned Training Features Classifier 05-Nov-2019 Test Image Image Features 8 St Stanfor ord University
A simple pipeline - Training Training Labels Training Images Face identification Image Learned Training Features Classifier 05-Nov-2019 Test Image Image Learned Prediction Features Classifier 9 Stanfor St ord University
A simple pipeline - Training Training Labels Training Images Face identification Image Learned Training Features Classifier 05-Nov-2019 Test Image Image Learned Prediction Features Classifier 10 Stanfor St ord University
Image features Input image Invariance? Color: Quantize RGB values ? Translation ? Scale Face identification ? Rotation ? Occlusion 05-Nov-2019 11 St Stanfor ord University
Image features Input image Invariance? Color: Quantize RGB values ? Translation ? Scale Face identification ? Rotation (in-planar) ? Occlusion 05-Nov-2019 12 St Stanfor ord University
Image features Input image Invariance? Color: Quantize RGB values ? Translation ? Scale Face identification ? Rotation (in-planar) ? Occlusion Global shape: PCA space Invariance? ? Translation ? Scale ? Rotation (in-planar) ? Occlusion 05-Nov-2019 13 St Stanfor ord University
Image features Input image Invariance? Color: Quantize RGB values ? Translation ? Scale Face identification ? Rotation (in-planar) ? Occlusion Global shape: PCA space Invariance? ? Translation ? Scale ? Rotation (in-planar) ? Occlusion 05-Nov-2019 14 St Stanfor ord University
Image features Input image Invariance? Color: Quantize RGB values ? Translation ? Scale Face identification ? Rotation ? Occlusion Global shape: PCA space Invariance? ? Translation ? Scale ? Rotation ? Occlusion 05-Nov-2019 Local shape: shape context Invariance? ? Translation ? Scale ? Rotation (in-planar) ? Occlusion 15 St Stanfor ord University
Image features Input image Invariance? Color: Quantize RGB values ? Translation ? Scale Face identification ? Rotation ? Occlusion Global shape: PCA space Invariance? ? Translation ? Scale ? Rotation ? Occlusion 05-Nov-2019 Local shape: shape context Invariance? ? Translation ? Scale ? Rotation (in-planar) ? Occlusion 16 St Stanfor ord University
Image features Input image Invariance? Color: Quantize RGB values ? Translation ? Scale Face identification ? Rotation ? Occlusion Global shape: PCA space Invariance? ? Translation ? Scale ? Rotation ? Occlusion 05-Nov-2019 Local shape: shape context Texture: Filter banks Invariance? Invariance? ? Translation ? Translation ? Scale ? Scale ? Rotation (in-planar) ? Rotation (in-planar) ? Occlusion ? Occlusion 17 St Stanfor ord University
Image features Input image Invariance? Color: Quantize RGB values ? Translation ? Scale Face identification ? Rotation ? Occlusion Global shape: PCA space Invariance? ? Translation ? Scale ? Rotation ? Occlusion 05-Nov-2019 Local shape: shape context Texture: Filter banks Invariance? Invariance? ? Translation ? Translation ? Scale ? Scale ? Rotation (in-planar) ? Rotation (in-planar) ? Occlusion ? Occlusion 18 St Stanfor ord University
A simple pipeline - Training Training Labels Training Images Face identification Image Learned Training Features Classifier 05-Nov-2019 Test Image Image Learned Prediction Features Classifier 19 Stanfor St ord University
Classifiers: Nearest neighbor Face identification Training Training examples examples from class 2 from class 1 05-Nov-2019 Slide credit: L. Lazebnik 20 St Stanfor ord University
A simple pipeline - Training Training Labels Training Images Face identification Image Learned Training Features Classifier 05-Nov-2019 Test Image Image Learned Prediction Features Classifier 21 Stanfor St ord University
Classifiers: Nearest neighbor Face identification Training Test Training examples example examples from class 2 from class 1 05-Nov-2019 Slide credit: L. Lazebnik 22 St Stanfor ord University
Let’s recap • A simple object recognition pipeline with kNN • PCA Face identification 05-Nov-2019 23 St Stanfor ord University
PCA compression: 144D -> 6D Face identification 05-Nov-2019 24 St Stanfor ord University
6 most important eigenvectors Face identification 2 2 2 4 4 4 6 6 6 8 8 8 10 10 10 12 12 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 2 2 2 4 4 4 6 6 6 8 8 8 05-Nov-2019 10 10 10 12 12 12 2 4 6 8 10 12 2 4 6 8 10 12 2 4 6 8 10 12 25 St Stanfor ord University
PCA compression: 144D ) 3D Face identification 05-Nov-2019 26 St Stanfor ord University
3 most important eigenvectors 2 2 4 4 Face identification 6 6 8 8 10 10 12 12 2 4 6 8 10 12 2 4 6 8 10 12 2 05-Nov-2019 4 6 8 10 12 2 4 6 8 10 12 27 Stanfor St ord University
What we will learn today • Introduction to face recognition • The Eigenfaces Algorithm Face identification • Linear Discriminant Analysis (LDA) 05-Nov-2019 Turk and Pentland, Eigenfaces for Recognition, Journal of Cognitive Neuroscience 3 (1): 71–86. P. Belhumeur, J. Hespanha, and D. Kriegman. "Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection". IEEE Transactions on pattern analysis and machine intelligence 19 (7): 711. 1997. 28 St Stanfor ord University
“Faces” in the brain Face identification 05-Nov-2019 Courtesy of Johannes M. Zanker 29 St Stanfor ord University
“Faces” in the brain fusiform face area Face identification 05-Nov-2019 30 Kanwisher, et al. 1997 St Stanfor ord University
Detection versus Recognition Face identification 05-Nov-2019 Detection finds the faces in images Recognition recognizes WHO the person is 31 St Stanfor ord University
Face Recognition • Digital photography Face identification 05-Nov-2019 32 St Stanfor ord University
Face Recognition • Digital photography Face identification • Surveillance 05-Nov-2019 33 St Stanfor ord University
Face Recognition • Digital photography Face identification • Surveillance • Album organization 05-Nov-2019 34 St Stanfor ord University
Face Recognition • Digital photography Face identification • Surveillance • Album organization • Person tracking/id. 05-Nov-2019 35 St Stanfor ord University
Face Recognition • Digital photography Face identification • Surveillance • Album organization • Person tracking/id. • Emotions and expressions 05-Nov-2019 36 St Stanfor ord University
Face Recognition • Digital photography Face identification • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare • Tele-conferencing 05-Nov-2019 • Etc. 37 St Stanfor ord University
The Space of Faces • An image is a point in a high dimensional space – If represented in grayscale intensity, Face identification an N x M image is a point in R NM – E.g. 100x100 image = 10,000 dim 05-Nov-2019 Slide credit: Chuck Dyer, Steve Seitz, Nishino 38 St Stanfor ord University
100x100 images can contain many things other than faces! Face identification 05-Nov-2019 39 St Stanfor ord University
The Space of Faces • An image is a point in a high ɸ 1 dimensional space – If represented in grayscale intensity, Face identification an N x M image is a point in R NM – E.g. 100x100 image = 10,000 dim • However, relatively few high dimensional vectors correspond to valid face images • We want to effectively model the 05-Nov-2019 subspace of face images Slide credit: Chuck Dyer, Steve Seitz, Nishino 40 Stanfor St ord University
Recommend
More recommend
