Face detection, local features face alignment, and Face detection - PowerPoint PPT Presentation

12/6/2013 Lecture overview • Brief introduction to Face detection, local features face alignment, and • Face detection http://www.ima.umn.edu/2008-2009/MM8.5-14.09/activities/Wohlberg-Brendt/sift.png face image parsing http://www.noio.nl/assets/2011-03-01-stitching-smiles/violajones.png • Face alignment and landmark localization http://www.mathworks.com/matlabcentral/fx_files/32704/4/icaam.jpg • Face image parsing Brandon M. Smith Guest Lecturer, CS 534 Monday, October 21, 2013 http://homes.cs.washington.edu/~neeraj/projects/face-parts/images/teaser.png CS 534: Computation Photography 12/6/2013 1 CS 534: Computation Photography 12/6/2013 2 Local features: broad goal Local features: motivation What are their uses? • What are local features trying to capture? o Matching The local appearance in a region of the image http://docs.opencv.org/_images/Featur_FlannMatcher_Result.jpg David G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2 (2004) CS 534: Computation Photography 12/6/2013 3 CS 534: Computation Photography 12/6/2013 4 1

12/6/2013 Local features: motivation Local features: motivation What are their uses? What are their uses? o Matching o Matching o Image indexing and retrieval o Image indexing and retrieval o Aligning images, e.g., for panorama stitching http://www.leet.it/home/lale/files/Garda-pano.jpg Shen et al., CVPR 2012 CS 534: Computation Photography 12/6/2013 5 CS 534: Computation Photography 12/6/2013 6 Local features: motivation Local features: motivation What are their uses? What are their uses? o Matching o Matching o Image indexing and retrieval o Image indexing and retrieval o Aligning images, e.g., for panorama stitching o Aligning images, e.g., for panorama stitching o Video stabilization o Video stabilization o 3D reconstruction http://www.nsf.gov/news/special_reports/science_nation/images/virtualrealitymaps/d uomopisa500.jpg CS 534: Computation Photography 12/6/2013 7 CS 534: Computation Photography 12/6/2013 8 2

12/6/2013 Local features: motivation Local features: types What are their uses? Types of features and feature descriptors o Matching o Image intensity or gradient patches o Image indexing and retrieval o Shift Invariance Feature Transform (SIFT) – very o Aligning images, e.g., for panorama stitching popular! o Video stabilization o DAISY o 3D reconstruction o SURF o Object recognition, including face recognition o Many more… http://doi.ieeecomputersociety.org/cms/Computer.org/dl/trans/tp/2007/11/figures/i192714.gif CS 534: Computation Photography 12/6/2013 9 CS 534: Computation Photography 12/6/2013 12 Face detection: goal Face detection: motivation Automatically detect the presence and • Automatic camera focus location of faces in images. http://cdn.conversations.nokia.com.s3.amazonaws.com/wp-content/uploads/2013/09/Nokia-Pro-Camera-auto-focus_half-press.jpg http://cdn.conversations.nokia.com.s3.amazonaws.com/wp-content/uploads/2013/09/Nokia-Pro-Camera-auto-focus_half-press.jpg Shen et al., Detecting and Aligning Faces by Image Retrieval, CVPR 2013 CS 534: Computation Photography 12/6/2013 13 CS 534: Computation Photography 12/6/2013 14 3

12/6/2013 Face detection: motivation Face detection: motivation • Automatic camera focus • Automatic camera focus • Easier photo tagging • Easier photo tagging • First step in any face recognition algorithm http://images.fastcompany.com/upload/camo1.jpg http://sphotos-d.ak.fbcdn.net/hphotos-ak-ash3/163475_10150118904661729_7246884_n.jpg CS 534: Computation Photography 12/6/2013 15 CS 534: Computation Photography 12/6/2013 16 Face detection: Viola-Jones* Face detection: challenges • Large face shape and appearance variation • Paul Viola and Michael Jones, Robust Real-time Face Detection , International Journal of Computer • Scale and rotation (yaw, roll, pitch) variation Vision (IJCV), 2004. • Background clutter • Occlusions o Feature type? • Image noise o Which features are important? • Efficiency o Decide: face or not a face • False positives * Next few slides are based on a presentation by Kostantina Pall & Alfredo Kalaitzis, available at http://www1.cs.columbia.edu/~belhumeur/courses/biometrics/2010/violajones.ppt CS 534: Computation Photography 12/6/2013 17 CS 534: Computation Photography 12/6/2013 18 4

12/6/2013 Face detection: Viola-Jones Face detection: Viola-Jones Feature type? Feature type? • Useful domain knowledge: • Rectangle features o The eye region is darker than the forehead or the o Value = ∑(pixels in black) upper cheeks - ∑(pixels in white) o The nose bridge region is brighter than the eyes o Three types: 2,3,4 rectangles o The mouth is darker than the chin o Very fast: integral image • Encoding o Location and size: eyes, nose bridge, mouth, etc. o Value: darker vs. brighter CS 534: Computation Photography 12/6/2013 19 CS 534: Computation Photography 12/6/2013 20 Face detection: Viola-Jones Face detection: Viola-Jones Which features are important? Final decision: face or not a face • Tens of thousands of features to choose from • Cascade of classifiers • AdaBoost (Singer and Schapire, 1997) 1. Two-feature classifier: >99% recall, >60% precision o Given a set of weak classifiers: ℎ 𝑢 𝑦 ∈ {−1,1} 2. Five-feature classifier o Iteratively combine classifiers to form a strong classifier: 3. 10-feature classifier … 𝐼 𝑦 = 1 𝑗𝑔 𝛽 𝑢 ℎ 𝑢 (𝑦) > 𝑢ℎ𝑠𝑓𝑡ℎ𝑝𝑚𝑒 𝑢 10. 200-feature classifier 0 𝑝𝑢ℎ𝑓𝑠𝑥𝑗𝑡𝑓 * From http://www.cs.ubc.ca/~lowe/425/slides/13-ViolaJones.pdf CS 534: Computation Photography 12/6/2013 21 CS 534: Computation Photography 12/6/2013 22 5

12/6/2013 Face detection: Viola-Jones Face detection: recent approaches Xiangxin Zhu and Deva Ramanan, Face Detection, Pose Estimation, and Landmark Localization in the Wild, CVPR 2012. http://vimeo.com/12774628# CS 534: Computation Photography 12/6/2013 23 CS 534: Computation Photography 12/6/2013 24 Face detection: recent approaches Face detection: recent approaches Shen et al., Detecting and Aligning Faces by Image Shen et al., Detecting and Aligning Faces by Image Retrieval, CVPR 2013. Retrieval, CVPR 2013. CS 534: Computation Photography 12/6/2013 25 CS 534: Computation Photography 12/6/2013 26 6

12/6/2013 Face alignment and landmark Face alignment and landmark localization: goal localization: motivation Goal of face alignment: automatically align a face • Preprocess for: (usually non-rigidly) to a canonical reference o Face recognition o Portrait editing wizards o Face image retrieval o … http://www.mathworks.com/matlabcentral/fx_files/32704/4/icaam.jpg Goal of face landmark localization: automatically http://static3.businessinsider.com/image/52127e2 • Face tracking 169bedd4d60000012-752-564/realeyes-facial- recognition.png locate face landmarks of interests • Expression recognition • Facial pose recognition http://homes.cs.washington.edu/~neeraj/projects/face-parts/images/teaser.png http://mission0ps.com/wp-content/uploads/2013/04/10-special-effects.jpg CS 534: Computation Photography 12/6/2013 27 CS 534: Computation Photography 12/6/2013 28 Face alignment and landmark Face alignment and landmark localization: challenges localization: approaches • Pose Parametric appearance models o Cootes, Edwards, and Taylor, Active Appearance Models , ECCV 1998 • Expression • Identity variation • Occlusions • Image noise CS 534: Computation Photography 12/6/2013 29 CS 534: Computation Photography 12/6/2013 30 7

12/6/2013 Face alignment and landmark Face alignment and landmark localization: approaches localization: approaches Parametric appearance models Part-based deformable models o Cootes, Edwards, and Taylor, Active Appearance Models , ECCV 1998 o Saragih et al., Face Alignment through Subspace Constrained Mean- Shifts, ICCV 2009 CS 534: Computation Photography 12/6/2013 31 CS 534: Computation Photography 12/6/2013 32 Face alignment and landmark Face alignment and landmark localization: approaches localization: approaches Part-based deformable models Supervised descent o Saragih et al., Face Alignment through Subspace Constrained Mean- o Xiong and De la Torre, Supervised Descent Method and its Applications to Shifts, ICCV 2009 Face Alignment, CVPR 2013 CS 534: Computation Photography 12/6/2013 33 CS 534: Computation Photography 12/6/2013 34 8

12/6/2013 Face alignment and landmark Face image parsing localization: approaches Smith, Zhang, Brandt, Lin, and Yang, Exemplar-Based Face Parsing, CVPR 2013. Exemplar-based/non-parametric methods o Shen et al., Detecting and Aligning Faces by Image Retrieval, CVPR 2013. CS 534: Computation Photography 12/6/2013 35 CS 534: Computation Photography 12/6/2013 36 Face image parsing: goal Face image parsing: motivation Given an input face image, automatically segment • Like face alignment, can be used as a preprocess the face into its constituent parts. for face recognition, automated portrait editing, etc. • Encodes ambiguity • Generalizes to hair, teeth, ears etc. across datasets CS 534: Computation Photography 12/6/2013 37 CS 534: Computation Photography 12/6/2013 38 9