Unconstrained Face Recognition and Analysis S. Kevin Zhou Siemens Corporate Research, Inc. kzhou@scr.siemens.com April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Roadmap to Unconstrained Face Recognition and Analysis • Introduction • Selected Approaches – Face recognition across illumination. – Face recognition across illumination and pose. – Video-based face recognition. – Age Estimation. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Roadmap to Unconstrained Face Recognition and Analysis • Introduction • Selected Approaches – Face recognition across illumination. – Face recognition across illumination and pose. – Video-based face recognition. – Age Estimation. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Why Face Recognition and Analysis? • Application. – Non-intrusive biometric. – Homeland security, law enforcement, surveillance. – Virtual reality, HCI, multimedia. • Theory. – Interdisciplinary: Image/video processing, mathematics, physics, vision, statistics and learning, psychophysics, neuroscience, etc. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
State-Of-The-Art • Current FR systems work well ONLY under controlled situations. – Neutral expression, no makeup (Intrinsic). – Frontal illumination, frontal view (Extrinsic). – Mugshot of good quality. • Apply pattern recognition techs. to face image. – Appearance-based: Subspace methods • PCA [Turk & Pentland, 91], LDA [Belhumeur et al., 97]. • Local feature analysis (LFA) [Penev & Atick 96], ICA • Neural network, evolutionary computing, genetic algorithm – Feature-based: • Elastic graph matching [Lades et al., ’93]. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Unconstrained Face Recognition and Analysis • Motivation: deal with unconstrained conditions – Intrinsic variations: expression, makeup, aging. – Extrinsic variations: illumination and pose. – Surveillance video. – Age-related: Aging process, age estimation. – Expression and animation. • Feasible approaches – Combine pattern recognition with variation modeling – Face modeling and animation – Utilized video characteristics – Statistical learning April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Roadmap to Unconstrained Face Recognition and Analysis • Introduction • Selected Approaches – Face recognition across illumination. – Face recognition across illumination and pose. – Video-based face recognition. – Age Estimation. * S. Zhou , R. Chellappa, and D. Jacobs, “Characterization of human faces under illumination variations using rank, integrability, and symmetry constraints,” European Conf. on Computer Vision, 2004. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Illumination affects appearance * Courtesy of Prof. David Jacobs. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Approach • Generalized photometric stereo. – Describes all possible human face images under all possible illumination conditions. – Combines a physical illumination model with statistical regularity in the human class. – Derive an illumination-invariant signature for robust face recognition under illumination variation. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Key Derivations of Generalized Photometric Stereo = = + + + f f f ( ... ) h Ts T T T s × d n m m 1 1 2 2 [ ] = ⊗ , ,..., ( ) T T T f s m 1 2 = ⊗ ( ) W f s × × × d m m 3 1 3 1 Statistical regularity in identity Lambertian illumination model April 23, 2005 @ WOCC S. Kevin Zhou, SCR
FR Across Illumination: Recognition Results Training set Yale Yale Vetter ( m=10 ) ( m=100 ) Method Eigenface Generalized Generalized Photometric Photometric Stereo Stereo Average 35% 67% 93% Recognition Rate April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Roadmap to Unconstrained Face Recognition and Analysis • Introduction to unconstrained face recognition. • Selected Approaches – Face recognition across illumination. – Face recognition across illumination and pose. – Video-based face recognition. – Age Estimation. * S. Zhou and R. Chellappa, “Image-based face recognition under illumination and pose variations,” Journal of Optical Society of America (JOSA), Feb., 2005. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Appearances under illumination and pose variation • 68 objects, 12 lights, 9 poses . c 22 c 02 c 37 c 05 P c 27 o s c 29 e c 11 c 14 c 34 l 16 l 15 l 13 l 21 l 12 l 11 l 08 l 06 l 10 l 18 l 04 l 02 Illumination April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Approach • Illuminating light field – Describes all possible human face images under all possible illumination conditions and at all possible poses. – Extends generalized photometric stereo to handle pose variation. – Derives an illumination- and pose-invariant signature for robust face recognition under illumination and pose variations. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Illuminating Light Field (ILF) [Zhou & Chellappa JOSA’05] • The concept of light field (LF). = ⊗ – s L ( ) W f s × × × × Vd Vd m m 1 3 1 3 1 – = ⊗ v v h s ( ) W f s × d 1 h v s – f : illumination- and pose-invariant. L s April 23, 2005 @ WOCC S. Kevin Zhou, SCR
FR Across Illumination and Pose: Recognition Results Across illuminations Across poses Illumination variation is easier to handle than pose variation!! April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Roadmap to Unconstrained Face Recognition and Analysis • Introduction • Selected Approaches – Face recognition across illumination. – Face recognition across illumination and pose. – Video-based face recognition. – Age Estimation. * S. Zhou , V. Krueger, and R. Chellappa, “Probabilistic recognition of human faces from video,” Computer Vision and Image Understanding (special issue on Face Recognition), Vol. 91, pp. 214-245, August 2003. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Video presents challenges and chances • Requires solving both tracking and recognition. • Appearance variation. • Poor image quality. • Multiple frames with temporal continuity. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Tracking-then-Recognition v.s. Tracking-and-Recognition Approaches Tracking-then-recognition Tracking-and-recognition Essentially still-image-based face Simultaneous tracking-and- recognition recognition Utilize temporal information for Utilize temporal information for tracking only tracking and recognition Recognition performance relies Improves tracking accuracy and on tracking accuracy recognition performance Probabilistic, interpretable April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Time Series State Space Model θ = θ + g ( − ) u • Motion equation: t t t 1 = n n • Identity equation: − t t 1 = θ = Ι + • Observation equation: T{ ; } h y v t t t n t t h = θ T{ ; } y t t t ? I n y Video frame t April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Model Solution θ p n ( , | ) y • Posterior distribution: t t 0 t : p n ( | ) : posterior recognition density. y t 0 t : θ p : posterior tracking density. ( | ) y t 0 t : • Particle filter with efficient computation. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Tracking Accuracy and Recognition Result • NIST database – Case 1: Pure tracking using a Laplacian density. – Case 2: Tracking-then-recognition using an IPS density. – Case 3: Tracking-and-recognition using a combined density. Case Case 1 Case 2 Case 3 Tracking 87% NA 100% Accuracy Recognition within NA 57% 93% top 1 Recognition within NA 83% 100% top 3 * Courtesy of the HumanID project April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Roadmap to Unconstrained Face Recognition and Analysis • Introduction to unconstrained face recognition. • Selected Approaches – Face recognition across illumination. – Face recognition across illumination and pose. – Video-based face recognition. – Age Estimation. * S. Zhou et al., “Image based regression using boosting method,” Submitted. April 23, 2005 @ WOCC S. Kevin Zhou, SCR
What is Image Based Regression? • Regression or function approximation x – Given an input image , infer or approximate an x output that is associated with the image . ( x ) y • Age estimation: = age ( x ) y April 23, 2005 @ WOCC S. Kevin Zhou, SCR
State-Of-The-Art: Data-Driven Approach N ∑ = ∝ w w h ( ) ( ) ( ); ( ) ( , ); g x x y x x x x • Nonparametric regression (NPR) n n n n = n 1 – Smoothed k-NN regressor N N ∑ ∑ = = φ φ n k T • Kernel ridge regression (KRR) ( ) ( , ) ( ) ( ) g x w x x w x x n n n = = n n 1 1 – Hyperplane in RKHS < I N ∑ = g w i k ( ) ( , ) x x x • Support vector regression (SVR) n n i = i 1 – Single output, ε -insensitive loss function = ∑ α ∈ ( ) ( ) ; ( ) g x h x h x H • Boosting regression m m m m – Using boosting method – Not data-driven April 23, 2005 @ WOCC S. Kevin Zhou, SCR
Recommend
More recommend
