Linear and Generalized Linear Models for Analyzing Face Recognition - PowerPoint PPT Presentation

Linear and Generalized Linear Models for Analyzing Face Recognition Performance J. Ross Beveridge Colorado State University Page 1 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Credit Where Credit is Due … • Bruce Draper ……... CSU Computer Science • Geof Givens ………. CSU Statistics • Jonathon Phillips …. NIST • Graduate Students – Wendy Yambor, Kai She, David Bolme, Kyungim Baek, Marcio Teixeira, David Bolme, Ben Randall, Trent Williams, Jilmil Saraf, Ward Fisher Page 2 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

What Factors (Covariates) ? Race Gender Age Eyes Glasses Bangs Facial Hair Mouth Smiling? Page 3 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Subject Image Data Page 4 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Yes, Yes, FER(R)ET Again … http://www.rollmop.org/ferrets/ Page 5 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Subject Image Data • 1,072 Human Subjects from the FERET Data • 2,144 FERET Images • Exactly 2 images per subject, taken on same day Page 6 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Collecting the Covariates Page 7 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Our Subject Covariates FERET Subject/Image Covariates Fixed Per Subject Age Young Old Gender Male Female Race White Black Asian Other Skin Clear Other Fixed Per Image Bangs No Yes Expression Neutral Other Eyes Open Other Facial Hair No Yes Makeup No Yes Mouth Closed Other Glasses No Yes Page 8 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Standard Algorithms to Test Three Algorithms PCA IIDC EBGM http://www.cs.colostate.edu/evalfacerec/index.html Page 9 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

NIST FERET Image Preprocessing • Integer to float conversion – 256 gray levels to single-floats • Geometric Normalization – Human chosen eye centers. • Masking – Elliptical mask around face. • Histogram Equalization – Equalize unmasked pixels • Pixel normalization – Shift and scale pixel values so mean pixel value is zero and standard deviation over all pixels is one. Refinement of NIST preprocessing used in FERET. Page 10 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Training • Best, but infeasible, solution – Disjoint images, same set of human subjects. – But, subject replicate images limited in FERET. • Next best choice – Train on exactly those images used in the study. Page 11 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Performance Variable? • Recognition Rate? – Defined over a set of people, not per person. • Similarity score? – Defined per person. – Linear models, ... – But, what does this tell us about actual performance? • Probability of being recognized at Rank 1? – Defined per person. – Non-linear modeling problem. • Probability of being correctly verified at given FAR? – Defined per person. – Non-linear modeling problem. Page 12 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Statistical Modeling Overview Sampled Normalized Similarity Scores Predict: similarity scores Linear Model from Covariates (ANOVA) covariate Algorithm combinations Age Race Gender Skin Glasses Facial Hair Makeup Bangs Expression Mouth Eyes Page 13 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Statistical Modeling Overview Sampled Normailzed Similarity Scores Predict: similarity scores Linear Model from Covariates (ANOVA) covariate Algorithm combinations Age Race Gender Skin Glasses Facial Hair Makeup Predict: Bangs Generalized Expression probability of Mouth Linear correct Eyes recognition from Model covariate combinations Sampled Recognition Ranks Page 14 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Linear Model - Similarity (Distance) Y i = Similiarity (Distance) metric for image pair i. X i = Algorithm & Human covariate factors for image pair i . β = Parameters quantifying factor effects. Y i = β 0 + β 1 X i1 + β 2 X i2 + … + ε i with ε i ~ iid Normal(0, σ 2 ) Page 15 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Generalized Linear Model Pr(correct rank one recognition) Y i = Was the i th image pair matched at rank 1 ? (i.e. Y i = 1 if R i = 1 and otherwise Y i = 0) X i = Algorithm & Human covariate factors for image pair i . β = Parameters quantifying factor effects. g( µ Yi|Xi ) = β 0 + β 1 X i1 + β 2 X i2 + … + ε i Y i | X i ~ f( µ Yi|Xi ) independently Now: g(z) = log (z/(1-z)), f( µ Yi|Xi ) = Bernoulli( µ Yi|Xi ) Page 16 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

What Do Models Tell Us? PCA Algorithm Example. Look at age holding all other covariates fixed. Covariate Base Old Similarity Scores - LM Age Young Old • 13.0% Increase in similarity Gender Male Male • p-value < 0.0001 Race White White • Older is easier. Skin Clear Clear Bangs No No Pr(rank-one) - GLM Expression Neutral Neutral • Pr(crk=1) = 0.916 Base Eyes Open Open • Pr(crk=1) = 0.951 Old Facial Hair No No • p-value = 0.009 Makeup No No • Older is easier. Mouth Closed Closed Glasses No No Page 17 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

What Do Models Tell Us? PCA Algorithm Example. Look at gender holding all other covariates fixed. Covariate Base Old Similarity Scores - LM Age Young Young • 1.7% decrease in similarity Gender Male Female • p-value < 0.33 Race White White • Gender is not significant. Skin Clear Clear Bangs No No Pr(rank-one) - GLM Expression Neutral Neutral • Pr(crk=1) = 0.915 Base Eyes Open Open • Pr(crk=1) = 0.884 Female Facial Hair No No • p-value = 0.0925 Makeup No No • Gender is not significant Mouth Closed Closed Glasses No No Page 18 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Model Validation & p-values • Don’t try to read this … • Standards for evaluating and reporting results important. Page 19 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

GLM with Three Algorithms Page 20 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Age: Young vs. Old HARDER EASIER Subject Old Change to Baseline Predicted Pr(crk=1) Page 21 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Eyes: Open vs. Closed HARDER EASIER Eyes Closed Change to Baseline Predicted Pr(crk=1) Page 22 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Verification Performance Covariates Age Gender Generalized Bangs Predict: Linear Facial Hair probability of Mixed Eyes correct verification effect from covariate and Model false alarm rate combinations (GLMM) Sampled verification outcomes at different false alarm rates Page 23 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Verification Outcomes at Fixed False Alarm Rate α Two Images per Subject Example 50 x 50 Similarity Matrix Page 24 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Verification Outcomes at Fixed False Alarm Rate α Two Images per Subject Example 50 x 50 Similarity Matrix 1) Set FAR α , e.g. α = 1/250 Page 25 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Verification Outcomes at Fixed False Alarm Rate α Two Images per Subject Example 50 x 50 Similarity Matrix 1) Set FAR α , e.g. α = 1/250 2) Indicate people correctly verified at threshold corresponding to α Page 26 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Verification Indicator Variable and FAR settings • Our study - 1,072 x 1,072 similarity matrix. – 1,072 match scores, – 1,148,112 non-match scores. Setting FAR ( � ) Rate per 10,000 Indicator Variable Y for 1 1/10,000 1 each subject for each 2 1/5,000 2 FAR setting: 3 1,2,500 4 1 verified 4 1/1,000 10 5 1/500 20 0 otherwise 6 1/250 40 7 settings total. 7 1/100 100 Page 27 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Linearity of Log Odds against Log FAR - FERET+PCA � � VR ln � � � 1 � VR � � � p ln , p � verification probability � � 1 � p � � Page 28 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Linearity of Log Odds against Log FAR - FRVT Page 29 Ross Beveridge, Biometric Quality Workshop, March 9, 2006

Generalized Linear Mixed Model (GLMM) Analysis is: Mixed Effects Logistic Regression with Repeated Measures on People. • Let A and B be 2 factors that might influence algorithm performance. For example, age and gender. – Example factor settings A=a and B=b. • Let j index the FAR setting, α j • Y pabj is – 1 if Person p is verified correctly, – 0 otherwise. • Y pabj depends on: – person p, – factors A and B, and – false alarm rate α j . Page 30 Ross Beveridge, Biometric Quality Workshop, March 9, 2006