Deciphering the Face Deciphering the Face Aleix M. Martinez - PowerPoint PPT Presentation

Deciphering the Face Deciphering the Face Aleix M. Martinez Computational Biology Computational Biology and Cognitive Science Lab aleix@ece.osu.edu l i @ d

Human-Computer Interaction Politics Human Human face face face face Art Sign Language Language Cognitive Cognitive Computer Vision Science

Models of Face Perception • Features: Shape vs. texture. … … • 2D vs. 3D • Form of the computational space: p p Continuous vs. Categorical

What we are going to show • What is the form of the computational space in human face perception? Hybrid approach : in human face perception? Hybrid approach : Linear combination of continuous representations of categories representations of categories. c 1 + c 2 + … + c n 1 2 n • What are the dimensions? Mostly configural . • In computer vision we need precise detailed detection of faces and facial features. detect o o aces a d ac a eatu es.

Same or different? Identity

Same or different? Identity

Same or different? Identity, expression, gender, etc. Identity

Dimensions of the Face Space Same or different? Configural processing

Form of the Computational Face Space Computational Face Space Exemplar-based model Exemplar based model Exemplar … cells cells Norm-based model Mid-level vision Low-level vision

Facial Expressions of Emotion

Muscle Positions Model

Muscle Positions Model • Global shape (bone structure) determines identity – configural. y g • But ONLY muscles are responsible for expression interaction for expression, interaction …

Angry Emotion perception in Configural Processing emotionless faces f l Sad Neth & Martinez, JOV, 2009. Neutral Neutral

100% Stimuli 75% Neth & Martinez, JOV, 2009. 50% 25%

Less, same, more. Experiment Experiment

Configural Processing Sad 90 * * * 80 * * * * 70 60 Less 50 * Same * * 40 * More 30 * * * * 20 * 10 0 -100% -75% -50% -25% 0% 25% 50% 75% 100% Neth & Martinez, JOV, 2009.

Configural Processing Angry 90 * * * 80 * * 70 * * 60 Less 50 Same * * 40 More * * 30 * * * 20 * * * 10 0 -100% -75% -50% -25% 0% 25% 50% 75% 100% Neth & Martinez, JOV, 2009.

Norm-based Face Space Sadness 100% Multidimensional 75% 75% Face Space S 50% - density + density + density 25% Easier MEAN - density + density density 100% More difficult Anger Neth & Martinez, JOV, 2009.

Configural Processing Neth & Martinez, JOV, 2009.

Computational Space Neth & Martinez, Vision Research, 2010

Thinner face Thinner face Computational Space Neth & Martinez, Vision Research, 2010 Wider face

American Gothic Illusion Neth & Martinez, Vision Research, 2010

Why Configural Features?

15 x 10 pixels

Why Configural cues? sad neutral angry Neth & Martinez, Vision Research, 2010; Du & Martinez, 2011

Proposed Hybrid Model: Recognizing other emotion labels Recognizing other emotion labels c 1 c + c 2 + c + + … + c n + c Happily Angrily g y surprised surprised Martinez, CVPR, 2011

Configural Processing = Precise detection of facial features detection of facial features 4 2 pixels 4.2 pixels 3,930 error images (1.5%) (1.5%) Ding & Martinez, PAMI, 2010

Face Detection

Features VS context Observation : Most detections are near the correct location – they are not incorrect, they are imprecise . location they are not incorrect, they are imprecise . Key idea : Use context information to train where not t d t to detect faces and facial features. t f d f i l f t Ding & Martinez, CVPR, 2008; PAMI, 2010

Features VS context Observation : Most detections are near the correct location – they are not incorrect, they are imprecise . location they are not incorrect, they are imprecise . Key idea : Use context information to train where not t d t to detect faces and facial features. t f d f i l f t Ding & Martinez, CVPR, 2008; PAMI, 2010

Features VS context Observation : Most detections are near the correct location – they are not incorrect, they are imprecise . location they are not incorrect, they are imprecise . Key idea : Use context information to train where not t d t to detect faces and facial features. t f d f i l f t Ding & Martinez, CVPR, 2008; PAMI, 2010

Subclass Discriminant Analysis y Between subclass Between-subclass scatter matrix: ( ( ) ( ) ( ) ) H C ∑∑ ∑∑ i = μ − μ μ − μ T T Σ Σ p . B ij ij ij = = i 1 j 1 Basis vectors: Basis vectors: = Λ Σ V Σ V . B X How many subclasses (H): Minimize the conflict, K . Zhu & Martinez, PAMI, 2006

Precise Detailed Detection Error: 6.2 pixels (2%) vs Manual: 4.2 (1.5%) E 6 2 i l (2%) M l 4 2 (1 5%) Ding & Martinez, CVPR, 2008; PAMI, 2010

36 Detection + non-rigid SfM Gotardo & Martinez, PAMI, 2011; Gotardo & Martinez, CVPR, 2011.

Take Home Messages • What is the form of the computational space in human face perception? Linear combination of known categories. c 1 + c 2 + … + c n 1 2 n • What are the dimensions? Mostly configural . fi l Wh t th di i ? M tl • Precise detection of facial features .

CBCSL Paulo Gotardo, Shichuan Du, Don Neth, Liya Ding, Onur Paulo Gotardo, Shichuan Du, Don Neth, Liya Ding, Onur Hamsici, Samuel Rivera, Fabian Benitez, Hongjun Jia, Di You. National Institutes of Health National Institutes of Health National Science Foundation