A State-of-the-art Neural Network for Robust Face Verification - PowerPoint PPT Presentation

A State-of-the-art Neural Network for Robust Face Verification Sebastien Marcel and Samy Bengio ����������������� ��������� �������� .

Outline ➨ General Framework for Face Verification � The proposed approach � The XM2VTS database � Results � Future Work 2 ������������������

Face Verification (1/4) � Accepting or Rejecting a claimed identity (Client vs Impostor) � Building a model for each client Training Model 1 Training Model # Training Model N C 1 C <> 1 C # C <> # C N C <> N M 1 M # M N 3 ������������������

Face Verification (1/4) � Building a model for each client Training Model # (1) Reference images of client # Reference images Reference images of of client # other persons than # (2) Reference images of non-client # � Generative vs Discriminant models � Generative if only (1) � Discriminant if (1) and (2) Model # 4 ������������������

Face Verification (2/4) � Accepting a client claiming identity # C Model X is claiming Decision X is accepted # identity # I 5 ������������������

Face Verification (3/4) � Rejecting an impostor claiming identity # C Model Y is claiming Decision Y is rejected identity # # I 6 ������������������

Outline ✘ General Framework for Face Verification ➨ The proposed approach : MLP and Skin Color � The XM2VTS database � Results � Future Work 7 ������������������

MLP using Face Image and Color � Model use: MLP (client patterns vs other clients) � Features: 30x40 face template + skin color distribution Face template vector of dimension 1200 (30x40) Subwindow Downsizing Normalisation extraction Final feature vector A of dimension 1296 Decision MLP R Filtering Computing skin color skin pixels Skin feature vector pixels distribution 8 of dimension 96 (3x32) ������������������

The XM2VTS * database � Using XM2VTS Database : � Fusion (Face verification + Speech verification), � Face detection evaluation. � Content : 295 persons x 4 sessions x 2 shots � 2 audio digit sequences + 1 image 000_1_2 000_3_1 369_1_1 369_4_2 9 * http://www.ee.surrey.ac.uk/Research/VSSP/xm2vtsdb/ ������������������

XM2VTS : Lausanne Protocol * � Clients / Impostors � 200 clients, � 25 impostors for evaluation, � 70 impostors for test. � Protocols for training, evaluation and test : � Configuration I, � Configuration II. 10 * ftp://ftp.idiap.ch/pub/reports/1998/com98-05.ps.gz ������������������

XM2VTS : Lausanne Protocol � Configuration I : � Training : 200 C x 3 I (shot 1 of sessions 1,2,3) => 600 � Evaluation Clients : 200 C x 3 I (shot 2 of sessions 1,2,3) => 600 � Configuration II : � Training : 200 C x 4 I (shots of sessions 1,2) => 800 � Evaluation Clients : 200 C x 2 I (shots of session 3) => 400 � Common to both configurations : � Evaluation Impostors : 200 CM x 25 Imp x 8 I (shots of sessions 1-4) � Test Clients : 200 C x 2 I (shots of session 4) => 400 => 40000 � Test Impostors : 200 CM x 70 Imp x 8 I (shots of sessions 1-4) => 112000 11 ������������������

Evaluation using LP � Computing Errors for a given threshold � FAR = # False acceptance / # impostor accesses � FRR = # False rejection / # client access � Estimate threshold reaching the EER i.e. where FAR=FRR on the evaluation set � Compute FAR and FRR with the selected threshold on the test set � The unique measure is HTER = (FAR + FRR) / 2 12 ������������������

Comparative results � HTER on the test set: Method HTER LP1 HTER LP2 NC 61x57 3.15 1.50 MLP 30x40 - 2.807 MLP 30x40 + C - 2.125 MLP 30x40 + C 1.87 1.85 13 ������������������

Future Work � Investigate new models: � GMM, HMM2 � Investigate new features: � edges, gabor wavelets � Investigate full-automatic face verification: � integrate automatic face localisation, � evaluate degradation of performances compared to perfect face localisation. 14 ������������������