UMB-DB A Database of Partially Occluded 3D Faces Alessandro Colombo - PowerPoint PPT Presentation

UMB-DB A Database of Partially Occluded 3D Faces Alessandro Colombo Claudio Cusano Raimondo Schettini Universit` a degli Studi di Milano-Bicocca 13 November 2011

UMB-DB University of Milano-Bicocca DataBase ◮ Built to test algorithms and in challenging scenarios ◮ In particular, when faces are partially occluded Why occlusions? ◮ Less constrained scenarios ◮ Facilitate cooperative users ◮ Improve reliability with respect to ‘holes’, self occlusions, noisy data Claudio Cusano (cusano@disco.unimib.it) UMB-DB 2 / 17

Databases of 3D faces Database Subjects Acquisitions Expressions Poses Occlusions FRGCv.2 466 4007 6 1 0 BU3DFE 100 2500 6 1 0 ND2006 888 13450 5 1 0 York 350 5250 5 3? 0 CASIA 123 1845 5 1 0 FRAV3D 106 1696 2 8 0 BJUT-3D 500 500 1 1 0 GavabDB 61 720 4 4 0 3DRMA 120 720 1 4 0 Texas 3D 118 1149 ? 1 0 Bosphorus 105 4666 34 13 381 UMB-DB 143 1473 4 1 578 Claudio Cusano (cusano@disco.unimib.it) UMB-DB 3 / 17

Acquisitions ◮ 143 subjects ◮ 98 males, 45 females ◮ Most from 19 to 50 years old ◮ Minolta Vivid 900 laser scanner ◮ Slow mode, 25mm lens ◮ 640 × 480 samples ◮ Eyes closed for safety reasons ◮ 1.5–3m of distance from the device ◮ No further processing such as noise reduction or holes filling Claudio Cusano (cusano@disco.unimib.it) UMB-DB 4 / 17

Acquisitions ◮ At least 9 acquisitions per subject ◮ three with neutral expression ◮ with smiling, angry, bored expressions ◮ occluded by scarf, hat, and hands ◮ 1473 acquisitions ◮ 578 occluded faces ◮ On average, occlusions cover 42% of the face ◮ With a maximum of 84% Claudio Cusano (cusano@disco.unimib.it) UMB-DB 5 / 17

Some examples

Annotations Each acquisition includes Color image 3D model Up to 7 Mask of visible landmarks face and a set of labels indicating the facial expression, occluding object. . . Claudio Cusano (cusano@disco.unimib.it) UMB-DB 7 / 17

A system for the recognition of partially occluded 3D faces 3D Face ◮ Non-occluded faces are Face detection recognized as they are ◮ Small occlusions are restored Normalization and the face is then recognized Occlusion detection ◮ Faces with large occlusions are rejected Rejection Restoration Recognition Claudio Cusano (cusano@disco.unimib.it) UMB-DB 8 / 17

Occlusion tolerant 3D face detection 1 Selection of candidate features by curvature analysis Input face Mean curvature Gaussian curv. HK classification Thresholded HK Candidate eyes Candidate noses 1 A. Colombo, C. Cusano, R. Schettini, “Gappy PCA classification for occlusion tolerant 3D face detection,” J. of Math. Imaging and Vision, 35(3):193–207, 2009. Claudio Cusano (cusano@disco.unimib.it) UMB-DB 9 / 17

Occlusion tolerant 3D face detection Pairs or triplets of feature points form candidate faces ◮ Filtered by orientation and size ◮ Registered on the mean face by a variation of ICP ◮ Points too from the mean face are discarded ◮ Classified by a ‘gappy’ PCA classifier Claudio Cusano (cusano@disco.unimib.it) UMB-DB 10 / 17

Detection results 56 false positives on the whole DB Acquisition type Number of faces Detected Faces % Neutral 441 437 99.1 Non-neutral 442 431 97.5 Occluded 578 553 95.7 Scarf 151 141 93.4 Glasses 75 71 94.7 Hair 33 30 90.9 Hand 165 150 90.9 Hat 183 179 97.8 Misc 28 26 92.85 Whole DB 1473 1421 96.5 Claudio Cusano (cusano@disco.unimib.it) UMB-DB 11 / 17

Restoration of occlusions, and face recognition 2 Occluded face ◮ Occlusions are detected on the basis of the reconstruction error in an eigenspace ◮ Occluded regions are restored by a ‘Gappy’ PCA Detected occlusions ◮ Restored faces are recognized Restored face 2 A. Colombo,C. Cusano, R. Schettini, “Three-dimensional occlusion detection and restoration of partially occluded faces,” J. of Math. Imaging and Vision, 40(1):105–119, 2011. Claudio Cusano (cusano@disco.unimib.it) UMB-DB 12 / 17

Restoration examples Claudio Cusano (cusano@disco.unimib.it) UMB-DB 13 / 17

Recognition results Normalization Test set EER (%) IR (%) Manual All cases 18.6 71.0 Automatic All cases 19.5 69.6 Automatic Neutral 1.9 98.0 Automatic Expressions 18.4 66.7 Automatic Occlusions 23.8 56.5 Claudio Cusano (cusano@disco.unimib.it) UMB-DB 14 / 17

Results DB Acquisitions Subjects EER (%) IR (%) UMB-DB 578 143 23.8 56.5 UND + occl. 477 158 10.9 96.1 Bosphorus 360 105 11.5 87.7 Claudio Cusano (cusano@disco.unimib.it) UMB-DB 15 / 17

Results 1 FRR 0.1 scarf hand hat glasses misc 0.01 0.01 0.1 1 FAR Claudio Cusano (cusano@disco.unimib.it) UMB-DB 16 / 17

Conclusions UMB-DB ◮ Large number of occlusions ◮ Great variability in terms of position, extent, occluding object ◮ Very challenging! The database is publicly available http://www.ivl.disco.unimib.it/umbdb Claudio Cusano (cusano@disco.unimib.it) UMB-DB 17 / 17