Author: thor: Gonza nzalo lo Benito to gonzabenito - PowerPoint PPT Presentation

Mast ster er The hesis is Dissertat ertation ion Comparing classical and deep approaches for face recognition in a smartgym application Author: thor: Gonza nzalo lo Benito to gonzabenito nzabenito@gma gmail.co l.com Advisors: Sergio io Escaler lera, a, UB and d CVC Josep ep Lladó dós, , UAB and CVC 1

Organization 1.Context and motivation 2.Objectives 3.State of the Art 4.Methods 5.Experimental setup 6.Results 7.Conclusions and future work 8.Appendix 2

Motivation 1. 1. Foo ootba tball ll Club Club Barcelona’s smart smartgym ym project aims to develop a system to aid gymnasium ium training ing . 2. System should identif tify and tra track ck down wn at athlet letes es within the facility, allowing self configuration of the exercise machines according to the user and computing training times of each one of them. 3. The fac ace re reco cognit gnition method should perform with precision in real-time on reduced sets of subjects and samples per individual. 3

Objectives 1. Compare th three different different approaches to the face face recogn gniti ition task, 2 classical and one deep learning based. 2. Evaluate precis precision ion perf performance ormance over different sa samples ples per subj bject ct size. 3. Compute face recognition time mes . 4. Analyze viability for ex experim periment ental al fiel ield impl mplemen ementation ation and further tests. 4

State of the Art Tra Training ning da data is crucial for the final algorithms perf performance ormance . Many current methods rely on really big big da datab abas ases es , most of them being still propri prieta etary ry . Publ Public ic da datab tabases ases provide a platform for the study of possible solutions for the project. 5

State of the Art Face recognition Databases examples [1] FERET ET: [2] SCface: e: [3] MIT-CBC BCL: L: 1. 1. 14126 14126 images. 1. 1. 4160 images. 4160 1. 1. 3240 3240 images. 2. 2. 1199 1199 subjects. 2. 2. 130 130 subjects. 2. 2. 10 subjects. 10 3. 3. ~12 samples per subject. ~12 3. 3. 32 32 samples per subject. 3. 3. 324 324 samples per subject. 4. 4. 180 180° face rotation. 4. 4. 180° face rotation. 180 4. 4. 0° to 34° face rotation. 5. 5. First t big face database. 5. Samples with survei eillan lance ce cameras 5. 5. 3D synthetic ic masks. [1] - P. J. Phillips, S. Z. Der, P. J. Rauss, and O. Z. Der, FERET (face recognition technology) recognition algorithm development and test results. Army Research Laboratory Adelphi, MD, 1996. [2] - M. Grgic, K. Delac, and S. Grgic, “Scface – surveillance cameras face database,” Multimedia Tools and Applications, vol. 5 1, no. 3, pp. 863 – 879, 2011. [3] - B. Heisele, B. Weyrauch, V. Blanz, and J. Huang, “Component - based face recognition with 3d morphable models,” 2012 IEEE Co mputer Society Conference on 6 Computer Vision and Pattern Recognition Workshops, vol. 05, p. 85, 2004.

State of the Art Face recognition method examples [1] [2] [3] [4] [5] [6] Top Face recognition methods tested on Labeled Faces in the Wild. [1] - F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering.” in CVPR. IEEE Computer Society, 2015. [2] - Y. Sun, D. Liang, X. Wang, and X. Tang, “Deepid3: Face recognition with very deep neural networks.” CoRR, vol. abs/1502.00 873, 2015. [3] - O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition.” in BMVC, X. Xie, M. W. Jones, and G. K. L. Tam, Eds. BMVA Press, 2015 . [4] - Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “DeepFace: Closing the Gap to Human - Level Performance in Face Verification,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Jun. 2014. [5] - X. Cao, D. P. Wipf, F. Wen, G. Duan, and J. Sun, “A practical transfer learning algorithm for face verification.” in ICCV. IEEE Computer Society, 2013. [6] - D. Chen, X. Cao, L. Wang, F. Wen, and J. Sun, “Bayesian face revisited: A joint formulation.” in ECCV (3), ser. Lecture No tes in Computer Science, A. W. Fitzgibbon, S. Lazebnik, P. Perona, Y. Sato, and C. Schmid, Eds., vol. 7574. Springer, 2012. 7

Methods The three methods tested are: 1.Eigenfaces [1] } 2.Fisherfaces [2] Representative of classica cal l approa oaches ches . 3.VGG Face net [3] } } Only CNN method with offici cial l code made public lic . [1] - M. Turk and A. Pentland, “Eigenfaces for recognition,” J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71– 86, Jan. 1991. [2] - P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. fisherfaces: Recognition using class specific linear projection.” pp. 711– 720, 1997. [3] - O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition.” in BMVC, X. Xie, M. W. Jones, and G. K. L. Tam, Eds. BMVA Press, 2015, pp. 41.1 – 41.12. 8

Eigenfaces Eigenfaces seeks to ma maxi ximi mize ze da data ta scatter tter , and find the principal vectors to describe the image space. Face space A new image x is project cted ed into the face space by: = + . . . mean face With u k an eigenface. This gives the descripto tor by the weight vector Eigenface Eigenface Eigenface Eigenface Eigenface #1 #2 #3 #4 #5 9

Fisherfaces Fishe herfac faces es adds the condition of maximi mizing the ratio of the Max overall scatter determinant of the between ween- clas ass scatte tter matrix of projected samples, to to the determinant of the withi hin- class scatter tter matrix of the projected samples. Optimized scatter 10

VGG Face net 1. Based on the CNN from 2014 2014 Simo monyan and Zisserman’s proposal. 2. 2. VGG VGG Face ce was trained specifically to identi dentify 2622 subj ubjects ects from a cus ustom tom built uilt database on celebrities’ face pictures [1]. 3. With triplet plet loss loss tr traini ning ng reaches 0.9913 accuracy on LFW dataset. Triplet training loss used in [1] [1] - O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition.” in BMVC, X. Xie, VGG Face architecture M. W. Jones, and G. K. L. Tam, Eds. BMVA Press, 2015, pp. 41.1 – 41.12. 11

Experiments Hardware platform: ● Intel Core i7 6500U CPU processor with 2.6 GHz clock, 12 GB DDR3L 1600 MHz SDRAM and one NVIDIA IA GeFor orce ce GT 940M 2GB DDR3. Program language: ● Eigenf enfaces ces and Fisher herfa faces ces programmed on Matla lab R2016b for Windows. ● VGG G Face programmed on Pycaff ffe for Linux. Dataset preparations: All samples were processed with fronta ntaliza lizatio tion , cropping the face detect ction on after. MIT-CBCL BCL datab tabase used. Original 20° Frontalized Cropped detection 12

Results 13

Results Face recognition with trai train and te test st se sets ts spl plit its over a database of frontaliz talized ed faces using a 10 10-folds olds scheme . 14

Results Performance comparison for each face rotation angle between the 3 models, when varying samples per subject. Eigenfaces in blues ues , Fisherfaces in greens eens , VGG Face in browns owns . 15

VGG overfitting observations epochs 16

Conclusions and future work Reached near real-tim time recognition with over 95% accuracy ( Eigenf nface ces and d ▪ Fishe herfa faces ces ) for non optimized code, on MIT-CBCL CBCL datab tabase . Viable le field d tests ts with chosen methods. ▪ MIT-CBCL dataset taset is too small ll for VGG Face and facilitates overfittin ting without ▪ intense supervision vision . Futur ure e work includes udes: Construction of FCB’s athletes face database . ▪ VGG Face tripl plet et loss traini ning to boost perfo forma mance nce . ▪ Tests ts on data with more subjects ects . ▪ Tests ts on simulta ltaneous neous recogniti gnitions ns to define final hardw dware are requirements. ▪ 17

Appendix Ex Experimen perimental tal setup etup for face recognition in FCB’s gy gym facilities based on Lin Linux ux and Rasp Raspber erry Pi Pi platform was developed. It includes the cameras, server and dedicated application to to build uild FCB’s database tabase . Application to build FCB’s gym database. Raspberry Pi unit Tests for face recognition with the setup in laboratory. 18

Thank you! Aknowle nowledgement ements: To F.C. Barcelon Barcelona for their support and disposition to carry this collaboration project, in particular to Raul Peláez Blanco nco and Joan Ramón ón Tarragó gó . To Carlos los Báez and Coen Antens ens from CVC . Questions? 19