Predicting and hiding personal information from from face images - PowerPoint PPT Presentation

Big Data Madison August 27, 2019 https://www.meetup.com/BigDataMadison/events/258898788/ Predicting and hiding personal information from from face images using deep learning Sebastian Raschka, Ph.D. Assistant Professor sraschka@wisc.edu Department of Statistics http://stat.wisc.edu/~sraschka/ Sebastian Raschka Big Data Madison, Aug 2019

Biometric (Face) Recognition A. Identification B. Verification Determine identity of an unknown person Verify claimed identity of a person 1-to- n matching 1-to-1 matching ... (CelebA dataset) (MUCT dataset) Sebastian Raschka Big Data Madison, Aug 2019 � 2

Applications of Biometric (Face) Recognition A. Identification B. Verification Determine identity of an unknown person Verify claimed identity of a person 1-to- n matching 1-to-1 matching ... (CelebA dataset) (MUCT dataset) https://www.vosizneias.com/205332/2015/06/04/toronto-foreigners-entering- https://www.zdnet.com/article/too-many-false-alarms-for-population-wide-facial-surveillance-nec/ canada-to-undergo-biometric-screening/ Sebastian Raschka Big Data Madison, Aug 2019 � 3

https://www.nytimes.com/interactive/2019/04/16/opinion/facial-recognition-new-york-city.html Sebastian Raschka Big Data Madison, Aug 2019 � 4

Beyond/In Addition to Traits for Biometric Recognition: 
 Soft Biometric Attributes Identity John Doe Gender Male Age 65 Race Caucasian Medical Healthy S OFT BIOMETRIC ATTRIBUTES https://media.pitchfork.com/photos/59c0335abe5bf47cb9787b75/2:1/w_790/lynch.jpg Sebastian Raschka Big Data Madison, Aug 2019 � 5

Part I: Extracting Soft-Biometric Attributes 
 from Face Images Sebastian Raschka Big Data Madison, Aug 2019 � 6

Predicting Gender (... A Simple Task) Identity John Doe Gender Male Age 65 Race Caucasian Medical Healthy S OFT BIOMETRIC ATTRIBUTES https://media.pitchfork.com/photos/59c0335abe5bf47cb9787b75/2:1/w_790/lynch.jpg Sebastian Raschka Big Data Madison, Aug 2019 � 7

VGG-19 34-layer plain 34-layer residual image image image ResNet-101 Applied to output 3x3 conv, 64 size: 224 3x3 conv, 64 pool, /2 output size: 112 3x3 conv, 128 Gender Classification 3x3 conv, 128 7x7 conv, 64, /2 7x7 conv, 64, /2 pool, /2 pool, /2 pool, /2 output size: 56 3x3 conv, 256 3x3 conv, 64 3x3 conv, 64 3x3 conv, 256 3x3 conv, 64 3x3 conv, 64 3x3 conv, 256 3x3 conv, 64 3x3 conv, 64 3x3 conv, 256 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 3x3 conv, 64 pool, /2 3x3 conv, 128, /2 3x3 conv, 128, /2 output size: 28 3x3 conv, 512 3x3 conv, 128 3x3 conv, 128 3x3 conv, 512 3x3 conv, 128 3x3 conv, 128 3x3 conv, 512 3x3 conv, 128 3x3 conv, 128 3x3 conv, 512 3x3 conv, 128 3x3 conv, 128 3x3 conv, 128 3x3 conv, 128 3x3 conv, 128 3x3 conv, 128 3x3 conv, 128 3x3 conv, 128 He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. + Large-scale CelebFaces Attributes (CelebA) Dataset http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html PyTorch code available at: 
 https://github.com/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-resnet101-celeba.ipynb Sebastian Raschka Big Data Madison, Aug 2019 � 8

Estimating the Apparent Age (... A Bit More Difficult) Identity John Doe Gender Male Age 65 Race Caucasian Medical Healthy S OFT BIOMETRIC ATTRIBUTES https://media.pitchfork.com/photos/59c0335abe5bf47cb9787b75/2:1/w_790/lynch.jpg Sebastian Raschka Big Data Madison, Aug 2019 � 9

Types of Labels in Supervised Learning Tasks Color Size Price green M 10.1 red L 13.5 blue XXL 15.3 Nominal type 
 Ordinal type 
 Continuous 
 Task: classification Task: ordinal regression Task: metric regression Sebastian Raschka Big Data Madison, Aug 2019 � 10

Ordinal Regression Ordinal regression, also called ordinal classification or ranking 
 (although ranking is a bit different) Order dependence like in metric regression, discrete values like in classification, 
 but no metric distance but order dependence/information r K ≻ r K − 1 ≻ . . . ≻ r 1 great ≻ good ≻ okay ≻ for genre fans ≻ bad E.g., movie ratings: Sebastian Raschka Big Data Madison, Aug 2019 � 11

Supervised Learning: Ordinal Regression • Ranking: Correct order matters 
 (0 loss if order is correct, e.g., rank a collection of movies by "goodness") ≻ ≻ • Ordinal Regression: Correct label matters (as well) 
 (E.g., age of a person in years; here, regard aging as a non-stationary process) ≻ ≻ Excerpt from the UTKFace dataset 
 https://susanqq.github.io/UTKFace/ Age: 29 18 41 Sebastian Raschka Big Data Madison, Aug 2019 � 12