Evaluation of Privacy Protection Performance of De-id Faces STSM - PowerPoint PPT Presentation

Evaluation of Privacy Protection Performance of De-id Faces STSM Report by Zongji Sun University of Hertfordshire, UK Host institution: Department of Electronic System, Aalborg University, Denmark 1

Aim of this STSM • Evaluation of the UH face de-identification method using the AAU face recognition system. • UH face de-identification: k -Diff-furthest • Methods in AAU face recognition system: • LBP • LPQ • HOG • Face datasets used in the evaluation • FERET • Multi-PIE 2

Our k -Diff-furthest method Face features in a simplified 2D illustration 05/11/2015 3

k -Diff-furthest de-identified faces • Original faces • De-identified faces 05/11/2015 4

Re-identification risk of de-identified faces against their original faces 05/11/2015 5

Types of re-identification attacks Gallery Probe Naïve recognition Original De-identified Reverse De-identified Original recognition Parrot recognition De-identified Mock De- identified 6

Performance of Threshold method [1] Newton, E. M., Sweeney, L., & Malin, B. (2005). Preserving privacy by de-identifying face images. IEEE Transactions on Knowledge and Data Engineering, 17(2), 232 – 243. doi:10.1109/TKDE.2005.32 7

Face datasets - FERET • 963 subjects each with two face images captured at the same shooting session • Nearly identical head pose, hair details, upper body, clothing and background • Details within face region (e.g. facial expression) may be different id: 002 id: 0043 id: 194 8

Face datasets – Multi-PIE • 224 subjects each with two face images captured at the same shooting session • Nearly identical head pose, hair details, upper body, clothing and background • Details within face region are different – one neutral and the other happy id: 001 id: 002 id: 040 9

De-id face De-id face with Original without BG Examples of de-identified faces BG 10

Re-identification Test id: 00043 id: 00050 id: 00048 id: 00130 id: 00155 Original fa Original fb De-id fb 11

Re-identification Test Without background Without background Without background 200  200 Original fa Original fb De-id fb 200 x 200 Without background Orig De-id PCA + K-NN (Euclidean distance) 47.25% 0.10% LBP + K-NN (Euclidean distance) 74.25% 0.21% HOG + K-NN (Cosine distance) 47.14% 0.10% LPQ + K-NN (Cosine distance) 51.92% 0.21% LPQ + SVM 48.39% 0.21% 12

Re-identification Test (cont.) Without background Without background Without background 300  300 Original fa Original fb De-id fb 300 x 300 Without background FERET dataset Orig De-id PCA + k -NN (Euclidean distance) 42.37% 0% LBP + k -NN (Euclidean distance) 63.03% 0.21% HOG + k -NN (Cosine distance) 18.38% 0% LPQ + k -NN (Cosine distance) 45.79% 0.21% 300 x 300 Without background Multi-PIE dataset Orig De-id LBP + k -NN (Euclidean distance) 76.79% 0.45% LPQ + k -NN (Cosine distance) 61.16% 0% 13

Re-identification Test with BG With background With background With background 200  200 Original fa Original fb De-id fb 200 x 200 With background Orig De-id PCA + K-NN (Euclidean distance) 54.83% 4.05% LBP + K-NN (Euclidean distance) 83.39% 1.25% HOG + K-NN (Cosine distance) 74.87% 5.92% LPQ + K-NN (Cosine distance) 80.69% 21.91% LPQ + SVM 78.09% 17.45% 14

Re-identification Test with BG (cont.) With background With background With background 300  300 Original fa Original fb De-id fb 300 x 300 With background Without background FERET dataset Orig De-id Orig De-id PCA + k -NN (Euclidean distance) 61.27% 38.94% 42.37% 0% LBP + k -NN (Euclidean distance) 87.23% 54.83% 63.03% 0.21% HOG + k -NN (Cosine distance) 78.09% 56.70% 18.38% 0% LPQ + k -NN (Cosine distance) 82.24% 61.37% 45.79% 0.21% 300 x 300 With background Without background Multi-PIE dataset Orig De-id Orig De-id LBP + k -NN (Euclidean distance) 95.5% 66.96% 76.79% 0.45% LPQ + k -NN (Cosine distance) 96.88% 75.89% 61.16% 0% 15

Same person? 16

Same person? 17

Potential risk of background • Inverse crop based on their facial landmarks • This is a generic attack to any face de-identification method, which modifies the face region only. id: 00049 id: 00002 id: 00130 id: 00155 Original fa 300 x 300 inverse crop Orig De-id Original fb PCA + k -NN 56.39% 30.11% LBP + k -NN 78.19% 55.24% HOG + k -NN 53.27% 26.17% LPQ + k -NN 60.85% 32.09% De-id fb 18

Summary of activities during this STSM • Extensive re-identification tests with FERET and Multi PIE datasets using different face descriptors and different distance measurements. • Recognition test of a potential re-identification attack based on background image region and evaluation of the risk (recognition using background information) 19

Conclusions • k -Diff-furthest face de-identification method has high privacy protection performance within the face region. • However, when the de-identified face region is merged with the background from the original image, it may significantly increase the risk of re-identification to an unacceptable level. • To build a complete face de-identified system, not only the face region need to be considered, but also the soft-biometric and non-biometric parts outside the face region (work of WG2) need to be considered. 20

Questions? 21

Thank you! 22