Secure Face Matching Using Fully Homomorphic Encryption Vishnu - PowerPoint PPT Presentation

Secure Face Matching Using Fully Homomorphic Encryption Vishnu Boddeti Michigan State University October 23rd, 2018 [˜]$ [1/1]

>>> Face Representation and Matching * Face Representation: Alignment Embedding Function Representation Detection Normalization . . . y ∈ R d [˜]$ [2/1]

>>> Face Representation and Matching * Face Representation: Alignment Embedding Function Representation Detection Normalization . . . y ∈ R d * Face Matching: . . . . . . similarity best match . R R . . R . . . . . . R R [˜]$ [2/1]

High Resp. Low Resp. High Resp. Low Resp. Activations Neurons Test Image Gender Bangs Brown Hair Pale Skin Narrow Eyes High Cheek. (a.1) (a.2) Hair Color (b.1) (a.3) Age Race (b.2) Eyeglasses Mustache Black Hair Smiling Big Nose (a.4) (b.3) (a.5) Face Shape Eye Shape Wear. Hat Blond Hair Wear. Lipstick Asian Big Eyes (a.6) (a) ANet (FC) ANet (C4) ANet (C3) Identity-related Attributes Identity-non-related Attributes 100% 90% 95% 85% Accuracy 90% 80% 85% 75% 80% 70% Male White Black Asian Smiling Wearing Rosy 5oClock Hat Cheeks Shadow (b) ANet (After fine-tuning) HOG (After PCA) Average Accuracy 80% single best 70% performing neuron 60% 50% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Percentage of Best Performing Neurons Used >>> Security Vulnerabilities * Attacks on Biometric Systems: Database Feature Sensor Matcher Decision Extractor 1 Mai, Guangcan, Kai Cao, C. YUEN Pong, and Anil K. Jain. “On the Reconstruction of Face Images from Deep Face Templates.” PAMI 2018 [˜]$ [3/1]

High Resp. Low Resp. High Resp. Low Resp. Activations Neurons Test Image Gender Bangs Brown Hair Pale Skin Narrow Eyes High Cheek. (a.1) (a.2) Hair Color (b.1) (a.3) Age Race (b.2) Eyeglasses Mustache Black Hair Smiling Big Nose (a.4) (b.3) (a.5) Face Shape Eye Shape Wear. Hat Blond Hair Wear. Lipstick Asian Big Eyes (a.6) (a) ANet (FC) ANet (C4) ANet (C3) Identity-related Attributes Identity-non-related Attributes 100% 90% 85% 95% Accuracy 90% 80% 85% 75% 80% 70% Male White Black Asian Smiling Wearing Rosy 5oClock Hat Cheeks Shadow (b) ANet (After fine-tuning) HOG (After PCA) Average Accuracy 80% single best 70% performing neuron 60% 50% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Percentage of Best Performing Neurons Used >>> Security Vulnerabilities * Attacks on Biometric Systems: Database Feature Sensor Matcher Decision Extractor * Attacks on Templates: * Face reconstruction from template 1 0.84 0.78 0.82 0.93 1 Mai, Guangcan, Kai Cao, C. YUEN Pong, and Anil K. Jain. “On the Reconstruction of Face Images from Deep Face Templates.” PAMI 2018 [˜]$ [3/1]

>>> Security Vulnerabilities * Attacks on Biometric Systems: Database Feature Sensor Matcher Decision Extractor * Attacks on Templates: * Face reconstruction from template 1 * Privacy leakage through attribute prediction from template High Resp. Low Resp. High Resp. Low Resp. Neurons Test Image Activations Gender Bangs Brown Hair Pale Skin Narrow Eyes High Cheek. (a.1) (a.2) Hair Color (b.1) (a.3) Age Race (b.2) Eyeglasses Mustache Black Hair Smiling Big Nose (a.4) 0.84 0.78 0.82 0.93 (b.3) (a.5) Wear. Hat Blond Hair Wear. Lipstick Asian Big Eyes Face Shape (a.6) Eye Shape 1 Mai, Guangcan, Kai Cao, C. YUEN Pong, and Anil K. Jain. “On the Reconstruction of Face Images from Deep Face Templates.” PAMI 2018 (a) ANet (FC) ANet (C4) ANet (C3) Identity-related Attributes Identity-non-related Attributes 100% 90% [˜]$ 85% [3/1] 95% Accuracy 90% 80% 85% 75% 80% 70% Male White Black Asian Smiling Wearing Rosy 5oClock Hat Cheeks Shadow (b) ANet (After fine-tuning) HOG (After PCA) Average Accuracy 80% single best 70% performing neuron 60% 50% 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Percentage of Best Performing Neurons Used

>>> Template Protection (a) Fuzzy Vault [˜]$ [4/1]

>>> Template Protection (a) Fuzzy Vault (b) Geometrical Transformations [˜]$ [4/1]

>>> Template Protection (a) Fuzzy Vault (b) Geometrical Transformations (c) Correlation with Random Masks [˜]$ [4/1]

>>> Template Protection (a) Fuzzy Vault (b) Geometrical Transformations (c) Correlation with Random Masks (d) Biohashing [˜]$ [4/1]

>>> Template Protection (a) Fuzzy Vault (b) Geometrical Transformations (c) Correlation with Random Masks (d) Biohashing * Drawback: Trade-Off matching performance for template security. [˜]$ [4/1]

>>> Encryption: The Holy Grail? * Data encryption is an attractive option. [˜]$ [5/1]

>>> Encryption: The Holy Grail? * Data encryption is an attractive option. * protects user’s privacy [˜]$ [5/1]

>>> Encryption: The Holy Grail? * Data encryption is an attractive option. * protects user’s privacy * enables free and open sharing [˜]$ [5/1]

>>> Encryption: The Holy Grail? * Data encryption is an attractive option. * protects user’s privacy * enables free and open sharing * mitigate legal and ethical issues [˜]$ [5/1]

>>> Encryption: The Holy Grail? * Data encryption is an attractive option. * protects user’s privacy * enables free and open sharing * mitigate legal and ethical issues * Can we encrypt the biometric signatures? [˜]$ [5/1]

>>> Encryption: The Holy Grail? * Data encryption is an attractive option. * protects user’s privacy * enables free and open sharing * mitigate legal and ethical issues * Can we encrypt the biometric signatures? * Can we perform biometric matching in the encryption domain? [˜]$ [5/1]

>>> Encryption: The Holy Grail? * Data encryption is an attractive option. * protects user’s privacy * enables free and open sharing * mitigate legal and ethical issues * Can we encrypt the biometric signatures? * Can we perform biometric matching in the encryption domain? * Can we maintain matching performance in the encrypted domain? [˜]$ [5/1]

>>> Encryption: The Holy Grail? * Data encryption is an attractive option. * protects user’s privacy * enables free and open sharing * mitigate legal and ethical issues * Can we encrypt the biometric signatures? * Can we perform biometric matching in the encryption domain? * Can we maintain matching performance in the encrypted domain? * Encryption scheme needs to allow computations directly on the encrypted data. [˜]$ [5/1]

>>> What is Homomorphic Encryption? * Encryption that allows computations on ciphertext. [˜]$ [6/1]

>>> What is Homomorphic Encryption? * Encryption that allows computations on ciphertext. * Partially Homomorphic Encryption: allows homomorphic additions or multiplications [˜]$ [6/1]

>>> What is Homomorphic Encryption? * Encryption that allows computations on ciphertext. * Partially Homomorphic Encryption: allows homomorphic additions or multiplications * Somewhat Homomorphic Encryption: allows limited number of homomorphic additions and multiplications [˜]$ [6/1]

>>> What is Homomorphic Encryption? * Encryption that allows computations on ciphertext. * Partially Homomorphic Encryption: allows homomorphic additions or multiplications * Somewhat Homomorphic Encryption: allows limited number of homomorphic additions and multiplications * Fully Homomorphic Encryption: allows unlimited number of additions and multiplications [˜]$ [6/1]

>>> What is Homomorphic Encryption? * Encryption that allows computations on ciphertext. * Partially Homomorphic Encryption: allows homomorphic additions or multiplications * Somewhat Homomorphic Encryption: allows limited number of homomorphic additions and multiplications * Fully Homomorphic Encryption: allows unlimited number of additions and multiplications This Paper Explores: [˜]$ [6/1]

>>> What is Homomorphic Encryption? * Encryption that allows computations on ciphertext. * Partially Homomorphic Encryption: allows homomorphic additions or multiplications * Somewhat Homomorphic Encryption: allows limited number of homomorphic additions and multiplications * Fully Homomorphic Encryption: allows unlimited number of additions and multiplications This Paper Explores: * feasibility of fully homomorphic encryption for secure face matching. * efficiency of fully homomorphic encryption for secure face matching. [˜]$ [6/1]

>>> Enrollment Protocol * Client device: * generates cryptographic keys Client Device Key Gen θ d [˜]$ [7/1]

>>> Enrollment Protocol * Client device: * generates cryptographic keys * captures biometric signature + extracts feature Client Device x Key Gen θ d [˜]$ [7/1]

>>> Enrollment Protocol * Client device: * generates cryptographic keys * captures biometric signature + extracts feature * encrypts feature Client Device x Encryption θ e Key Gen θ d [˜]$ [7/1]

>>> Enrollment Protocol * Client device: * generates cryptographic keys * captures biometric signature + extracts feature * encrypts feature * transmits encrypted feature + identity label to remote database Encrypted Database Client Device x Encryption ( E ( x ) , c ) θ e Key Gen θ d [˜]$ [7/1]

>>> Authentication Protocol * Client device: * captures biometric signature + extracts feature Client Device y [˜]$ [8/1]