A TTACKING D EEP N EURAL N ETWORKS WITH A DVERSARIAL I MAGES Fabrizio Falchi ISTI, CNR, Pisa, italy www.fabriziofalchi.it COST ACTION CA16101 - Dubrovnik, November 7th
fabrizio.falchi@cnr.it
fabrizio.falchi@cnr.it
fabrizio.falchi@cnr.it
W HAT ’ S THAT ? fabrizio.falchi@cnr.it
W HAT ’ S THAT ? fabrizio.falchi@cnr.it
W HAT ’ S THAT ? fabrizio.falchi@cnr.it
W HAT ’ S THAT ? fabrizio.falchi@cnr.it
A DVERSARIAL E XAMPLES fabrizio.falchi@cnr.it
I LLUSIONS Edward H. Adelson fabrizio.falchi@cnr.it
I LLUSIONS Edward H. Adelson fabrizio.falchi@cnr.it
fabrizio.falchi@cnr.it
DUBROVNIK fabrizio.falchi@cnr.it
D UBROVNIK – D EEP D REAM fabrizio.falchi@cnr.it
K NOW Y OUR E NEMY
A DVERSARY Goal Knowledge Capability fabrizio.falchi@cnr.it
A DVERSARY ’ S G OAL
G ENUINE I MAGES … Mushrooms Pineapple Toucan … fabrizio.falchi@cnr.it
19 N ON -T ARGETED A TTACK Goal + NON-TARGETED = … Mushrooms <whatever> … fabrizio.falchi@cnr.it
20 T ARGETED A TTACK Goal + TARGETED = … Mushrooms Toucan … fabrizio.falchi@cnr.it
Goal Knowledge Capability fabrizio.falchi@cnr.it
Slide credit: Biggio fabrizio.falchi@cnr.it
A TTACKING D EEP N EURAL N ETWORKS fabrizio.falchi@cnr.it
B LACK B OX A DVERSARIAL E XAMPLE A TTACKS Practical Black-Box Attacks against Machine Learning Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, Ananthram Swami fabrizio.falchi@cnr.it
A TTACKING F ACE R ECOGNITION S YSTEMS
A DVERIAL F ACES Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi fabrizio.falchi@cnr.it
A DVERSARIAL F ACES Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi fabrizio.falchi@cnr.it
Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi fabrizio.falchi@cnr.it
A DVERSARIAL F ACES Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi fabrizio.falchi@cnr.it
Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi fabrizio.falchi@cnr.it
Fast Geometrically-Perturbed Adversarial Faces Ali Dabouei, Sobhan Soleymani, Jeremy Dawson, Nasser M. Nasrabadi fabrizio.falchi@cnr.it
A TTACKING IN R EAL W ORLD
33 A DVERSARIAL I MAGE Photo: labsix fabrizio.falchi@cnr.it
34 R OTATE A DVERSARIAL I MAGE Photo: labsix fabrizio.falchi@cnr.it
fabrizio.falchi@cnr.it
36 fabrizio.falchi@cnr.it
Robust Physical-World Attacks on Deep Learning Models Eykholt, Evtimov, Fernandes, Bo Li, Rahmati, Xiao, Prakash, Kohno, Song fabrizio.falchi@cnr.it
fabrizio.falchi@cnr.it
Adversarial Generative Nets: Neural Network Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter fabrizio.falchi@cnr.it
A TTACKING DNN IN R EAL W ORLD Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter fabrizio.falchi@cnr.it
A TTACKING DNN IN R EAL W ORLD Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter fabrizio.falchi@cnr.it
A TTACKING DNN IN R EAL W ORLD Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter fabrizio.falchi@cnr.it
A TTACKING F ACE V ERIFICATION S YSTEMS
F ACE R COGNITION ID1 ID2 ID3 ... ID10 ... IDn fabrizio.falchi@cnr.it
F ACE V ERIFICATION fabrizio.falchi@cnr.it
46 F ACE V ERIFIATION Unravelling Robustness of Deep Learning based Face Recognition Against Adversarial Attacks Goswami, Ratha, Agarwal, Singh, Vatsa fabrizio.falchi@cnr.it
47 F ACE V ERIFIATION Unravelling Robustness of Deep Learning based Face Recognition Against Adversarial Attacks Goswami, Ratha, Agarwal, Singh, Vatsa fabrizio.falchi@cnr.it
A DVERSARY -A WARE M ACHINE L EARNING
49 A DVERSARY -A WARE M ACHINE L EARNING Machine learning system should be aware of the arms race with the adversary Security evaluation of pattern classifiers under attack Biggio, Fumera, Roli fabrizio.falchi@cnr.it
fabrizio.falchi@cnr.it
A DVERSARIAL E XAMPLE D ETECTION
52 G ENIUNE I MAGES … Mushrooms Pineapple Toucan … fabrizio.falchi@cnr.it
53 N ON -T ARGETED A TTACK + = … Mushrooms <whatever> … fabrizio.falchi@cnr.it
54 D EFENSE + Increase robustness = … Mushrooms … fabrizio.falchi@cnr.it
55 D ETECTION + Attack detection = … Mushrooms … fabrizio.falchi@cnr.it
A DVERSARIAL E XAMPLES D ETECTION Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, Michael K. Reiter fabrizio.falchi@cnr.it
O UR A PPROACH
D EEP LEARNING ( FROM N ATURE ) AI Machine Learning Repres. Learning Deep Learning fabrizio.falchi@cnr.it
D EEP LEARNING ( FROM N ATURE ) Representation learning methods that allow a machine to be fed with raw data and to automatically discover the representations needed for detection or classification. Deep-learning are representation learning methods o with multiple levels of representation, obtained by o composing simple but non-linear modules that each o transform the representation at one level into a representation at a higher, slightly more abstract level. fabrizio.falchi@cnr.it
M ULTIPLE L EVELS O F A BSTRACTION fabrizio.falchi@cnr.it
M ULTIPLE L EVELS O F A BSTRACTION fabrizio.falchi@cnr.it
O UR A PPROACH A detection scheme for adversarial images based on internal representation (aka deep features ) of the neural network classifier. • Main intuition : look at the evolution of features, i.e. the path formed by their positions in the feature spaces, during the forward pass of the network. • Claim : The trajectories traced by authentic inputs and adversarial examples differ and can be used to discern them. Adversarial examples detection in features distance spaces F. Carrara, R. Becarelli, R. Caldelli, F. Falchi, G. Amato ECCV WOCM Workshop 2018 fabrizio.falchi@cnr.it
M ULTIPLE L EVELS O F A BSTRACTION fabrizio.falchi@cnr.it
O UR A PPROACH : R ESULTS fabrizio.falchi@cnr.it
66 E ASY TO I DENTIFY A DVERSARIAL IMAGES fabrizio.falchi@cnr.it
H ARD TO I DENTIFY A DVERSARIAL I MAGES fabrizio.falchi@cnr.it
O THER D ETECTION A PPROACHES • Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods [2017] Nicholas Carlini, David Wagner • On Detecting Adversarial Perturbations [2017] Jan Hendrik Metzen, Tim Genewein, Volker Fischer, Bastian Bischoff • Trace and detect adversarial attacks on CNNs using feature response maps [2018] Mohammadreza, Friedhelm, Thilo • Adversarial examples detection in features distance spaces [2018] F. Carrara, R. Becarelli, R. Caldelli, F. Falchi, G. Amato fabrizio.falchi@cnr.it
R ELATED T OPICS
D ETECTING F ACE M ORPHING A TTACKS Detection of Face Morphing Attacks by Deep Learning C. Seibold, W. Samek, A. Hilsmann, P. Eisert fabrizio.falchi@cnr.it
A DVERSARIAL E XAMPLES D ETECTION HiDDeN: Hiding Data With Deep Networks Jiren Zhu, Russell Kaplan, Justin Johnson, Li Fei-Fei fabrizio.falchi@cnr.it
fabrizio.falchi@cnr.it
fabrizio.falchi@cnr.it
T HANKS ! Questions are welcomed Fabrizio Falchi fabrizio.falchi@cnr.it fabrizio.falchi@cnr.it
C ONCLUSIONS • Machine Learning and Deep Learning in particular can be attacked o Slightly modifying images but also in real world o Even if our neural network is a black box for the enemy • Many approaches have been proposed to make DL more robust • Adversarial examples detection is its early stages • We need adversary-aware machine learning fabrizio.falchi@cnr.it
Recommend
More recommend
