Adversarial Training for Deep Learning : A Framework for Improving - PowerPoint PPT Presentation

Adversarial Training for Deep Learning : A Framework for Improving Robustness, Generalization and Interpretability Zhanxing Zhu School of Mathematical Sciences, Peking University 
 zhanxing.zhu@pku.edu.cn 
 https://sites.google.com/view/zhanxingzhu/

The Success of Deep Learning • Computer vision • Human-level image recognition 
 performance on ImageNet, 
 eg. ResNet and variants… • Natural language processing • Excellent neural machine translation • Dialog generation • Game play • Reinforcement learning + deep learning: 
 AlphaGo, AlphaGo Zero, AlphaZero… • …

<latexit sha1_base64="FlU3TxvbwpvAOItbMDmscx+Q1M=">ACTnicbVFNaxsxENW6beI6X257EXUCXgpMStfEgiFkF56yCGFOg54zaLVztrC2tUizYaYZX9hLyW3/oxeckgprewY0nw8EDy9N8NonuJCSYtB8NrvHj5am29+bq1sbm1vdN+8/bc6tIGAitLmIuQUlcxigRAUXhQGexQqG8ezwh9egrFS59wXsA45NcplJwdFLUht20e3VEw1irJOM4rUKcAvLap5/oMDqloZWTjHeHUXW6z+oH134dikSjvRf795Rd0Y80jpjv+9TfjdqdoBcsQZ8StiIdsJZ1L4OEy3KDHIUils7YkGB4oblEJB3QpLCwUXMz6BkaM5z8COq2UcNd1zSkJTbdzJkS7V/zsqnlk7z2JXuVjZPvYW4nPeqMT0cFzJvCgRcnE3KC0VRU0X2dJEGhCo5o5wYaR7KxVTbrhA9wMtFwJ7vPJTct7vsaDHvY7xyerOJrkPflAuoSRA3JMvpAzMiCfCe/yC357f3wbrw/3t+70oa36nlHqDR/AeriLBk</latexit> <latexit sha1_base64="FlU3TxvbwpvAOItbMDmscx+Q1M=">ACTnicbVFNaxsxENW6beI6X257EXUCXgpMStfEgiFkF56yCGFOg54zaLVztrC2tUizYaYZX9hLyW3/oxeckgprewY0nw8EDy9N8NonuJCSYtB8NrvHj5am29+bq1sbm1vdN+8/bc6tIGAitLmIuQUlcxigRAUXhQGexQqG8ezwh9egrFS59wXsA45NcplJwdFLUht20e3VEw1irJOM4rUKcAvLap5/oMDqloZWTjHeHUXW6z+oH134dikSjvRf795Rd0Y80jpjv+9TfjdqdoBcsQZ8StiIdsJZ1L4OEy3KDHIUils7YkGB4oblEJB3QpLCwUXMz6BkaM5z8COq2UcNd1zSkJTbdzJkS7V/zsqnlk7z2JXuVjZPvYW4nPeqMT0cFzJvCgRcnE3KC0VRU0X2dJEGhCo5o5wYaR7KxVTbrhA9wMtFwJ7vPJTct7vsaDHvY7xyerOJrkPflAuoSRA3JMvpAzMiCfCe/yC357f3wbrw/3t+70oa36nlHqDR/AeriLBk</latexit> <latexit sha1_base64="FlU3TxvbwpvAOItbMDmscx+Q1M=">ACTnicbVFNaxsxENW6beI6X257EXUCXgpMStfEgiFkF56yCGFOg54zaLVztrC2tUizYaYZX9hLyW3/oxeckgprewY0nw8EDy9N8NonuJCSYtB8NrvHj5am29+bq1sbm1vdN+8/bc6tIGAitLmIuQUlcxigRAUXhQGexQqG8ezwh9egrFS59wXsA45NcplJwdFLUht20e3VEw1irJOM4rUKcAvLap5/oMDqloZWTjHeHUXW6z+oH134dikSjvRf795Rd0Y80jpjv+9TfjdqdoBcsQZ8StiIdsJZ1L4OEy3KDHIUils7YkGB4oblEJB3QpLCwUXMz6BkaM5z8COq2UcNd1zSkJTbdzJkS7V/zsqnlk7z2JXuVjZPvYW4nPeqMT0cFzJvCgRcnE3KC0VRU0X2dJEGhCo5o5wYaR7KxVTbrhA9wMtFwJ7vPJTct7vsaDHvY7xyerOJrkPflAuoSRA3JMvpAzMiCfCe/yC357f3wbrw/3t+70oa36nlHqDR/AeriLBk</latexit> <latexit sha1_base64="FlU3TxvbwpvAOItbMDmscx+Q1M=">ACTnicbVFNaxsxENW6beI6X257EXUCXgpMStfEgiFkF56yCGFOg54zaLVztrC2tUizYaYZX9hLyW3/oxeckgprewY0nw8EDy9N8NonuJCSYtB8NrvHj5am29+bq1sbm1vdN+8/bc6tIGAitLmIuQUlcxigRAUXhQGexQqG8ezwh9egrFS59wXsA45NcplJwdFLUht20e3VEw1irJOM4rUKcAvLap5/oMDqloZWTjHeHUXW6z+oH134dikSjvRf795Rd0Y80jpjv+9TfjdqdoBcsQZ8StiIdsJZ1L4OEy3KDHIUils7YkGB4oblEJB3QpLCwUXMz6BkaM5z8COq2UcNd1zSkJTbdzJkS7V/zsqnlk7z2JXuVjZPvYW4nPeqMT0cFzJvCgRcnE3KC0VRU0X2dJEGhCo5o5wYaR7KxVTbrhA9wMtFwJ7vPJTct7vsaDHvY7xyerOJrkPflAuoSRA3JMvpAzMiCfCe/yC357f3wbrw/3t+70oa36nlHqDR/AeriLBk</latexit> <latexit sha1_base64="2ACtyip0MFR/NmkKfx0NY4O8aM4=">ACc3icbVHLahsxFNVMX6n7clvoJouKuAGnpGYmLbQAqHZFCnUSsJxBI9+xRSTNIN0pMWJ+IJ/Xf+im+4r20MfS8IDufcqyOdm1dKOkyS71F86/adu/fW7ncePHz0+En36bMTV9ZWwFCUqrRnOXegpIEhSlRwVlngOldwml8cLPTr2CdLM0XnFcw1nxqZCEFx0Bl3Sumpck8wxkgbyhTUCDz9FN/xWzRPcoKy4VPG38UdFfrzMu9tDk/YqAU7Rf9y0zut3bdJ7JLcqsnM6QNZ1NuhLOXy/uQbhEz+0ODZ/LH9bZd1eMkiWRW+CtAU90tZx1v3GJqWoNRgUijs3SpMKx8EBpVDQdFjtoOLigk9hFKDhGtzYLzNr6GZgJrQobTgG6ZL9e8Jz7dxc56FTc5y569qC/J82qrH4MPbSVDWCESujolYUS7pYAJ1ICwLVPAurAxvpWLGQ8QY1tQJIaTXv3wTnOwM0reDnc/vevsf2zjWyDrZIH2SkvdknxySYzIkgvyIXkQvIxr9jNfjfjVqjWO2pn5J+K3/wCj4G8Qg=</latexit> Deep Neural Networks f ( x ; θ ) = W L σ ( W L − 1 σ ( W L − 2 · · · σ ( W 2 σ ( W 1 x + b 1 )))) human 
 or animal? ( ) N L ( ✓ ) = 1 X min ` ( f ( x i ; ✓ ) , y i ) N θ i =1 highly non-convex/ 
 multiple global minima

Why does deep learning work in these cases? Does it really work?

A Holistic View on Deep Learning Data Model Learning Minimizing the training loss Minima/ 
 Loss Landscape Solution Test 
 (Generalization/ Robustness/ 
 Interpretability)

Deep Learning Theory ๏ Representation power of deep neural networks ๏ Generalization : why deep nets still generalize well with over-parameterization 
 (ICML’17 W) 
 # training samples << # parameters ๏ Understanding training process ‣ Why does stochastic gradient descent work? (ICML’19a) ‣ Better optimization algorithms (NIPS’15, AAAI’16, NIPS’17, IJCAI’18, NIPS’18a) ๏ Robustness: adversarial examples and its defense mechanism 
 (NeurIPS’18b, ICML’19b, CVPR’19 Oral, NeurIPS’19, ICLR’20a,b under review)

Benefits of Studying Deep Learning Theory • Help to design better models and algorithms for practical use • Know CAN and CAN NOT: what is the limit of deep learning models? • Model-level, statistically, algorithmically, and computationally. • Raise more interesting mathematical problems • Understanding compositional and over-parameterized computational structure • Many more…

Does deep learning really work? ?

<latexit sha1_base64="EDaYHDKsP+WqxHAqS2T1WD3gJgI=">ACDHicbVDLTgIxFL3jE/GFunTSAywkMygiS6Jbly4wEQeCTMhndKBhs7DtmMgwAe48VfcuNAYt36AO/GArNQ8CRNT845N+09bsSZVKb5bSwtr6yurac20ptb2zu7mb39mgxjQWiVhDwUDRdLylAq4opThuRoNh3Oa27vauJX3+gQrIwuFODiDo+7gTMYwQrLbUyWXvk5fu5AjpB+i4ge4RsTu/RjSb9nFb79kinzKI5BVokVkKykKDSynzZ7ZDEPg0U4VjKpmVGyhlioRjhdJy2Y0kjTHq4Q5uaBtin0hlOlxmjY620kRcKfQKFpurviSH2pRz4rk76WHXlvDcR/OasfIunCELoljRgMwe8mKOVIgmzaA2E5QoPtAE8H0XxHpYoGJ0v2ldQnW/MqLpFYqWqfF0u1ZtnyZ1JGCQziCPFhwDmW4hgpUgcAjPMrvBlPxovxbnzMoktGMnMAf2B8/gDLxJhN</latexit> Failure of Deep Learning in Adversarial Environments • Deep neural networks are easily fooled by adversarial examples! f(x; w*) P(“panda”) = 57.7% Uncontrollable Lipschitz constant k f ( x 0 ) � f ( x ) k  L k x 0 � x k f(x+eta; w*) P(“gorilla”) = 99.3% ?!

Various Types of Adversarial Attacks ๏ One-pixel attack (Su et.al 2017)

• Universal adversarial 
 perturbation 
 (Moosavi-Dezfooli et.al 2017)

• Adversarial Patch (Brown et.al 2017, Thys et.al 2019) • Spatially transformed attacks (Brown et.al 2017)

๏ 3D adversarial examples Athalye et.al . Synthesizing Robust Adversarial Examples. ICML 2018

Ubiquitousness of Adversarial Examples ๏ Natural language processing Input reviewaaa x … made one of the best films… CNN Positive x 1 x 2 x 3 x 4 x 5 x 6 Substitution words best made films better of accomplished one the movies finest delivered film S ( x, 2) S ( x, 3) S ( x, 4) nicest cinema S ( x, 1) good S ( x, 6) S ( x, 5) Perturbed reviewaaa ˜ x …delivered one of the better movies… CNN Negative ˜ ˜ ˜ ˜ ˜ ˜ x 2 x 4 x 5 x 1 x 3 x 6 ๏ Speech recognition Fig. from Jia et.al 2019 ‣ Some examples Jia et.al. Certified robustness to adversarial word substitutions. EMNLP 2019. Qin et.al . Imperceptible, Robust and Targeted Adversarial Examples for Automatic Speech Recognition Fig. from Carlini and Wagner 2019 ICML 2019

Weak Robustness of 
 Current Deep Learning Systems • Neural networks are fragile, vulnerable, not robust as expected • A large gap between deep networks and human visual systems • Serious security issues arise when deploying AI systems based on neural networks • Autonomous vehicles / medical and health domains

<latexit sha1_base64="grwBElVeYwroFHmc23iRgKY59KE=">AB83icbVDLSsNAFJ34rPVdelmsAjtpiRWfOCm6MZlhb6gCWUynbRDJw9mbsQS+htuXCji1p9x5984SYOo9cCFwzn3cu89biS4AtP8NJaWV1bX1gsbxc2t7Z3d0t5+R4WxpKxNQxHKnksUEzxgbeAgWC+SjPiuYF13cpP63XsmFQ+DFkwj5vhkFHCPUwJasr1Kq/JwZTMg1eqgVDZrZga8SKyclFGO5qD0YQ9DGvsACqIUn3LjMBJiAROBZsV7VixiNAJGbG+pgHxmXKS7OYZPtbKEHuh1BUAztSfEwnxlZr6ru70CYzVXy8V/P6MXgXTsKDKAYW0PkiLxYQpwGgIdcMgpiqgmhkutbMR0TSjomIpZCJcpzr5fXiSdk5pVr9XvTsuN6zyOAjpER6iCLHSOGugWNVEbURShR/SMXozYeDJejbd565KRzxygXzDevwB6T5DO</latexit> Constructing Adversarial Examples ๏ An optimization problem f ( T ( x ; η )) ‣ Fast Gradient Sign Method (FGSM, Goodfellow et.al 2015) white-box attacks l ∞ norm ‣ Projected Gradient Descent (Iterative Gradient Method)

More unfortunately… adversarial examples can transfer ๏ Adversarial examples constructed based on f(x) can also easily fool another network g(x), even without any queries VGG White-box attack adversarial 
 f(x) P(“gibbon”) = 99.3% example Black-box attack g(x) P(“gibbon”) = 89% ResNet Lei Wu and Zhanxing Zhu. Understanding and Enhancing the Transferability of Adversarial Examples, arXiv-preprint.

How can we defense adversarial examples? Learning with involvement of adv. examples Adversarial Learning