Meta-Explanations, Interpretable Clustering & Other Recent - PowerPoint PPT Presentation

Fraunhofer 
 Image Processing Heinrich Hertz Institute Meta-Explanations, Interpretable Clustering & Other Recent Developments Fraunhofer HHI, Machine Learning Group Wojciech Samek ICCV 2019 Visual XAI Workshop Seoul, Korea, 2th November 2019


 Explaining Predictions Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 2


 Today’s Talk Which one to choose ? Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 3


 Today’s Talk From individual explanations to common prediction strategies Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 4


 Today’s Talk What can we do with it ? Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 5


 Today’s Talk Explaining more than classifiers Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 6

Explanation Methods


 Explanation Methods Perturbation-Based Function-Based Occlusion-Based (Zeiler & Fergus 14) Sensitivity Analysis (Simonyan et al. 14) Meaningful Perturbations (Fong & Vedaldi 17) (Simple) Taylor Expansions … Gradient x Input (Shrikumar et al. 16) … Structure-Based Surrogate- / Sampling-Based LIME (Ribeiro et al. 16) LRP (Bach et al. 15) SmoothGrad (Smilkov et al. 16) Deep Taylor Decomposition (Montavon et al. 17) … Excitation Backprop (Zhang et al. 16) … Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 8


 Approach 1: Perturbation Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 9


 Approach 1: Perturbation Disadvantages - slow - assumes locality - perturbation may introduce artefacts —> unreliable Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 10


 Approach 2: (Simple) Taylor Expansions Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 11


 Approach 2: (Simple) Taylor Expansions Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 12


 Approach 3: Gradient x Input Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 13


 Approach 3: Gradient x Input Observation : Explanations are noise Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 14


 Approach 3: Gradient x Input Two reasons why gradient-based explanation are noisy Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 15


 Layer-wise Relevance Propagation hard to explain easy to explain (Bach et al., 2015 Montavon et al. 2017) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 16


 Layer-wise Relevance Propagation Black Box Layer-wise Relevance Propagation (LRP) (Bach et al., PLOS ONE, 2015) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 17


 Layer-wise Relevance Propagation Classification cat rooster dog Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 18


 Layer-wise Relevance Propagation Theoretical interpretation Deep Taylor Decomposition (Montavon et al., 2017) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 19


 Layer-wise Relevance Propagation Theoretical interpretation Deep Taylor Decomposition (Montavon et al., 2017) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 19


 Layer-wise Relevance Propagation Explanation cat rooster dog Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 20


 Equivalence = Deep Taylor Decomposition Layer-wise Relevance Propagation (Montavon’17, arXiv in 2015) (Bach’15) = = Marginal Winning Probability A1 activations non-negative Excitation Backprop (Zhang’16) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 21


 Simple Taylor Decomposition Limitations: Idea : Use Taylor expansion to redistributed - difficult to find good root point relevance from output to input - gradient shattering Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 22


 Deep Taylor Decomposition Advantage: Idea : Use Taylor expansion to redistributed - easy to find good root point relevance from one layer to another - no gradient shattering Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 23


 Deep Taylor Decomposition (Montavon et al., 2017) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 24


 Deep Taylor Decomposition (Montavon et al., 2017) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 25


 Various LRP Rules Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 26


 Best Practice for LRP Principle : Explain each layer type (input, conv., fully connected layer) with the optimal rule according to DTD. (Montavon et al., 2019) (Kohlbrenner et al., 2019) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 27

Which one to choose ?


 Evaluating Explanations Perturbation Analysis [Bach’15, Samek’17, Arras’17, …] Pointing Game [Zhang’16] Using Axioms [Montavon’17, Sundararajan’17, Lundberg’17, …] Solve other Tasks Task Specific Evaluation [Arras’17, Arjona-Medina’18, …] [Poerner’18] Using Ground Truth Human Judgement [Arras’19] [Ribeiro’16, Nguyen’18 …] Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 29

Applications of XAI


 LRP Applied to Different Problems General Images (Bach’ 15, Lapuschkin’16) Text Analysis (Arras’16 &17) Speech (Becker’18) Morphing Attacks (Seibold’18) Games (Lapuschkin’19) VQA (Samek’19) Video (Anders’19) Gait Patterns (Horst’19) EEG (Sturm’16) Faces (Lapuschkin’17) fMRI (Thomas’18) Digits (Bach’ 15) Histopathology (Hägele’19) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 31


 LRP Applied to Different Models Convolutional NNs (Bach’15, Arras’17 …) LSTM (Arras’17, Arras’19) “Explaining and Interpreting LSTMs” (with S. Hochreiter) BoW / Fisher Vector models (Bach’15, Arras’16, Lapuschkin’16 …) One-class SVM (Kauffmann’18) Clustering (Kauffmann’19) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 32


 Unmasking Clever Hans Predictors Leading method (Fisher-Vector / SVM Model) of PASCAL VOC challenge Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 33


 Unmasking Clever Hans Predictors Leading method (Fisher-Vector / SVM Model) of PASCAL VOC challenge Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 34


 Unmasking Clever Hans Predictors ‘horse’ images in PASCAL VOC 2007 Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 35


 Identifying Biases Smiling as a contradictor of age Predictions 25-32 years old Strategy to solve the problem: Focus on the laughing … 60+ years old laughing speaks against 60+ (i.e., model learned that old people do not laugh) pretraining on ImageNet State-of-the-art DNN model, Adience Dataset (26k faces) (Lapuschkin et al. 2017) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 36


 Scientific Insights Our approach: - Recurrent neural networks (CNN + LSTM) for whole-brain analysis - LRP allows to interpret the results (Thomas et al. 2018) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 37


 Scientific Insights Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 38


 Understanding Learning Behaviour (Lapuschkin et al., 2019) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 39


 Understanding Learning Behaviour (Lapuschkin et al., 2019) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 39


 Understanding Learning Behaviour model learns 1. track the ball 2. focus on paddle 3. focus on the tunnel (Lapuschkin et al., 2019) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 40


 Understanding Learning Behaviour (Lapuschkin et al., 2019) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 41


 Understanding Learning Behaviour (Lapuschkin et al., 2019) Wojciech Samek: Meta-Explanations, Interpretable Clustering & Other Recent Developments 41

Meta-Explanations