tutorial on interpreting and explaining deep models in
play

Tutorial on Interpreting and Explaining Deep Models in Computer - PowerPoint PPT Presentation

Mar 23, 2023 •46 likes •676 views

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire Montavon Klaus-Robert Mller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability

  1. Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grégoire Montavon Klaus-Robert Müller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability GM 10:00 - 10:30 Coffee Break ALL 10:30 - 11:15 Applications of Interpretability WS 11:15 - 12:00 Further Applications and Wrap-Up KRM

  2. Opening the Black Box with LRP Black Box CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 4

  3. Opening the Black Box with LRP Black Box Theoretical Interpretation ( Deep) Taylor decomposition Excitation Backprop (Zhang et al., 2016) is special case of LRP ( α =1). . CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 2

  4. LRP applied to different Data Molecules Text Analysis (Arras’16 &17) Translation (Ding’17) General Images (Bach’ 15, Lapuschkin’16) (Schütt’17) Speech (Becker’18) Morphing (Seibold’18) Games (Lapuschkin’18, in prep.) Video (Anders’18) VQA (Arras’18) Gait Patterns (Horst’18, in prep.) EEG (Sturm’16) Faces (Arbabzadeh’16, Lapuschkin’17) Digits (Bach’ 15) fMRI (Thomas’18) Histopathology (Binder’18) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 3

  5. LRP applied to different Models Convolutional NNs (Bach’15, Arras’17 …) Local Renormalization LSTM (Arras’17, Thomas’18) Layers (Binder’16) Bag-of-words / Fisher Vector models (Bach’15, Arras’16, Lapuschkin’17, Binder’18) One-class SVM (Kauffmann’18) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 4

  6. Now What ?

  7. Compare Explanation Methods Algorithm (“Pixel Flipping”) Sort pixels / patches by relevance Iterate Idea: Compare selectivity (Bach’15, Samek’17): destroy pixel / patch “If input features are deemed relevant, removing them evaluate f(x) should reduce evidence at the output of the network.” Measure decrease of f(x) Important : Remove information in a non-specific manner (e.g. sample from uniform distribution) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 6

  8. Compare Explanation Methods LRP CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 7

  9. Compare Explanation Methods LRP CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 8

  10. Compare Explanation Methods LRP CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 9

  11. Compare Explanation Methods LRP CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 10

  12. Compare Explanation Methods Sensitivity CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 11

  13. Compare Explanation Methods Sensitivity CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 12

  14. Compare Explanation Methods Sensitivity CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 13

  15. Compare Explanation Methods Random CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 14

  16. Compare Explanation Methods Random CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 15

  17. Compare Explanation Methods Random CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 16

  18. Compare Explanation Methods LRP: 0.722 Sensitivity: 0.691 Random: 0.523 LRP produces quantitatively better heatmaps than sensitivity analysis and random. What about more complex datasets ? SUN397 ILSVRC2012 MIT Places 397 scene categories 1000 categories 205 scene categories (108,754 images in total) (1.2 million training images) (2.5 millions of images) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 17

  19. Compare Explanation Methods Sensitivity Analysis Deconvolution Method LRP Algorithm (Simonyan et al. 2014) (Zeiler & Fergus 2014) (Bach et al. 2015) (Samek et al. 2017) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 18

  20. Compare Explanation Methods - ImageNet: Caffe reference model - Places & SUN: Classifier from MIT - AOPC averages over 5040 images - perturb 9 × 9 nonoverlapping regions - 100 steps (15.7% of the image) - uniform sampling in pixel space (Samek et al. 2017) LRP produces better heatmaps - Sensitivity heatmaps are noisy (gradient shuttering) - Deconvolution and sensitivity analysis solve a different problem CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 19

  21. Compare Explanation Methods Same idea can be applied for other “Pixel flipping” domains (e.g. text document classification) = “Word deleting” Text classified as “sci.med” —> LRP identifies most relevant words. (Arras et al. 2017) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 20

  22. Compare Explanation Methods - word2vec / CNN model Deleting most relevant Deleting least relevant - Conv → ReLU → 1-Max-Pool → FC from correctly classified from falsely classified - trained on 20Newsgroup Dataset - accuracy: 80.19% LRP better than SA LRP distinguishes between positive and negative evidence (Arras et al. 2016) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 21

  23. Compare Explanation Methods Deleting most relevant Deleting least relevant from correctly classified from falsely classified (Ding et al. ACL, 2017) - bidirectional LSTM model (Li’16) - Stanford Sentiment Treebank dataset - delete up to 5 words per sentence LRP outperforms baselines (also recently proposed contextual decomposition) LRP ≠ Gradient x Input (Arras et al. 2018) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 22

  24. Compare Explanation Methods Highly efficient (e.g., 0.01 sec per VGG16 explanation) ! New Keras Toolbox available for explanation methods: https://github.com/albermax/innvestigate CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 23

  25. Application of LRP Compare models

  26. Application: Compare Classifiers word2vec/CNN : Performance: 80.19% Strategy to solve the problem: identify semantically meaningful words related to the topic. BoW/SVM : Performance: 80.10% Strategy to solve the problem: identify statistical patterns, i.e., use word statistics (Arras et al. 2016 & 2017) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 25

  27. Application: Compare Classifiers word2vec / CNN model BoW/SVM m odel Words with maximum relevance (Arras et al. 2016 & 2017) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 26

  28. Application: Compare Classifiers BVLC: - 8 Layers - ILSRCV: 16.4% GoogleNet: - 22 Layers - ILSRCV: 6.7% - Inception layers CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 27

  29. Application: Compare Classifiers GoogleNet focuses on faces of animal. —> suppresses background noise BVLC CaffeNet heatmaps are much more noisy. Is it related to the architecture ? Is it related to the performance ? structure heatmap ? performance (Binder et al. 2016) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 28

  30. Application of LRP Quantify Context Use

  31. Application: Measure Context Use how important how important is context ? is context ? classifier relevance outside bbox LRP decomposition allows importance = of context meaningful pooling over bbox ! relevance inside bbox CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 30

  32. Application: Measure Context Use - BVLC reference model + fine tuning - PASCAL VOC 2007 (Lapuschkin et al., 2016) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 31

  33. Application: Measure Context Use BVLC CaffeNet - Differen models (BVLC CaffeNet, GoogleNet, VGG CNN S) - ILSVCR 2012 Context use Context use anti-correlated GoogleNet with performance. VGG CNN S BVLC CaffeNet GoogleNet VGG CNN S (Lapuschkin et al. 2016) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 32

  34. Application of LRP Compare Configuration, Detect Biases & Improve Models

  35. Application: Face analysis - Compare AdienceNet, CaffeNet, GoogleNet, VGG-16 - state-of-the-art performance in age and gender classification - Adience dataset, 26,580 images Age classification Gender classification A = AdienceNet [i] = in-place face alignment C = CaffeNet [r] = rotation based alignment G = GoogleNet [m] = mixing aligned images for training (Lapuschkin et al., 2017) V = VGG-16 [n] = initialization on Imagenet [w] = initialization on IMDB-WIKI 34

  36. Application: Face analysis Gender classification with pretraining without pretraining Strategy to solve the problem: Focus on chin / beard, eyes & hear, but without pretraining the model overfits (Lapuschkin et al., 2017) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 35

  37. Application: Face analysis Age classification Predictions 25-32 years old Strategy to solve the problem: Focus on the laughing … 60+ years old laughing speaks against 60+ (i.e., model learned that old people do not laugh) pretraining on ImageNet pretraining on IMDB-WIKI (Lapuschkin et al., 2017) CVPR 2018 Tutorial — W. Samek, G. Montavon & K.-R. Müller 36

Recommend

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire Montavon Klaus-Robert Mller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability

630 views • 33 slides
Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire Montavon Klaus-Robert Mller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability

416 views • 23 slides
Tutorial on Methods for Interpreting and Understanding Deep Neural Networks Wojciech Samek

Tutorial on Methods for Interpreting and Understanding Deep Neural Networks Wojciech Samek

Tutorial on Methods for Interpreting and Understanding Deep Neural Networks Wojciech Samek Grgoire Montavon Klaus-Robert Mller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 1:30 - 2:00 Part 1: Introduction 2:00 - 3:00 Part 2a: Making Deep

431 views • 29 slides
ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G. Montavon, W. Samek, K.-R. Mller Part 2: Making Deep Neural Networks Transparent 5 March 2017 Making Deep Neural Nets Transparent DNN transparency

448 views • 44 slides
From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box Models Huge volumes of data Solve task Computing power Deep Neural Network Information (implicit) ICIP18 Tutorial on Interpretable Deep

540 views • 33 slides
A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine Learning Summer School Buenos Aires, Argentina June 2018 lips.cs.princeton.edu @ryan_p_adams Tutorial Outline What is generative modeling?

2.33k views • 156 slides
4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept.

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept.

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept. of Neuroradiology, Heidelberg University Hospital Div. of Medical Image Computing, DKFZ Heidelberg Faculty of Physics & Astronomy, Heidelberg

1.48k views • 65 slides
outline of this tutorial motivations 1 ACISS09 tutorial on deep belief nets deep

outline of this tutorial motivations 1 ACISS09 tutorial on deep belief nets deep

outline of this tutorial motivations 1 ACISS09 tutorial on deep belief nets deep autoencoders deep belief nets sigmoid belief nets 2 Marcus Frean why are they hard to train? could layer-by-layer training work? Melbourne, 2009

509 views • 15 slides
Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information Engineering The Chinese University of Hong Kong Deep Neural Networks are Everywhere Playing Go Making Medical Decision Understanding Scenes Deep Neural

1.1k views • 46 slides
UCL Tutorial on: Deep Belief Nets (An updated and extended version of my 2007 NIPS tutorial)

UCL Tutorial on: Deep Belief Nets (An updated and extended version of my 2007 NIPS tutorial)

UCL Tutorial on: Deep Belief Nets (An updated and extended version of my 2007 NIPS tutorial) Geoffrey Hinton Canadian Institute for Advanced Research & Department of Computer Science University of Toronto Schedule for the Tutorial

1.66k views • 126 slides
Explaining Deep Learning Predictions and Isaac Ahern Integrating Domain Ontologies Outline

Explaining Deep Learning Predictions and Isaac Ahern Integrating Domain Ontologies Outline

Explaining Deep Learning Predictions and Integrating Domain Ontologies Explaining Deep Learning Predictions and Isaac Ahern Integrating Domain Ontologies Outline Project Background problems Isaac Ahern domains Explaining any

646 views • 35 slides
Deep Learning Tutorial Part I Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning

Deep Learning Tutorial Part I Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning

Deep Learning Tutorial Part I Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning Tutorial,Part I 1 Overview Goals of the tutorial Somewhat organized overview of basics, and some more advanced topics Demistify jargon Pointers for

1.25k views • 105 slides
Deep Learning Tutorial Part II Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning

Deep Learning Tutorial Part II Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning

Deep Learning Tutorial Part II Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning Tutorial,Part II 1 Overview Goals of the tutorial Somewhat organized overview of basics, and some more advanced topics Demistify jargon Pointers

925 views • 63 slides
Interpreting Models for Categorical and Count Outcomes Rose Medeiros StataCorp LLC Stata

Interpreting Models for Categorical and Count Outcomes Rose Medeiros StataCorp LLC Stata

Introduction Estimation Postestimation Conclusion Interpreting Models for Categorical and Count Outcomes Rose Medeiros StataCorp LLC Stata Webinar March 21, 2019 Interpreting Models for Categorical and Count Outcomes Handout page: 1

559 views • 54 slides
Interpreting Embedding Models of Knowledge Bases: Model Agnostic Approaches 2018 ICML Workshop on

Interpreting Embedding Models of Knowledge Bases: Model Agnostic Approaches 2018 ICML Workshop on

Introduction & Background Interpreting Embedding Models of KBs Experiments Conclusion Interpreting Embedding Models of Knowledge Bases: Model Agnostic Approaches 2018 ICML Workshop on Human Interpretability in Machine Learning Arthur C.

751 views • 24 slides
Why Did You Say That? Explaining and Diversifying Captioning Models Kate Saenko VQA Workshop,

Why Did You Say That? Explaining and Diversifying Captioning Models Kate Saenko VQA Workshop,

Why Did You Say That? Explaining and Diversifying Captioning Models Kate Saenko VQA Workshop, CVPR, July 26, 2017 Explaining: Top-down saliency guided by captions http://ai.bu.edu/caption-guided-saliency/ Vasili Abir Jianming Kate

726 views • 53 slides
Interpreting Fine-grained Dermatological Classification with Deep Learning S Mishra [1] , H

Interpreting Fine-grained Dermatological Classification with Deep Learning S Mishra [1] , H

Interpreting Fine-grained Dermatological Classification with Deep Learning S Mishra [1] , H Imaizumi [2] , T Yamasaki [1] 1 The University of Tokyo 2 ExMedio Inc ISIC Skin Image Analysis Workshop 1 Sco cope Analyze model accuracy gap on

542 views • 28 slides
Methods for interpreting and understanding deep neural networks Grgoire Montavon,

Methods for interpreting and understanding deep neural networks Grgoire Montavon,

Methods for interpreting and understanding deep neural networks Grgoire Montavon, Wojciech Samek, Klaus-Robert Mller 2017 Presented by Philipp Wimmer M o t i v a t i o n U n d e r s t a n d i n g a n d v

400 views • 22 slides
Data Science in the Wild Lecture 14: Explaining Models Eran Toch Data Science in the Wild,

Data Science in the Wild Lecture 14: Explaining Models Eran Toch Data Science in the Wild,

Data Science in the Wild Lecture 14: Explaining Models Eran Toch Data Science in the Wild, Spring 2019 1 Agenda 1. Explaining models 2. Transparent model explanations 3. Obscure model explanations 4. LIME: Local Interpretable

679 views • 37 slides
Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead Cynthia Rudin, Duke University Presenters : Sreya Dutta Roy, Ziqian Lin 1 Overview Introduction Explainable ML Vs

1.04k views • 39 slides
Toward a Toward a Overview Sociology of Sociology of Introduction Interpreting

Toward a Toward a Overview Sociology of Sociology of Introduction Interpreting

6/9/2010 Toward a Toward a Overview Sociology of Sociology of Introduction Interpreting Interpreting Traditional foci in Sign Language Interpreting Research What a sociology of interpreting would look like. Jeremy L. Brunson

82 views • 5 slides
Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation

Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation

Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation Marco Ancona , Cengiz ztireli 2 , Markus Gross 1,2 1 Department of Computer Science, ETH Zurich, Switzerland 2 Disney Research, Zurich, Switzerland

1.09k views • 41 slides
Explaining the Postgres Query Optimizer B RUCE M OMJIAN The optimizer is the "brain" of

Explaining the Postgres Query Optimizer B RUCE M OMJIAN The optimizer is the "brain" of

Explaining the Postgres Query Optimizer B RUCE M OMJIAN The optimizer is the "brain" of the database, interpreting SQL queries and determining the fastest method of execution. This talk uses the EXPLAIN command to show how the optimizer

576 views • 56 slides
Deep Learning for Natural Language Inference NAACL-HLT 2019 Tutorial Follow the slides:

Deep Learning for Natural Language Inference NAACL-HLT 2019 Tutorial Follow the slides:

Deep Learning for Natural Language Inference NAACL-HLT 2019 Tutorial Follow the slides: nlitutorial.github.io Sam Bowman Xiaodan Zhu NYU (New York) Queens University, Canada Introduction Motivations of the Tutorial Overview Starting

2.13k views • 182 slides

More recommend