tutorial on interpreting and explaining deep models in
play

Tutorial on Interpreting and Explaining Deep Models in Computer - PowerPoint PPT Presentation

Nov 16, 2023 •291 likes •630 views

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire Montavon Klaus-Robert Mller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability

  1. Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grégoire Montavon Klaus-Robert Müller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability GM 10:00 - 10:30 Coffee Break ALL 10:30 - 11:15 Applications of Interpretability WS 11:15 - 12:00 Further Applications and Wrap-Up KRM

  2. O v e r v i e w o f E x p l a n a t i o n M e t h o d s Q u e s t i o n : W h i c h o n e t o c h o o s e ? 2 / 3 3

  3. F i r s t A t t e mp t : D i s t a n c e t o G r o u n d T r u t h i n p u t e v i d e n c e f o r D N N “ t r u c k ” e x p l a n a t i o n e r r o r g r o u n d ? ? ? t r u t h 3 / 3 3

  4. F i r s t A t t e mp t : D i s t a n c e t o G r o u n d T r u t h i n p u t e v i d e n c e f o r D N N “ c a r ” e x p l a n a t i o n e r r o r g r o u n d ? ? ? t r u t h ? ? ? 4 / 3 3

  5. F r o m G r o u n d T r u t h E x p l a n a t i o n s t o A x i o ms I d e a : E v a l u a t e t h e e x p l a n a t i o n t e c h n i q u e a x i o m a t i c a l l y , i . e . i t m u s t p a s s a n u m b e r o f p r e d e fi n e d “ u n i t t e s t s ” . [ S u n ’ 1 1 , B a c h ’ 1 5 , M o n t a v o n ’ 1 7 , S a m e k ’ 1 7 , S u n d a r a j a n ’ 1 7 , K i n d e r m a n s ’ 1 7 , M o n t a v o n ’ 1 8 ] . e x p l a n a t i o n t e c h n i q u e D N N 5 / 3 3

  6. P r o p e r t i e s 1 - 2 : C o n s e r v a t i o n a n d P o s i t i v i t y [ M o n t a v o n ’ 1 7 , s e e a l s o S u n ’ 1 1 , L a n d e c k e r ’ 1 3 , B a c h ’ 1 5 ] e x p l a n a t i o n D N N C o n s e r v a t i o n : T o t a l a t t r i b u t i o n o n t h e i n p u t f e a t u r e s s h o u l d b e p r o p o r t i o n a l t o t h e a m o u n t o f ( e x p l a i n a b l e ) e v i d e n c e a t t h e o u t p u t . P o s i t i v i t y : I f t h e n e u r a l n e t w o r k i s c e r t a i n a b o u t i t s p r e d i c t i o n , i n p u t f e a t u r e s a r e e i t h e r r e l e v a n t ( p o s i t i v e ) o r i r r e l e v a n t ( z e r o ) . 6 / 3 3

  7. P r o p e r t y 3 : C o n t i n u i t y [ M o n t a v o n ’ 1 8 ] I f t w o i n p u t s a r e t h e a l m o s t t h e s a m e , a n d t h e p r e d i c t i o n i s a l s o a l m o s t t h e s a m e , t h e n t h e e x p l a n a t i o n s h o u l d a l s o b e a l m o s t t h e s a m e . E x a mp l e : M e t h o d 1 M e t h o d 2 7 / 3 3

  8. T e s t i n g C o n t i n u i t y e x p l a n a t i o n i n p u t s c o r e s f ( x ) L R P - α β 1 0 R i S e n s i t i v i t y a n a l y s i s 8 / 3 3

  9. P r o p e r t y 4 : S e l e c t i v i t y [ B a c h ’ 1 5 , S a m e k ’ 1 7 ] M o d e l m u s t a g r e e w i t h t h e e x p l a n a t i o n : I f i n p u t f e a t u r e s a r e a t t r i b u t e d r e l e v a n c e , r e m o v i n g t h e m s h o u l d r e d u c e e v i d e n c e a t t h e o u t p u t . E x a mp l e : M e t h o d 1 M e t h o d 2 9 / 3 3

  10. T e s t i n g S e l e c t i v i t y w i t h P i x e l - F l i p p i n g [ B a c h ’ 1 5 , S a m e k ’ 1 7 ] e x p l a n a t i o n i n p u t s c o r e s f ( x ) L R P - α β 1 0 ( ) f x S e n s i t i v i t y a n a l y s i s 1 0 / 3 3

  11. t u t p u n p I n I x ) 2 P x ) 2 P t B t B n β n d m e d i e 0 r e i e α d o d - d 1 f i d a i i P a r u r u n R G G . U ( G G L ( . E x p l a n a t i o n t e c h n i q u e s . P r o p e r t i e s ✔ ✔ ✔ ✔ 1 . C o n s e r v a t i o n ✔ ✔ ✔ ✔ ✔ 2 . P o s i t i v i t y ✔ ✔ ✔ ✔ 3 . C o n t i n u i t y ✔ ✔ ✔ ✔ ✔ 4 . S e l e c t i v i t y . . . 1 1 / 3 3

  12. Q u e s t i o n : C a n w e d e d u c e s o m e p r o p e r t i e s w i t h o u t e x p e r i m e n t s , d i r e c t l y f r o m t h e e q u a t i o n s ? 1 2 / 3 3

  13. R e mi n d e r B a c k p r o p i n t e r n a l s ( f o r p r o p a g a t i n g g r a d i e n t ) L R P - i n t e r n a l s ( f o r p r o p a g a t i n g r e l e v a n c e ) α β 1 0 1 3 / 3 3

  14. E x a mp l e : D e d u c i n g C o n s e r v a t i o n v s . g r a d i n p u t × L R P - p r o p a g a t i o n r u l e α β 1 0 × i n p u t S u m m i n g g i v e s t h e p r o p e r t y W h e n b i a s i s n e g a t i v e , g r a d × i n p u t w i l l t e n d t o i n fl a t e s c o r e s . 1 4 / 3 3

  15. E x a mp l e : D e d u c i n g C o n t i n u i t y L R P - p r o p a g a t i o n r u l e α β v s . g r a d i n p u t × 1 0 ( w h e n b i a s n e g a t i v e , c o n t i n u i t y d u e t o d e n o m i n a t o r u p p e r - b o u n d i n g n u m e r a t o r . ) 1 5 / 3 3

  16. I n t e r me d i a t e C o n c l u s i o n G r o u n d - t r u t h e x p l a n a t i o n s a r e e l u s i v e . I n p r a c t i c e , w e a r e r e d u c e d t o v i s u a l a s s e s s m e n t o r t o t e s t t h e e x p l a n a t i o n f o r a n u m b e r o f a x i o m s . S o m e p r o p e r t i e s c a n b e d e d u c e d f r o m t h e s t r u c t u r e o f t h e e x p l a n a t i o n m e t h o d . O t h e r c a n b e t e s t e d e m p i r i c a l l y . α 1 β 0 - L R P s a t i s fi e s k e y p r o p e r t i e s o f a n e x p l a n a t i o n . S e n s i t i v i t y a n a l y s i s a n d g r a d i e n t i n p u t h a v e × c r u c i a l l i m i t a t i o n s . 1 6 / 3 3

  17. F r o m L R P t o D e e p T a y l o r D e c o mp o s i t i o n T h e L R P - α β r u l e 1 0 c a n b e s e e n a s a d e e p T a y l o r [ M o n t a v o n ’ 1 7 ] d e c o m p o s i t i o n ( D T D ) w h i c h t h e n y i e l d s d o m a i n - a n d l a y e r - s p e c i fi c r u l e s 1 7 / 3 3

  18. D T D : T h e S t r u c t u r e o f R e l e v a n c e P r o p o s i t i o n : R e l e v a n c e a t e a c h l a y e r i s a p r o d u c t o f t h e a c t i v a t i o n a n d a n a p p r o x i m a t e l y c o n s t a n t t e r m . 1 8 / 3 3

  19. D T D : T h e R e l e v a n c e a s a N e u r o n 1 9 / 3 3

  20. D T D : T a y l o r E x p a n s i o n o f t h e R e l e v a n c e 2 0 / 3 3

  21. D T D : D e c o mp o s i n g t h e R e l e v a n c e T a y l o r e x p a n s i o n a t r o o t p o i n t : R e l e v a n c e c a n n o w b e b a c k w a r d p r o p a g a t e d 2 1 / 3 3

  22. D T D : C h o o s i n g t h e R o o t P o i n t ( D e e p T a y l o r g e n e r i c ) C h o i c e o f r o o t p o i n t 1 . n e a r e s t r o o t ✔ ✔ 2 . r e s c a l e d a c t i v a t i o n 3 . r e s c a l e d e x c i t a t i o n s ✔ ✔ ( L R P - ) α β 1 0 2 2 / 3 3

  23. D T D : V e r i f y i n g t h e P r o d u c t S t r u c t u r e B u t w a s i t t r u e ? 1 . a s s u m e i t h o l d s i n h i g h e r - l a y e r 2 . a p p l y L R P - r u l e α β 1 0 3 . o b s e r v e i t a l s o h o l d s i n l o w e r - l a y e r 2 3 / 3 3

Recommend

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire Montavon Klaus-Robert Mller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability

676 views • 63 slides
Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire

Tutorial on Interpreting and Explaining Deep Models in Computer Vision Wojciech Samek Grgoire Montavon Klaus-Robert Mller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 08:30 - 09:15 Introduction KRM 09:15 - 10:00 Techniques for Interpretability

416 views • 23 slides
Tutorial on Methods for Interpreting and Understanding Deep Neural Networks Wojciech Samek

Tutorial on Methods for Interpreting and Understanding Deep Neural Networks Wojciech Samek

Tutorial on Methods for Interpreting and Understanding Deep Neural Networks Wojciech Samek Grgoire Montavon Klaus-Robert Mller (Fraunhofer HHI) (TU Berlin) (TU Berlin) 1:30 - 2:00 Part 1: Introduction 2:00 - 3:00 Part 2a: Making Deep

431 views • 29 slides
ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G. Montavon, W. Samek, K.-R. Mller Part 2: Making Deep Neural Networks Transparent 5 March 2017 Making Deep Neural Nets Transparent DNN transparency

448 views • 44 slides
From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box

From ML Successes to Applications ICIP18 Tutorial on Interpretable Deep Learning 2 Black Box Models Huge volumes of data Solve task Computing power Deep Neural Network Information (implicit) ICIP18 Tutorial on Interpretable Deep

540 views • 33 slides
A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine

A Tutorial on Deep Probabilistic Generative Models Ryan P. Adams Princeton University Machine Learning Summer School Buenos Aires, Argentina June 2018 lips.cs.princeton.edu @ryan_p_adams Tutorial Outline What is generative modeling?

2.33k views • 156 slides
4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept.

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept.

4 Deep Generative Models BVM 2018 Tutorial: Advanced Deep Learning Methods Jens Petersen Dept. of Neuroradiology, Heidelberg University Hospital Div. of Medical Image Computing, DKFZ Heidelberg Faculty of Physics & Astronomy, Heidelberg

1.48k views • 65 slides
outline of this tutorial motivations 1 ACISS09 tutorial on deep belief nets deep

outline of this tutorial motivations 1 ACISS09 tutorial on deep belief nets deep

outline of this tutorial motivations 1 ACISS09 tutorial on deep belief nets deep autoencoders deep belief nets sigmoid belief nets 2 Marcus Frean why are they hard to train? could layer-by-layer training work? Melbourne, 2009

509 views • 15 slides
Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information Engineering The Chinese University of Hong Kong Deep Neural Networks are Everywhere Playing Go Making Medical Decision Understanding Scenes Deep Neural

1.1k views • 46 slides
UCL Tutorial on: Deep Belief Nets (An updated and extended version of my 2007 NIPS tutorial)

UCL Tutorial on: Deep Belief Nets (An updated and extended version of my 2007 NIPS tutorial)

UCL Tutorial on: Deep Belief Nets (An updated and extended version of my 2007 NIPS tutorial) Geoffrey Hinton Canadian Institute for Advanced Research & Department of Computer Science University of Toronto Schedule for the Tutorial

1.66k views • 126 slides
Explaining Deep Learning Predictions and Isaac Ahern Integrating Domain Ontologies Outline

Explaining Deep Learning Predictions and Isaac Ahern Integrating Domain Ontologies Outline

Explaining Deep Learning Predictions and Integrating Domain Ontologies Explaining Deep Learning Predictions and Isaac Ahern Integrating Domain Ontologies Outline Project Background problems Isaac Ahern domains Explaining any

646 views • 35 slides
Deep Learning Tutorial Part I Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning

Deep Learning Tutorial Part I Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning

Deep Learning Tutorial Part I Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning Tutorial,Part I 1 Overview Goals of the tutorial Somewhat organized overview of basics, and some more advanced topics Demistify jargon Pointers for

1.25k views • 105 slides
Deep Learning Tutorial Part II Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning

Deep Learning Tutorial Part II Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning

Deep Learning Tutorial Part II Greg Shakhnarovich TTI-Chicago December 2016 Deep Learning Tutorial,Part II 1 Overview Goals of the tutorial Somewhat organized overview of basics, and some more advanced topics Demistify jargon Pointers

925 views • 63 slides
Interpreting Models for Categorical and Count Outcomes Rose Medeiros StataCorp LLC Stata

Interpreting Models for Categorical and Count Outcomes Rose Medeiros StataCorp LLC Stata

Introduction Estimation Postestimation Conclusion Interpreting Models for Categorical and Count Outcomes Rose Medeiros StataCorp LLC Stata Webinar March 21, 2019 Interpreting Models for Categorical and Count Outcomes Handout page: 1

559 views • 54 slides
Interpreting Embedding Models of Knowledge Bases: Model Agnostic Approaches 2018 ICML Workshop on

Interpreting Embedding Models of Knowledge Bases: Model Agnostic Approaches 2018 ICML Workshop on

Introduction & Background Interpreting Embedding Models of KBs Experiments Conclusion Interpreting Embedding Models of Knowledge Bases: Model Agnostic Approaches 2018 ICML Workshop on Human Interpretability in Machine Learning Arthur C.

751 views • 24 slides
Why Did You Say That? Explaining and Diversifying Captioning Models Kate Saenko VQA Workshop,

Why Did You Say That? Explaining and Diversifying Captioning Models Kate Saenko VQA Workshop,

Why Did You Say That? Explaining and Diversifying Captioning Models Kate Saenko VQA Workshop, CVPR, July 26, 2017 Explaining: Top-down saliency guided by captions http://ai.bu.edu/caption-guided-saliency/ Vasili Abir Jianming Kate

726 views • 53 slides
Interpreting Fine-grained Dermatological Classification with Deep Learning S Mishra [1] , H

Interpreting Fine-grained Dermatological Classification with Deep Learning S Mishra [1] , H

Interpreting Fine-grained Dermatological Classification with Deep Learning S Mishra [1] , H Imaizumi [2] , T Yamasaki [1] 1 The University of Tokyo 2 ExMedio Inc ISIC Skin Image Analysis Workshop 1 Sco cope Analyze model accuracy gap on

542 views • 28 slides
Methods for interpreting and understanding deep neural networks Grgoire Montavon,

Methods for interpreting and understanding deep neural networks Grgoire Montavon,

Methods for interpreting and understanding deep neural networks Grgoire Montavon, Wojciech Samek, Klaus-Robert Mller 2017 Presented by Philipp Wimmer M o t i v a t i o n U n d e r s t a n d i n g a n d v

400 views • 22 slides
Data Science in the Wild Lecture 14: Explaining Models Eran Toch Data Science in the Wild,

Data Science in the Wild Lecture 14: Explaining Models Eran Toch Data Science in the Wild,

Data Science in the Wild Lecture 14: Explaining Models Eran Toch Data Science in the Wild, Spring 2019 1 Agenda 1. Explaining models 2. Transparent model explanations 3. Obscure model explanations 4. LIME: Local Interpretable

679 views • 37 slides
Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable

Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead Cynthia Rudin, Duke University Presenters : Sreya Dutta Roy, Ziqian Lin 1 Overview Introduction Explainable ML Vs

1.04k views • 39 slides
Toward a Toward a Overview Sociology of Sociology of Introduction Interpreting

Toward a Toward a Overview Sociology of Sociology of Introduction Interpreting

6/9/2010 Toward a Toward a Overview Sociology of Sociology of Introduction Interpreting Interpreting Traditional foci in Sign Language Interpreting Research What a sociology of interpreting would look like. Jeremy L. Brunson

82 views • 5 slides
Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation

Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation

Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation Marco Ancona , Cengiz ztireli 2 , Markus Gross 1,2 1 Department of Computer Science, ETH Zurich, Switzerland 2 Disney Research, Zurich, Switzerland

1.09k views • 41 slides
Explaining the Postgres Query Optimizer B RUCE M OMJIAN The optimizer is the "brain" of

Explaining the Postgres Query Optimizer B RUCE M OMJIAN The optimizer is the "brain" of

Explaining the Postgres Query Optimizer B RUCE M OMJIAN The optimizer is the "brain" of the database, interpreting SQL queries and determining the fastest method of execution. This talk uses the EXPLAIN command to show how the optimizer

576 views • 56 slides
Deep Learning for Natural Language Inference NAACL-HLT 2019 Tutorial Follow the slides:

Deep Learning for Natural Language Inference NAACL-HLT 2019 Tutorial Follow the slides:

Deep Learning for Natural Language Inference NAACL-HLT 2019 Tutorial Follow the slides: nlitutorial.github.io Sam Bowman Xiaodan Zhu NYU (New York) Queens University, Canada Introduction Motivations of the Tutorial Overview Starting

2.13k views • 182 slides

More recommend