Layer-wise Relevance Propagation in Neural Neural Networks Deep - PowerPoint PPT Presentation

LRP Ariyan Zarei Motivation Having More interpretable Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks to have more interpretable Papers and Demo Introduction Machine Learning models Terminology and Notations Relevance Properties Examples of Relevance Taylor Decomposition as Relevance Ariyan Zarei Layer-wise Relevance Propagation Local Layer-wise Relevance University of Arizona Notes on Relevance Rules General Algorithm ariyanzarei@email.arizona.edu LRP Rules LRP-0 LRP-Epsilon February 25, 2020 LRP-Gamma LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Conclusion

LRP Overview Ariyan Zarei Motivation Having More interpretable Neural Networks Motivation Deep Learning Shortcomings Having More interpretable Neural Networks Papers and Demo Deep Learning Shortcomings Introduction Papers and Demo Terminology and Notations Introduction Relevance Properties Terminology and Notations Relevance Properties Examples of Relevance Examples of Relevance Taylor Decomposition as Relevance Taylor Decomposition as Relevance Layer-wise Relevance Propagation Layer-wise Relevance Local Layer-wise Relevance Propagation Notes on Relevance Rules Local Layer-wise Relevance General Algorithm Notes on Relevance Rules General Algorithm LRP Rules LRP Rules LRP-0 LRP-0 LRP-Epsilon LRP-Epsilon LRP-Gamma LRP-Gamma LRP Rules Comparison Which Rule to use for each LRP Rules Comparison layer Different starting relevance Which Rule to use for each layer for the output layer Different starting relevance for the output layer Conclusion Conclusion

LRP Ariyan Zarei Motivation Having More interpretable Neural Networks Deep Learning Shortcomings Papers and Demo Introduction Terminology and Notations Relevance Properties Motivation Examples of Relevance Taylor Decomposition as Relevance Layer-wise Relevance Propagation Local Layer-wise Relevance Notes on Relevance Rules General Algorithm LRP Rules LRP-0 LRP-Epsilon LRP-Gamma LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Conclusion

LRP Having More interpretable Neural Networks Ariyan Zarei ◮ Interpretable Machine Learning (ML) Theme in our Motivation Colloquium Having More interpretable Neural Networks Deep Learning ◮ Medical Applications of ML, specially Medical Image Shortcomings Papers and Demo Analysis Introduction ◮ Deep Learning (DL) for analyzing histopathological Terminology and Notations Relevance Properties Slides Examples of Relevance Taylor Decomposition as Relevance Layer-wise Relevance Propagation Local Layer-wise Relevance Notes on Relevance Rules General Algorithm LRP Rules LRP-0 LRP-Epsilon LRP-Gamma LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Figure: A sampled window inside the cancerous region of a Slide Conclusion

LRP Deep Learning Shortcomings Ariyan Zarei Motivation ◮ Paying Attention to irrelevant and spurious features Having More interpretable Neural Networks Deep Learning Shortcomings Papers and Demo Introduction Terminology and Notations Relevance Properties Examples of Relevance Taylor Decomposition as Relevance Layer-wise Relevance Propagation Local Layer-wise Relevance Notes on Relevance Rules General Algorithm LRP Rules LRP-0 LRP-Epsilon LRP-Gamma LRP Rules Comparison Which Rule to use for each layer ◮ Feature Selection not useful. Different starting relevance for the output layer Conclusion

LRP Deep Learning Shortcomings Ariyan Zarei ◮ Paying Attention to irrelevant and spurious features Motivation Having More interpretable Simple example: Neural Networks Deep Learning Shortcomings Papers and Demo Introduction Terminology and Notations Relevance Properties Examples of Relevance Taylor Decomposition as Relevance Layer-wise Relevance Propagation Local Layer-wise Relevance Notes on Relevance Rules General Algorithm LRP Rules LRP-0 LRP-Epsilon LRP-Gamma LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Conclusion

LRP Deep Learning Shortcomings Ariyan Zarei Motivation Having More interpretable Neural Networks Deep Learning Shortcomings Papers and Demo Introduction Terminology and Notations Relevance Properties ◮ Deep Neural Networks’ Challenges Medical Sciences Examples of Relevance Taylor Decomposition as ◮ Fix this problem Relevance Layer-wise ◮ Explain the predictions of the Models Relevance Propagation Local Layer-wise Relevance Notes on Relevance Rules General Algorithm LRP Rules LRP-0 LRP-Epsilon LRP-Gamma LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Conclusion

LRP Papers and Demo Ariyan Zarei Motivation Having More interpretable Neural Networks Deep Learning Shortcomings Papers and Demo Introduction ◮ Layer-Wise Relevance Propagation: An Overview Terminology and Notations Relevance Properties (Explainable AI: Interpreting, Explaining and Visualizing Examples of Relevance Taylor Decomposition as Deep Learning Chapter 10) Relevance Layer-wise ◮ Explaining nonlinear classification decisions with deep Relevance Propagation Taylor decomposition (Elsevier Pattern Recognition) Local Layer-wise Relevance Notes on Relevance Rules Demo: Link General Algorithm LRP Rules LRP-0 LRP-Epsilon LRP-Gamma LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Conclusion

LRP Introduction Ariyan Zarei Motivation Having More interpretable Neural Networks Deep Learning Shortcomings Papers and Demo Why the neural network is making a particular decision. Introduction Terminology and Notations ◮ Assess and Validate the prediction and the reason Relevance Properties Examples of Relevance behind it with another inexpensive method. Taylor Decomposition as Relevance ◮ Given the final output of a class (softmax), where in the Layer-wise Relevance input the network is attending. Propagation Local Layer-wise Relevance ◮ Which parts of the input affect the prediction Notes on Relevance Rules General Algorithm (positively and negatively). LRP Rules LRP-0 LRP-Epsilon LRP-Gamma LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Conclusion

LRP Terminology and Notations Ariyan Zarei Motivation Having More interpretable Neural Networks Deep Learning Note: we focus on images and CNNs in this talk but LRP Shortcomings Papers and Demo can be applied to all other forms of data and networks and Introduction models. Terminology and Notations Relevance Properties ◮ Input Image: x ∈ R d = { x p } , p ∈ { 1 , 2 , ..., d } Examples of Relevance Taylor Decomposition as ◮ Prediction: f ( x ) : R d → R + quantifies the presence of Relevance Layer-wise an object in the input. Relevance Propagation ◮ Zero: absence of the object Local Layer-wise Relevance ◮ Other values: degree of certainty Notes on Relevance Rules General Algorithm ◮ Relevance: R ( x ) : R d → R + d Heatmap with the same LRP Rules LRP-0 LRP-Epsilon size as the input LRP-Gamma LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Conclusion

LRP Relevance Properties Ariyan Zarei Motivation Having More interpretable Neural Networks Deep Learning Shortcomings Papers and Demo Introduction Terminology and Notations 1. Conservation: ∀ x : f ( x ) = � p R ( x ) p Relevance Properties Examples of Relevance 2. Being Positive: ∀ x , p : R ( x ) p ≥ 0 Taylor Decomposition as Relevance Layer-wise 3. Consistent: if properties 1 and 2 hold. if Relevance f ( x ) = 0 ⇒ ∀ p : R ( x ) p = 0 Propagation Local Layer-wise Relevance Notes on Relevance Rules General Algorithm LRP Rules LRP-0 LRP-Epsilon LRP-Gamma LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Conclusion

LRP Examples of Relevance Ariyan Zarei Motivation Having More interpretable Neural Networks 1. Put all relevance to one pixel Deep Learning Shortcomings Papers and Demo 2. Divide the relevance equally between all input pixels Introduction ∀ p : R ( x ) p = 1 d f ( x ) Terminology and Notations Relevance Properties 3. Natural Decomposition: if the function f has some sort Examples of Relevance Taylor Decomposition as of natural decomposition between the input pixels. Relevance f ( x ) = � p f p ( x p ) ⇒ ∀ p : R ( x ) p = f p ( x p ) Layer-wise Relevance Propagation 4. Taylor Decomposition around a reference point. Local Layer-wise Relevance x ) + ( ∂ f x ) ⊤ ( x − ˜ f ( x ) = f (˜ ∂ x | x = ˜ x ) + ǫ Notes on Relevance Rules General Algorithm LRP Rules ∂ f f ( x ) = 0 + � ∂ x p | x = ˜ x ( x p − ˜ x p ) + ǫ LRP-0 p LRP-Epsilon LRP-Gamma ∀ p : R ( x ) p = ∂ f ∂ x p | x = ˜ x ( x p − ˜ x p ) LRP Rules Comparison Which Rule to use for each layer Different starting relevance for the output layer Conclusion