Visualization for Explainable Classifiers Yao MING THE HONG KONG - PowerPoint PPT Presentation

A Survey on Visualization for Explainable Classifiers Yao MING THE HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY

In Intro rodu duction Ex Expl plai ainabl able Cl Classifiers Visualization f for E Explainable Cl Classifiers Conc Conclusion on H K U S T 2

In Introductio ion Mo Motivat ation Concep Con epts Ex Explainable Cl Classifier ers Visualization f for E Explainable Cl Classifier ers Conclusion Con on H K U S T 3

Mo Moti tivati tion https://xkcd.com/1838/ Does this matter? H K U S T 4

Mo Moti tivati tion A study from Cost-Effective HealthCare (CEHC) (Cooper et al. 1997) Predicting the probability of death (POD) for patients with pneumonia If HighRisk(x): admit to hospital Else: treat as outpatient The rule-based model learned: HasAsthma(x) => LowerRisk(x) High risk --> aggressive treatment We want the system to be explainable sometime! H K U S T 5

Mo Moti tivati tion “ Strategy 2: Developing Effective Methods for AI-Human Collaboration Better visualization and user interfaces are additional areas that need much greater development to help humans understand large-volume modern datasets and information coming from a variety of sources. ” H K U S T 6

Mo Moti tivati tion The concept of XAI. DARPA, Explainable AI Project 2017 H K U S T 7

Cl Classification on Identifying any observation 𝒚 ∈ 𝒴 as a class 𝑧 ∈ 𝒵 , Classification: 𝒵 = {1,2, … , 𝐿} , given a training set 𝒠 ⊂ 𝒴 × 𝒵 An algorithm 𝑔 , learned from 𝒠 , specified by parameters 𝜄 , Classification Model output is a vector representing a probability distribution: (Classifier): 𝒛 = 𝑔 & 𝒚 , where 𝒛 = 𝑧 ) ∈ ℝ - , 𝑧 ) = 𝑞 𝑧 = 𝑗 𝒚, 𝒠 . 𝒚 𝑔 𝒛 & H K U S T 8

What i is ex expla lainabili lity ty? The explainability of a classifier: The ability to explain the reasoning of its predictions so that humans can understand. (Doshi-Velez and Kim 2017) Aliases in literature: interpretability, intelligibility DARPA, Explainable AI Project 2017 H K U S T 9

Why e explainable? The Curiosity of Humans • What has the classifier learned from the data? Limitations of Machines • Human knowledge as a complement Moral and Legal Issues • The "right to explanation" • Fairness (non-discrimination) H K U S T 10

Why e explainable? The Curiosity of Humans • What has the classifier learned from the data? Zeiler and Fergus 2014 H K U S T 11

Why e explainable? Limitations of Machines • Human knowledge as a complement • Robustness of the model Adversarial examples attack (https://blog.openai.com/adversarial-example-research/) H K U S T 12

Why e explainable? Moral and Legal Issues • The "right to explanation” The EU general data protection regulation (GDPR 2018) Recital 71: In any case, such processing should be subject to suitable safeguards, which should include specific information to the data subject and the right to obtain human intervention, to express his or her point of view, to obtain an explanation of the decision reached after such assessment and to challenge the decision. • Fairness (non-discrimination) - Classification systems for loan approval. - Resume filter for hiring. H K U S T 13

In Intr trod oduction tion Ex Explaina nable Cl Classifier ers Interpretable A Architecture Explaining C Complex C Classifiers Visualization f for E Explainable Cl Classifier ers Con Conclusion on H K U S T 14

Explainable C Classifier Two strategies to provide explainability: Interpretable Architecture Interpretable Learning Sparse Models Classifiers Explainable Classifiers Explaining Complex Local Explanation Classifiers Global Explanation Explanation 𝑔 𝒚 𝒛 & Interpretable H K U S T 15

Interpretable C Classifiers Classifiers that are commonly recognized as understandable, and hence need little effort to explain them 𝑔 𝒚 𝒛 & Interpretable architecture: - 𝑔 consists of computation blocks that are easy to understand - E.g., decision trees Learning sparse models: - |𝜄| is smaller so that it is easy to understand - E.g., simplification H K U S T 16

Interpretable C Classifiers Classifiers that are commonly recognized as understandable, and hence need little effort to explain them 𝑔 𝒚 𝒛 & Not as explainable as they seemed to be! H K U S T 17

Interpretable C Classifiers Interpretable Architecture – Classic Methods kNN (instance-based) t is classified as Y because a, b, and c are similar to t. Limits: lack close instances to t Decision Tree (rule-based) Seem to be interpretable Limits: performance V.S. explainability H K U S T 18

Explainable C Classifier Two strategies to provide explainability: • Interpretable Classifiers • Explaining Complex Classifiers Explanation 𝑔 𝒚 𝒛 & Interpretable H K U S T 19

Explaining C Complex C Classifiers What are explanations of classifiers? Cognitive Science (Lombrozo 2006) : Explanations are characterized as arguments that demonstrate all or a subset of the causes of the explanandum (the subject being explained), usually following deductions from natural laws or empirical conditions. What is the explanandum? 1. The prediction of the classifier. ( Local explanation ) Why is 𝒚 classified as 𝑧 ? • A summary of local 2. The classifier itself. ( Global explanation ) explanations on 𝒴 • What has the classifier learned in general? H K U S T 20

Explaining C Complex C Classifiers What is explanations? Cognitive Science (Lombrozo 2006) : Arguments … of the causes of the explanandum … What are the causes of the prediction(s) of a classifier? 𝑔 𝒚 𝒛 & 1. Inputs 2. Model/ 3*. Training Parameters Data Model-aware / Model-unaware H K U S T 21

Explaining C Complex C Classifiers H K U S T 22

Local e explanations Sensitivity Analysis - Why is 𝒚 classified as 𝑧 ? Gradients (ImageNet 2013) => ? 1. Too noisy! =𝒚 (𝒚 ABCA ) (Simonyan et al. 2014) 2. High grad => important? H K U S T 23

Local e explanations Sensitivity Analysis - Why is 𝒚 classified as 𝑧 ? M 1 𝑜 F 𝜖𝑧 ) SmoothGrad (Smilkov et al. 2017) 𝜖𝒚 (𝒚 ABCA + 𝒪(0, 𝜏 L )) Sampling noisy images and average the gradient map NOP Limit: Expensive; Non-deterministic H K U S T 24

Local m model-aware e explanations Utilizing the structure of the model - CNN De-convolution (Zeiler and Fergus 2014): Inverse operations of different layers Pros: • Can apply to neurons • Better explanations Cons: • Only for layer-wise, invertible models • No relations H K U S T 25

Local m model-unaware e explanations Model Induction Locally approximate a complex classifier using a simple one (linear) 0-1 explanation (Ribeiro et al. 2016) Limits: 1. induction of a simple one is by random sampling local points; 2. expensive 3. generating image patch require extra efforts H K U S T 26

Global m model-unaware e explanations Sampling local explanations 1. Select top-k instances with max activations (Zeiler and Fergus 2014) 2. Select local explanations that greedily covers the most important features (Ribeiro et al. 2016) Limit to the data; special case; expensive H K U S T 27

Explainable C Classifiers The lack of human in the study! Visualization for Explainable Classifier Explanation 𝑔 𝒚 𝒛 & Interpretable H K U S T 28

Intr In trod oduction tion Explainable C Classifiers Visualization f for E Explainable Cl Classifier ers Vis f for E Exploratory D Data A Analysis Vis f for M Model D Development Vis f for O Operation Conclusion Con on H K U S T 29

Visualization f for E Explainable C Classifiers What r role i is v visualization p playing i in e explainable c classifiers? DARPA, Explainable AI Project 2017 H K U S T 30

The L Life C Cycle o of a a C Classifier Problem Problem Collection Collection Analysis Preparation Analysis Preparation Architecture Architecture Training Training Evaluation Evaluation Deployment Deployment Operation Operation De fj nition De fj nition Data Engineering Data Engineering Model Development Model Development Operation Operation H K U S T 31

What a are t the p problems? Vis f for E Exploratory D Data A Analysis - - What does my dataset look like? Any mislabels? What does my dataset look like? Any mislabels? Vis f for M Model D Development - Architecture: What is the classifier? How to compute? - Training: How the model gradually improves? How to diagnose? - Evaluation: What has the model learned from the data? - Comparison: Which classifier should I choose? Vis f for O Operation - Deploy: How to establish users’ trust? - Operation: How to identify possible failure? H K U S T 32

Visualization f for E Exploratory D Data A Analysis What does my dataset look like? It might be difficult to classify between (3,5) and (4,9)! Methods: • PCA • Multidimensional Scaling • t-SNE Augmenting: • Glyph (Smilkov et al. 2016) • Color (Wang and Ma 2013) MNIST using t-SNE. Maaten and Hinton 2008 MNIST. Smilkov et al. 2016 H K U S T 33