Counterfactual Visual Explanations Yash Goyal Ziyan Wu Jan Ernst - PowerPoint PPT Presentation

Counterfactual Visual Explanations Yash Goyal Ziyan Wu Jan Ernst Dhruv Batra Devi Parikh Stefan Lee (Georgia Tech) (Siemens) (Siemens) (Georgia Tech) (Georgia Tech) (Georgia Tech)

Counterfactual Visual Explanations A Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull Bird Classification Deep Network An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe ? 2

Counterfactual Visual Explanations A Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull Bird Classification Deep Network An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe ? For input X, why did the model predict Y instead of Z? 3

Counterfactual Visual Explanations A Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull Bird Classification Deep Network An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe ? For input X, why did the model predict Y instead of Z? Explanation through Counterfactual: If X was X *, then the outcome would have been Z rather than Y . 4

Counterfactual Visual Explanations A Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull Bird Classification Deep Network An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe ? Explanation through Counterfactual: What would have to change in image A to make the model predict Horned Grebe ? 5

Counterfactual Visual Explanations A Eared Grebe Horned Grebe Western Grebe Pied-Bill Grebe Herring Gull Bird Classification Deep Network An XAI Question: Why did the model predict Eared Grebe instead of Horned Grebe ? B Explanation through Counterfactual: What would have to change in image A to make the model predict Horned Grebe ? An image where the network predicts Horned Grebe. 6

Counterfactual Visual Explanations What would have to change in image A to make the model predict Horned Grebe ? A B An image where the network predicts Horned Grebe. 7

Counterfactual Visual Explanations What would have to change in image A to make the model predict Horned Grebe ? If looked more like A B An image where the network predicts Horned Grebe. 8

Counterfactual Visual Explanations What would have to change in image A to make the model predict Horned Grebe ? If looked more like A B If looked more like An image where the network predicts Horned Grebe. 9

Counterfactual Visual Explanations What would have to change in image A to make the model predict Horned Grebe ? If looked more like A B If looked more like An image where the network predicts Horned Grebe. How can we identify these region pairs important to the model? 10

Counterfactual Visual Explanations How can we identify these region pairs important to the model? Bird Classification Deep Network 11

Counterfactual Visual Explanations How can we identify these region pairs important to the model? Bird Classification Deep Network 𝑒 Spatial Features ℎ𝑥 ℎ … 𝑥 𝑒 𝑔(𝐽) Spatial Feature Extractor 12

Counterfactual Visual Explanations How can we identify these region pairs important to the model? Bird Classification Deep Network 𝑒 Spatial Features ℎ𝑥 log 𝑄 𝑧 . 𝐽) ℎ log 𝑄 𝑧 / 𝐽) ⋮ … ⋮ 𝑥 log 𝑄(𝑧 1 |𝐽) 𝑒 𝑔(𝐽) 𝑕(𝑔 𝐽 ) Spatial Feature Extractor Decision Network 13

Counterfactual Visual Explanations A B 𝑔(𝐽 3 ) 𝑔(𝐽 4 ) Bird Classification Deep Network … … Query Image Distractor Image Features Features 14

Counterfactual Visual Explanations A B * dramatization – permutation is in feature space. 𝑔(𝐽 3 ) 𝑄 𝑔(𝐽 4 ) Bird Classification … Deep Network … … … … … … … … … … … … … Query Image Permutation Distractor Image Features Matrix Features 15

Counterfactual Visual Explanations A B * dramatization – permutation is in feature space. 1 − 𝑏 𝑔(𝐽 3 ) 𝑏 𝑄 𝑔(𝐽 4 ) Bird Classification … Deep Network … … … … … … … … … … … … … … … Inverted Query Image Gating Permutation Distractor Image Gating Vector Features Vector Matrix Features 16

Counterfactual Visual Explanations A B * dramatization – permutation is in feature space. 𝑔(𝐽 ∗ ) 1 − 𝑏 𝑔(𝐽 3 ) 𝑏 𝑄 𝑔(𝐽 4 ) Bird Classification … Deep Network … … … … … … … … … … … … … … … … Inverted Query Image Gating Permutation Distractor Image Counterfactual Edit Gating Vector Features Vector Matrix Features Image Features 17

Counterfactual Visual Explanations A B * dramatization – permutation is in feature space. * dramatization – replacement is in feature space. 𝑔(𝐽 ∗ ) 1 − 𝑏 𝑔(𝐽 3 ) 𝑏 𝑄 𝑔(𝐽 4 ) Bird Classification … Deep Network … … … … … … … … … … … … … … … … Inverted Query Image Gating Permutation Distractor Image Counterfactual Edit Gating Vector Features Vector Matrix Features Image Features 18

Counterfactual Visual Explanations A B * dramatization – permutation is in feature space. * dramatization – replacement is in feature space. Counterfactual Visual Explanation Generation: Find 1) binary gating vector 𝒃, and 2) a permutation matrix 𝑄 𝑔(𝐽 ∗ ) 1 − 𝑏 𝑔(𝐽 3 ) 𝑏 𝑄 𝑔(𝐽 4 ) Bird Classification … Deep Network … … … … … … … … … … … … … … … … Inverted Query Image Gating Permutation Distractor Image Counterfactual Edit Gating Vector Features Vector Matrix Features Image Features 19

Counterfactual Visual Explanations A B * dramatization – permutation is in feature space. * dramatization – replacement is in feature space. Counterfactual Visual Explanation Generation: Find 1) binary gating vector 𝒃, and 2) a permutation matrix 𝑄 𝑔(𝐽 ∗ ) 1 − 𝑏 𝑔(𝐽 3 ) 𝑏 𝑄 𝑔(𝐽 4 ) Bird Classification … such that the model changes its decision to the distractor class Deep Network … … … … … … … … … … … … … … … … Inverted Query Image Gating Permutation Distractor Image Counterfactual Edit Gating Vector Features Vector Matrix Features Image Features 20

Counterfactual Visual Explanations A B * dramatization – permutation is in feature space. * dramatization – replacement is in feature space. Counterfactual Visual Explanation Generation: Find 1) binary gating vector 𝒃, and 2) a permutation matrix 𝑄 𝑔(𝐽 ∗ ) 1 − 𝑏 𝑔(𝐽 3 ) 𝑏 𝑄 𝑔(𝐽 4 ) Bird Classification … such that the model changes its decision to the distractor class Deep Network … … with the fewest edits (i.e. 𝒏𝒋𝒐 𝒃 𝟐 ) … … … … … … … … … … … … … … Inverted Query Image Gating Permutation Distractor Image Counterfactual Edit Gating Vector Features Vector Matrix Features Image Features 21

Results – Single Edit If the highlighted region in image A looked like the highlighted region in image B , then image A is more likely to be classified as class B . Composite Image Query Image A Distractor Image B (for visualization only) Eared Grebe Horned Grebe 22

Results – Single Edit If the highlighted region in image A looked like the highlighted region in image B , then image A is more likely to be classified as class B . Composite Image Query Image A Distractor Image B (for visualization only) Eared Grebe Horned Grebe 23

Results – Single Edit Query Distractor Composite Image Image A Image B (for visualization only) 24

Results – Single Edit Query Distractor Composite Image Image A Image B (for visualization only) 25

Results – Single Edit Query Distractor Composite Image Image A Image B (for visualization only) Eared Grebe Horned Grebe Olive sided Flycatcher Myrtle Warbler 26 Blue Grosbeak Indigo Bunting

Machine Teaching – Bird Classification Do our counterfactual explanations help untrained participants learn to identify fine-grained classes? 27

Machine Teaching – Bird Classification Do our counterfactual explanations help untrained participants learn to identify fine-grained classes? Training Feedback: Sorry, its not a Bravo. It is actually an Alpha. 28

Machine Teaching – Bird Classification Do our counterfactual explanations help untrained participants learn to identify fine-grained classes? Testing Training Feedback: Sorry, its not a Bravo. It is actually an Alpha. 29

Machine Teaching – Bird Classification Do our counterfactual explanations help untrained participants learn to identify fine-grained classes? 85 78.77 80 74.29 75 71.09 70 65 60 55 50 IF + Feature Attribution IF + Counterfactual Instant Feedback (IF) Explanation (GradCAM) Explanation 30

Counterfactual Visual Explanations Composite Image Query Image A Distractor Image B (for visualization only) Eared Grebe Horned Grebe Questions? Stop by our poster at #149 in Pacific Ballroom!