Dermoscopy Image? Catarina Barata and Carlos Santiago Computer and - PowerPoint PPT Presentation

How Important Is Each Dermoscopy Image? Catarina Barata and Carlos Santiago Computer and Robot Vision Lab

Motivation Dermoscopy Datasets 100000 10000 1000 100 2013 2014 2015 2016 2017 2018 2019 Computer and Robot Vision Lab 2

Motivation De Deep Ne Networks works Li Like ke Da Data ta Computer and Robot Vision Lab 3

Motivation Class Distribution Dermoscopy Datasets 200 150 MEL 100 NV 50 0 2013 2014 2015 2016 2017 2018 2019 Computer and Robot Vision Lab 4

Motivation Class Distribution Dermoscopy Datasets 1000 800 600 MEL 400 NV 200 0 2013 2014 2015 2016 2017 2018 2019 Computer and Robot Vision Lab 5

Motivation Class Distribution Dermoscopy Datasets 2500 2000 1500 BKL MEL 1000 NV 500 0 2013 2014 2015 2016 2017 2018 2019 Computer and Robot Vision Lab 6

Motivation Class Distribution Dermoscopy Datasets 10000 VASC 8000 DF BCC 6000 AKIEC 4000 BKL MEL 2000 NV 0 2013 2014 2015 2016 2017 2018 2019 Computer and Robot Vision Lab 7

Motivation Class Distribution Dermoscopy Datasets 20000 SCC VASC 15000 DF BCC 10000 AKIEC BKL 5000 MEL NV 0 2013 2014 2015 2016 2017 2018 2019 Computer and Robot Vision Lab 8

Motivation Why is this a problem? Network bias • Poor Generalization • Deep Net Class # Samples Recall (%) NV 6741 95 MEL 1119 66 BKL 1101 77 331 45 AKIEC BCC 517 88 DF 116 43 VASC 143 68 Computer and Robot Vision Lab 9

Me Li Likes kes Bala alanced nced Data ta More… Computer and Robot Vision Lab 10

Challenges Deal with class imbalance • Deep Net Class # Samples Recall (%) 6741 95 NV Not all classes are equally hard • MEL 1119 66 BKL 1101 77 Are all samples equally important? • AKIEC 331 45 517 88 BCC DF 116 43 VASC 143 68 Computer and Robot Vision Lab 11

Goal How to make the most of the available data? Data Augmentation • Importance Sampling • Sample Weighting • Computer and Robot Vision Lab 12

Goal How to make the most of the available data? Data Augmentation • Importance Sampling • Sample Weighting • Computer and Robot Vision Lab 13

Sample Weighting Batch Samples DNN Loss 𝑁 ℒ = 1 … … 𝑁 ෍ ℓ 𝑘 𝑘 Feedforward and Backpropagate and Compute Sample Update Model Loss ℓ 𝑘 Parameters Computer and Robot Vision Lab 14

Sample Weighting Batch Samples DNN Loss Cross Entropy Loss (CEL) Focal Loss (CEL) 𝑁 ℒ = 1 … … 𝑁 ෍ ℓ 𝑘 𝑘 Feedforward and Backpropagate and Compute Sample Update Model Loss ℓ 𝑘 Parameters Computer and Robot Vision Lab 15

Sample Weighting Batch Samples DNN Sample Weights Loss 𝒙 𝟑 𝒙 𝟓 𝒙 𝟐 𝒙 𝟒 𝑁 ℒ = 1 𝒙 𝟔 𝒙 𝟕 𝒙 𝟖 𝒙 𝟗 … … 𝑁 ෍ 𝒙 𝒌 ℓ 𝑘 𝑘 𝒙 𝟘 𝒙 𝟐𝟏 𝒙 𝟐𝟐 𝒙 𝟐𝟑 Feedforward and Backpropagate and 𝒙 𝟐𝟒 𝒙 𝟐𝟓 𝒙 𝟐𝟔 𝒙 𝟐𝟕 Compute Sample Update Model Loss ℓ 𝑘 Parameters Computer and Robot Vision Lab 16

Weighting Strategies • Class-Balanced Losses – Class-balanced (CB) [1] : 𝑘 = 𝑂 𝑥 𝑂 𝑧 𝑘 – Effective Number of Samples (ES) [2] : 1−𝛾 𝑂−1 𝑂𝑧𝑘 , 𝛾 = 𝑥 𝑘 = 𝑂 1−𝛾 [1] Provost, Machine Learning From Imbalanced Datasets 101, AAAI 2000 [2] Cui et al., Class-balanced Loss Based on Effective Number of Samples, CVPR 2019 Computer and Robot Vision Lab 17

Weighting Strategies • Class-Balanced Losses 0.08 0.08 0.51 1.08 – Class-balanced (CB): 0.50 0.50 0.08 0.08 𝑘 = 𝑂 𝑥 0.51 0.51 1.08 1.69 𝑂 𝑧 𝑘 0.08 0.50 3.90 4.81 – Effective Number of Samples (ES): 0.11 0.11 0.52 1.08 1−𝛾 𝑂−1 𝑂𝑧𝑘 , 𝛾 = 0.51 0.51 0.11 0.11 𝑥 𝑘 = 𝑂 1−𝛾 0.52 0.52 1.08 1.67 0.11 0.51 3.82 4.71 Computer and Robot Vision Lab 18

Weighting Strategies • Curriculum Learning 𝑁 1 arg min 𝑁 ෍ 𝑥 𝑘 ℓ 𝑘 + 𝐻 𝒙; 𝜇 𝒙 𝑘 Computer and Robot Vision Lab 19

Weighting Strategies • Curriculum Learning – Self-paced Learning (SPL) [1] : 𝐻 𝒙; 𝜇 = −𝜇 𝒙 1 – Online Hard Example Mining (OHEM) [2] : 𝐻 𝒙; 𝜇 = +𝜇 𝒙 1 [1] Kumar et al., Self-Paced Learning for Latent Variable Models, NeurIPS 2010 [2] Shrivastava et al., Training Region-based Object Detectors with Online Hard Example Mining, CVPR 2016 Computer and Robot Vision Lab 20

Weighting Strategies • Curriculum Learning 0 1 1 0 ℓ 𝑘 < 𝜇 1 1 0 1 – Self-paced Learning (SPL): 0 1 1 0 𝐻 𝒙; 𝜇 = −𝜇 𝒙 1 0 0 1 1 – Online Hard Example Mining (OHEM): 0 1 0 1 ℓ 𝑘 > 𝜇 0 0 1 0 𝐻 𝒙; 𝜇 = +𝜇 𝒙 1 1 0 0 1 1 0 0 1 Computer and Robot Vision Lab 21

Experimental Setup DNN Architectures • Flat Classifier (VGG-16) – + CBAM [2] Hierarchical Classifier [1] – Dataset • ISIC 2018 – Performance Metrics • Accuracy Recall – – Precision Balanced Accuracy – – F1-Score – [1] Barata et al., Explainable Skin Lesion Diagnosis Using Taxonomies, Pattern Recognition 2020 [2] Woo et al., CBAM: Convolutional Block Attention Module, ECCV 2018 Computer and Robot Vision Lab 22

Results Computer and Robot Vision Lab 23

Results Computer and Robot Vision Lab 24

Results Computer and Robot Vision Lab 25

Results Computer and Robot Vision Lab 26

Results Computer and Robot Vision Lab 27

Conclusions Weighting strategies significantly affect the performance of a DNN • Some weighting schemes may induce bias • Features learned by DNNs change according to the learning strategy • OHEM achieves the best overall performance • Computer and Robot Vision Lab 28

Thank You! ana.c.fidalgo.barata@tecnico.ulisboa.pt 29 Computer and Robot Vision Lab