Advisors: Robert Chang, Jeff Ullman, Andreas Paepcke *All - PowerPoint PPT Presentation

Apaar Sadhwani, Leo Tam, Jason Su Advisors: Robert Chang, Jeff Ullman, Andreas Paepcke *All contributors are/were affiliated with Stanford University at time of their contributions. Leo Tam now works at Nvidia. Address correspondence to Apaar Sadhwani at apaars@stanford.edu or Jason Su at sujason@stanford.edu.

 Motivation:  Affects ~100M, many in developed, ~45% of diabetics  Make process faster, assist ophthalmologist, self-help  Widespread disease, enable early diagnosis/care  Given fundus image  Rate severity of Diabetic Retinopathy  5 Classes: 0 (Normal), 1, 2, 3, 4 (Severe)  Hard classification (may solve as ordinal though)  Metric: quadratic weighted kappa, (pred – real) 2 penalty  Data from Kaggle  ~35,000 training images, ~54,000 test images  High resolution: variable, more than 2560 x 1920  Other unlabeled data from Stanford

 Motivation:  Affects ~100M, many in developed, ~45% of diabetics  Make process faster, assist ophthalmologist, self-help  Widespread disease, enable early diagnosis/care  Given fundus image  Rate severity of Diabetic Retinopathy  5 Classes: 0 (Normal), 1, 2, 3, 4 (Severe)  Hard classification (may solve as ordinal though)  Metric: quadratic weighted kappa, (pred – real) 2 penalty  Data from Kaggle  ~35,000 training images, ~54,000 test images  High resolution: variable, more than 2560 x 1920  Other unlabeled data from Stanford

Class 0 (normal) Class 4 (severe)

 Motivation:  Affects ~100M, many in developed, ~45% of diabetics  Make process faster, assist ophthalmologist, self-help  Widespread disease, enable early diagnosis/care  Given fundus image  Rate severity of Diabetic Retinopathy  5 Classes: 0 (Normal), 1, 2, 3, 4 (Severe)  Hard classification (may solve as ordinal though)  Metric: quadratic weighted kappa, (pred – real) 2 penalty  Data from Kaggle  ~35,000 training images, ~54,000 test images  High resolution: variable, more than 2560 x 1920  Other unlabeled data from Stanford

 High resolution images Image size Batch Size Atypical in vision, GPU batch size issues  224 x 224 128  Discriminative features small 2K x 2K 2  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling  Artifacts in ~40% images  Optimizing approach to QWK  Severe class imbalance class 0 dominates   Too few training examples

 High resolution images 0 1 Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data  2 3  Incorrect labeling  Artifacts in ~40% images  Optimizing approach to QWK  Severe class imbalance class 0 dominates   Too few training examples 4

 High resolution images Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling  Artifacts in ~40% images  Optimizing approach to QWK  Severe class imbalance class 0 dominates   Too few training examples Class 2

 High resolution images Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling  Artifacts in ~40% images  Optimizing approach to QWK  Severe class imbalance class 0 dominates   Too few training examples

 High resolution images Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling - Mentioned in problem statement  Artifacts in ~40% images - Confirmed with doctors  Optimizing approach to QWK  Severe class imbalance class 0 dominates   Too few training examples

 High resolution images Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling  Artifacts in ~40% images  Optimizing approach to QWK  Severe class imbalance class 0 dominates   Too few training examples

Penalty/Loss  High resolution images Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling Class 0 1 2 3 4  Artifacts in ~40% images Truth  Optimizing approach to QWK  Severe class imbalance - Hard classification non-differentiable class 0 dominates  - Backprop difficult  Too few training examples

Penalty/Loss  High resolution images Atypical in vision, GPU batch size issues   Discriminative features small Predict  Grading criteria: 1 not clear (EyePACS guidelines)  learn from data   Incorrect labeling Class 0 1 2 3 4  Artifacts in ~40% images Truth  Optimizing approach to QWK  Severe class imbalance - Hard classification non-differentiable class 0 dominates  - Backprop difficult  Too few training examples

Penalty/Loss  High resolution images Atypical in vision, GPU batch size issues   Discriminative features small Predict 2  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling Class 0 1 2 3 4  Artifacts in ~40% images Truth  Optimizing approach to QWK  Severe class imbalance - Hard classification non-differentiable class 0 dominates  - Backprop difficult  Too few training examples

Penalty/Loss Predict  High resolution images 3 Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling Class 0 1 2 3 4  Artifacts in ~40% images Truth  Optimizing approach to QWK  Severe class imbalance - Hard classification non-differentiable class 0 dominates  - Backprop difficult  Too few training examples

Penalty/Loss  High resolution images Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling Class 0 1 2 3 4  Artifacts in ~40% images Truth  Optimizing approach to QWK  Severe class imbalance - Hard classification non-differentiable class 0 dominates  - Backprop difficult  Too few training examples

Penalty/Loss  High resolution images Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling Class 2.5 0 1 2 3 4  Artifacts in ~40% images Truth  Optimizing approach to QWK  Severe class imbalance - Squared error approximation? class 0 dominates  - Differentiable  Too few training examples

 High resolution images Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling  Artifacts in ~40% images  Optimizing approach to QWK  Severe class imbalance class 0 dominates  - Naïve: 3 class problem, or all zeros!  Too few training examples - Learn all classes separately: 1 vs All? - Balanced while training - At test time?

 High resolution images Atypical in vision, GPU batch size issues   Discriminative features small  Grading criteria: not clear (EyePACS guidelines)  learn from data   Incorrect labeling  Artifacts in ~40% images  Optimizing approach to QWK  Severe class imbalance class 0 dominates  - Big learning models take more data!  Too few training examples - Harness test set?

 Literature survey:  Hand-designed features to pick each component  Clean images, small datasets  Optic disk, exudate segmentation: fail due to artifacts  SVM: poor performance

 Literature survey:  Hand-designed features to pick each component  Clean images, small datasets  Optic disk, exudate segmentation: fail due to artifacts  SVM: poor performance

1. Registration, Pre-processing 2. Convolutional Neural Nets (CNNs) 3. Hybrid Architecture

 Registration  Hough circles, remove outside portion  Downsize to common size (224 x 224, 1K x 1K)  Color correction  Normalization (mean, variance)

Class probabilities  Network in Network architecture AvgPool  7.5M parameters MaxPool (stride2)  No FC layers, spatial average pooling 3 Conv layers (depth 1024) instead  Transfer learning (ImageNet) MaxPool (stride2) 3 Conv layers  Variable learning rates (depth 384)  Low for “ ImageNet ” layers MaxPool (stride2) 3 Conv layers  Schedule (depth 256)  Combat lack of data, over-fitting MaxPool (stride2)  Dropout, Early stopping 3 Conv layers (depth 96)  Data augmentation (flips, rotation) Input Image