Detecting skin cancer with an iPhone Tory Jarmain CEO & - PowerPoint PPT Presentation


 Detecting skin cancer with an iPhone Tory Jarmain CEO & Co-Founder

Skin disorders are prevalent in 43% of outpatient visits Source: “Why do patients visit their doctors? Assessing the most prevalent conditions in a defined US population.“ Stauver, et al.

PREVALENCE DISEASE GROUP 0-18 19-29 30-49 50-64 65+ All Skin disorders 33% 38% 41% 50% 66% 43% Arthritis and joint disorders 14% 25% 35% 50% 63% 34% Back problems 6% 20% 29% 34% 44% 24% Lipid metabolism disorders 0% 3% 19% 49% 70% 22% Upper respiratory disease 24% 19% 22% 22% 23% 22% Source: “Why do patients visit their doctors? Assessing the most prevalent conditions in a defined US population.“ Stauver, et al.

1 in 5 
 Americans will develop skin cancer in their lifetime Source: American Academy of Dermatology

1 in 3 
 cancer diagnoses is skin cancer Source: American Academy of Dermatology

Dermatologists have 52% accuracy classifying skin lesions with non-dermoscopic imagery Source: “Deep Networks for Early Stage Skin Disease and Skin Cancer Classification.” Esteva, et al.

MELANOMA BENIGN

Source: American Academy of Dermatology

The average wait time to see a dermatologist is 1 month in the U.S. and 3-6 months in much of the world. In that time, skin disorders can become life threatening. Source: Merritt Hawkins

OUR IDEA Use deep learning to triage 
 skin disorder cases

SOLUTION Snap a photo, detect a skin disorder and see visually similar cases

D Snap a photo View results Learn more

MELANOMA DETECTION

SCOPE Assist physicians in binary classification of skin lesions

GOALS Achieve or improve upon state-of-the-art results for skin lesion segmentation and classification. Measure the impact of segmentation on the accuracy of the classifier.

HYPOTHESIS Segmenting skin lesions improves the accuracy and sensitivity of a deep learning classification model

CHALLENGES Dermoscopic images may contain artifacts, be low contrast, and contain multiple lesions

CONVOLUTIONAL NEURAL NETWORKS

CONVOLUTION LAYER Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

CONVOLUTION LAYER Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

CONVOLUTION LAYER Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

ACTIVATION LAYER Source: “Skin Lesion Detection From Dermoscopic Images Using Convolutional Neural Networks”

POOLING LAYER Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

FULLY CONNECTED LAYER Source: “Deep Learning for Computer Vision.” Karpathy, Andrej.

MAIN SCHEME Source: LeCun, et al.

MAIN SCHEME Source: LeCun, et al.

MAIN SCHEME Source: LeCun, et al.

MAIN SCHEME Source: LeCun, et al.

Class Benign Malignant Total 727 173 900 Training subset Test subset 304 75 379

DATA AUGMENTATION Original image Random transformations

METHOD SCHEME

SEGMENTATION Original skin lesion Binary mask Source: “Convolutional Networks for Biomedical Image Segmentation.” Olaf Ronneberger, et al.

CLASSIFICATION WITH VGG-16 • Five convolutional blocks • 3 x 3 receptive field • ReLU as Activation Function • Max-Pooling • Classifier block • 3 fully-connected layers at the top of the network

TRANSFER LEARNING Pretrain a ConvNet on a very large dataset (e.g. ImageNet, which contains 1.2 million images with 1000 categories), and then use the ConvNet either as an initialization or a fixed feature extractor for the task of interest.

Train Freeze this these

EVALUATION

SEGMENTATION EVALUATION Ground truth Mask obtained Jaccard 
 Dice 
 Rank Participant Accuracy Sensitivity Specificity Index Coe ffi cient 1 1 Adrià Romero Lopez 0.918 0.869 0.918 0.930 0.954 1 Urko Sanchez 0.843 0.910 0.953 0.910 0.965 2 Lequan Yu 0.829 0.897 0.949 0.911 0.957 3 Mahmudur Rahman 0.822 0.895 0.952 0.880 0.969 1 MIDDLE Group

CLASSIFICATION EVALUATION

CLASSIFICATION EVALUATION Model Accuracy Loss Sensitivity Precision Unaltered lesion classifier 0.847 0.472 0.824 0.952 Perfectly segmented 
 0.840 0.496 0.865 0.962 lesion classifier Automatically segmented 0.817 0.514 0.892 0.968 lesion classifier

CLASSIFICATION EVALUATION Model Accuracy Loss Sensitivity Precision Unaltered lesion classifier 0.847 0.472 0.824 0.952 Perfectly segmented 
 0.840 0.496 0.865 0.962 lesion classifier Automatically segmented 0.817 0.514 0.892 0.968 lesion classifier With segmentation • Accuracy decreases • Loss increases

CLASSIFICATION EVALUATION Model Accuracy Loss Sensitivity Precision Unaltered lesion classifier 0.847 0.472 0.824 0.952 Perfectly segmented 
 0.840 0.496 0.865 0.962 lesion classifier Automatically segmented 0.817 0.514 0.892 0.968 lesion classifier With segmentation • Sensitivity increases • Precision increases

SENSITIVITY The most important metric in medical settings. 
 By missing a true melanoma case (False Negative) the model would fail in early diagnosis. It is better to raise a False Positive than to create a False Negative.

SENSITIVITY number of true positives Sensitivity = number of true positives + number of false negatives

CLASSIFICATION EVALUATION Model Accuracy Loss Sensitivity Precision Unaltered lesion classifier 0.847 0.472 0.824 0.952 Perfectly segmented 
 0.840 0.496 0.865 0.962 lesion classifier Automatically segmented 0.817 0.514 0.892 0.968 lesion classifier The automatically segmented classifier performs best

CONFUSION MATRICES Unaltered Perfectly Segmented Automatically Classifier Classifier Segmented Classifier False Negatives descending

23-WAY CLASSIFICATION

• Acne and rosacea • Nail diseases • Malignant lesions • Contact dermatitis • Atopic dermatitis • Psoriasis & lichen planus • Bullous disease • Infestations & bites • Bacterial infections • Benign tumors • Eczema • Systemic disease 23,000 images • Exanthems & drug eruptions • Fungal infections • Hair diseases • Urticaria 600 diseases • STDs • Vascular tumors • Pigmentation disorders • Vasculitis • Connective tissue diseases • Viral infections • Melanoma, nevi & moles Sources: “A Deep Learning Approach to Universal Skin Disease Classification.” Liao, Haofu. “Deep Networks for Early Stage Skin Disease and Skin Cancer Classification.“ Esteva, et al.

Deep Residual Networks

Source: “Deep Residual Networks: Deep Learning Gets Way Deeper.” He, Kaiming.

Source: “Deep Residual Networks: Deep Learning Gets Way Deeper.” He, Kaiming.

Source: “Deep Residual Networks: Deep Learning Gets Way Deeper.” He, Kaiming.

RESIDUAL LEARNING Source: “Deep Residual Learning for Image Recognition.” Kaiming, et al.

23-WAY CLASSIFICATION RESULTS

Accuracy Best in paper Triage “A Deep Learning Approach to Universal Skin Disease Top-1 73.1% 76.1% Classification” Liao, Haofu. Top-5 91.0% 92.4% Accuracy Best in paper Triage “Deep Networks for Early Stage Skin Disease and Skin Top-1 60.0% 64.8% Cancer Classification” Esteva, et al. Top-5 80.3% 80.5%

FUTURE WORK

THANK YOU

tory@triage.com