High-Resolution Breast Cancer Screening with Multi-View Deep - PowerPoint PPT Presentation

High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks Krzysztof J. Geras Joint work with Kyunghyun Cho , Linda Moy , Gene Kim , Stacey Wolfson and Artie Shen . GTC 2017

W HERE DEEP LEARNING IS USEFUL amount of data available difficulty of the task

W HERE DEEP LEARNING IS USEFUL natural image recognition speech machine recognition translation game playing amount of data available the learning tasks for which deep learning makes a difference difficulty of the task

Can we save the world?

W HERE DEEP LEARNING IS USEFUL natural image recognition speech machine recognition translation game playing amount of data available medical image analysis? the learning tasks for which deep learning makes a difference difficulty of the task

B REAST CANCER SCREENING

B REAST CANCER SCREENING About 40 million exams performed yearly in the US.

B REAST CANCER SCREENING About 40 million exams performed yearly in the US. About 250 thousand women are diagnosed with cancer.

B REAST CANCER SCREENING About 40 million exams performed yearly in the US. About 250 thousand women are diagnosed with cancer. About 40 thousand die.

B REAST CANCER SCREENING R-MLO L-MLO R-CC L-CC (left cranial caudal) (right cranial caudal) (left mediolateral oblique) (right mediolateral oblique)

B REAST CANCER SCREENING We try to mimic predictions of a radiologist. ◮ Class 0: incomplete ( ≈ 15%). ◮ Class 1: negative ( ≈ 50%). ◮ Class 2: bening findings ( ≈ 35%).

B REAST CANCER SCREENING We try to mimic predictions of a radiologist. ◮ Class 0: incomplete ( ≈ 15%). ◮ Class 1: negative ( ≈ 50%). ◮ Class 2: bening findings ( ≈ 35%). Radiologists call these classes BI-RADS (short for Breast Imaging-Reporting and Data System).

C HALLENGES (1)

C HALLENGES (1) You need a lot of data to do deep learning.

C HALLENGES (1) You need a lot of data to do deep learning. Publicly available data sets contain about 1k images.

C HALLENGES (1) You need a lot of data to do deep learning. Publicly available data sets contain about 1k images. We build our own data set: ◮ 23k exams, ◮ 103k images. ◮ Each image is at least 2600 × 2000 pixels.

C HALLENGES (2)

C HALLENGES (2)

C HALLENGES (2)

C HALLENGES (2)

C HALLENGES (2)

C HALLENGES (2)

C HALLENGES (2)

C HALLENGES (2) High resolution necessary - computational and engineering challenge.

C HALLENGES (3) Multi-view data. How to integrate information?

O UR MODEL Classifier p ( y | x ) Concatenation (256 × 4 dim) DCN DCN DCN DCN L-CC R-CC L-MLO R-MLO

O UR MODEL layer kernel size stride #maps repetition global average pooling 256 convolution 3 × 3 1 × 1 256 × 3 max pooling 2 × 2 2 × 2 128 Classifier p ( y | x ) convolution 3 × 3 1 × 1 128 × 3 Concatenation (256 × 4 dim) max pooling 2 × 2 2 × 2 128 convolution 3 × 3 1 × 1 128 × 3 DCN DCN DCN DCN max pooling 2 × 2 2 × 2 64 L-CC R-CC L-MLO R-MLO 3 × 3 1 × 1 × 2 convolution 64 convolution 3 × 3 2 × 2 64 max pooling 3 × 3 3 × 3 32 3 × 3 2 × 2 convolution 32 input 1

R ESULTS 1.0 0.8 AUC true positive rate 0.6 0 vs. others: 0.609 1 vs. others: 0.717 0.4 2 vs. others: 0.728 BI-RADS 0 0.2 Average: 0.685 BI-RADS 1 BI-RADS 2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 false positive rate

C ONFIDENT TEST DATA We can compute the entropy of predictions, � p ( y ′ | x ) log p ( y ′ | x ) , H ( y | x ) = − y ′ ∈C and sort examples according to it.

C ONFIDENT TEST DATA We can compute the entropy of predictions, � p ( y ′ | x ) log p ( y ′ | x ) , H ( y | x ) = − y ′ ∈C and sort examples according to it. We will consider the 30% with the lowest entropy to be “confident”.

R ESULTS FOR CONFIDENT TEST DATA 1.0 0.8 AUC true positive rate 0.6 0 vs. others: 0.636 1 vs. others: 0.816 0.4 2 vs. others: 0.844 BI-RADS 0 0.2 Average: 0.765 BI-RADS 1 BI-RADS 2 0.0 0.0 0.2 0.4 0.6 0.8 1.0 false positive rate

I MPACT OF DOWNSCALING 0.80 0.75 0.70 AUC 0.65 0.60 average AUC average AUC (confident) 0.55 1/8 1/4 1/2 1 resolution fraction

I MPACT OF THE DATA SET SIZE 0.80 0.75 0.70 AUC 0.65 0.60 0.55 average AUC average AUC (confident) 0.50 1/10 1/5 1/2 1 data set size fraction

V ISUALISATION � � ∂ H ( y | x ) � � We visualise � , � � ∂ x v � � ( i , j ) � � p ( y ′ | x ) log p ( y ′ | x ) . where H ( y | x ) = − y ′ ∈C

V ISUALISATION

V ISUALISATION

C ONCLUSIONS ◮ We made a first step in the direction of end-to-end breast cancer screening with neural networks.

C ONCLUSIONS ◮ We made a first step in the direction of end-to-end breast cancer screening with neural networks. ◮ It is much harder to learn the “incomplete” (0) class than other classes.

C ONCLUSIONS ◮ We made a first step in the direction of end-to-end breast cancer screening with neural networks. ◮ It is much harder to learn the “incomplete” (0) class than other classes. ◮ We need to use the full resolution. A lot more effort is necessary to develop archiectures appropriate for data of large dimensionality.

C ONCLUSIONS ◮ We made a first step in the direction of end-to-end breast cancer screening with neural networks. ◮ It is much harder to learn the “incomplete” (0) class than other classes. ◮ We need to use the full resolution. A lot more effort is necessary to develop archiectures appropriate for data of large dimensionality. ◮ We need more data.

C ONCLUSIONS ◮ We made a first step in the direction of end-to-end breast cancer screening with neural networks. ◮ It is much harder to learn the “incomplete” (0) class than other classes. ◮ We need to use the full resolution. A lot more effort is necessary to develop archiectures appropriate for data of large dimensionality. ◮ We need more data. (We are currently processing a 10 times bigger data set).

Thank you! High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks K. J. Geras, S. Wolfson, S. G. Kim, L. Moy, K. Cho arXiv:1703.07047