Learning Normalized Inputs for Iterative Estimation in Medical - PDF document

4/6/18 Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation Michal Drozdzal, Gabriel Chartrand, Eugene Vorontsov, Mahsa Shakeri, Lisa Di Jorio, An Tang, Adriana Romero, Yoshua Bengio, Chris Pal, Samuel Kadoury Medical imaging modalities - basics Magnetic Electron Computed Resonance Endoscopy Microscopy Tomography Imaging We will treat all data as 2D data. 2D 2D/3D 3D 3D 2D/3D Temporal No No Yes No dimension Signal Hounsfield Grayscale - RGB scale scale 1

4/6/18 CT Image segmentation Electron microscopy Endoscopy Medical imaging segmentation pipeline Pre-processing Model Post-processing 2

4/6/18 Medical imaging segmentation pipeline Pre-processing Model Post-processing Modality & model specific 3D for 2D model FP reduction What is used? Morphological operations 2D/3D CRF Medical imaging segmentation pipeline Pre-processing Model Post-processing Fully Convolutional Network Modality & model specific FCN8 3D for 2D model FP reduction What is used? UNET Morphological operations 2D/3D CRF Jon Long et. al. CVPR 2015 Olaf Ronneberger et. al. MICCAI 2015 3

4/6/18 Fully Convolutional Networks This can be ResNet, DenseNet, … This can be ResNet, DenseNet, … Down sampling Up sampling Skip - Pooling - Repeat + convolution - Concatenate - Strided convolutions - Transposed convolutions - Sum Medical imaging segmentation pipeline Pre-processing Model Post-processing Modality specific Fully Convolutional Network Modality & model specific (handcrafted) FCN8 3D for 2D model FP reduction What is used? UNET Morphological operations 2D/3D CRF Range normalization Value clipping Standardization Jon Long et. al. CVPR 2015 N4 (MRI) Olaf Ronneberger et. al. MICCAI 2015 4

4/6/18 Examples of segmentation pipelines Tools of medical imaging segmentation practitioner: Pre-processing Model Post-processing 2D Unet Range normalization 2D CRF 2D FCN8 Value clipping 3D CRF 2D FC-ResNet Standardization Morphological operations 3D Unet N4 (MRI) 3D FCN8 Histogram equalization 3D FC-ResNet Gaussian smoothing Examples of (hypothetical) segmentation pipelines: Lung segmentation in CT: Standardization + 2DUnet Lung segmentation in MRI: N4 + 2DUNet Liver segmentation in CT: Value clipping + 3DUnet + morphological operations Let’s design a model that can be trained with any imaging modality and does not require any pre-processing. 5

4/6/18 Let’s use ResNets Observations about ResNets: x x convolution Recent findings suggest that F() is a transformation close F(x) to identity [1 ] F(x) *N *N We found that FC-ResNets + + are more susceptible to data F(x) + x pre-processing than FCNs F(x) + convolution(x) x has high impact on feature ResNet uses initial convolution maps due to skip connections that can adapt input [1] Veit at al. Residual Networks are Exponential Ensembles of Relatively Shallow Networks What do we propose? EM EM Fully Fully Convolutional Convolutional Residual Network CT CT Network MRI MRI 6

4/6/18 Model Feature map [1xaxb] Fully Fully Input Segmentation map Convolutional Convolutional [1xaxb] Residual [1xaxb] Network Network +/ - 1 M parameters +/ - 11 M parameters Data Computed Tomography Magnetic Resonance Electron Microscopy Cell segmentation Lesion segmentation Prostate segmentation N= N= N= 30 training images 105 training volumes 50 training volumes 30 testing images 30 testing volumes 30 testing volumes 7

4/6/18 Experimental setup Data preparation: > No normalization! > Data augmentation Optimization: - Flips > RMSprop - Rotation > Weight decay - Shearing - Elastic transformations - Cropping EM data results (as of mid 2017) Comparison to published methods Qualitative results (test set) Pyramid-LSTM FC-ResNet optree-idsa SCI motif IDSIA Unet CUMedVision FusionNet IAL Ours 98.1 98 .8 .7 97 97 .3 .2 .2 97 97 97 .1 97 96.9 97 96.8 VRAND 8

4/6/18 CT data results (as of mid 2017) Qualitative results Comparison to standard FCNs (test set) FCN8 Unet FC-ResNet Ours 71.1 61.7 53.5 57 Image True segmentation Result DICE MRI data results (as of mid 2017) Comparison to published methods Qualitative results Situs Ours (test set) SRIBHME CAMP-TUM2 86.65 CUMED 83.02 82.39 79.92 74.17 2D FCN 3D FCN 9

4/6/18 Pre-processor effect EM EM Fully Fully Convolutional Convolutional Residual Network CT CT Network Normalized intensity histogram: EM EM Input intensity histogram: MRI MRI CT CT MRI MRI Can we quantify the pre-processing effect? Pre-processor quantification input data standardization pre-processor 5.71 d( , ) > d( , ) 3.89 3.57 d( , ) = d( , ) 3.38 3.35 2.99 2.98 2.87 2.48 Mean Jensen–Shannon distance on validation set d (0,0) d (1,0) d (2,0) d (1,0) d (1,1) d (2,1) d (2,0) d (2,1) d (2,2) EM CT MRI 10

4/6/18 Wrap up A low capacity FCN can serve as a learnable pre-processor. Combining learnable pre-processor with FC-ResNet yields very good results on a variety of image modalities. Single pipeline for all type of medical data! No need to handcraft data pre-processing. is hiring! Want to work at the confluence of academia and industry? MILA has open positions for: • Professors • Software engineers • Director of software • R&D & technology transfer • Linux sysadmins https:/ /tinyurl.com/mila-jobs 11

4/6/18 Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation Michal Drozdzal, Gabriel Chartrand, Eugene Vorontsov, Mahsa Shakeri, Lisa Di Jorio, An Tang, Adriana Romero, Yoshua Bengio, Chris Pal, Samuel Kadoury 12