Deformable Organ Contour Transfer with Deep Inverse Shape Encoding - PowerPoint PPT Presentation

Deformable Organ Contour Transfer with Deep Inverse Shape Encoding (DISE) Networks for Auto-segmentation in Low Contrast Regions Tiancheng Liu 1 Xiaobai Sun 1 Fang-fang Yin 2 Lei Ren 2 1 Department of Computer Science, Duke University, USA 2 Department of Radiation Oncology, Duke University School of Medicine, USA 60 th AAPM Annual Meeting, Nashville, TN Aug 2nd, 2018 1 / 15

Outline ⋄ Purpose: Robust automatic segmentation of thoracic and abdominal CT images with low CNR ⋄ Method: Deep Inverse Shape Encoding (DISE) networks ◦ Inverse Shapes for coarse regional partition ◦ Sparse registration via shape matching ◦ Reference guided labeling ⋄ Results and Evaluation 2 / 15

Introduction: Automatic segmentation Existing automatic methods: based on dense voxel-wise registration using ⋄ patch texture 1 ⋄ gradients 2 Reference Image Target Image Shortcomings: ⋄ Time consuming ⋄ Not robust to low CNR images Generated labels 3 Provided labels 1 Korfiatis et al. IEEE Trans Inf Technol Biomed, 2010 2 Sotiras et al. IEEE TMI, 2013 3 Liu et al. SU-K-201-14 (Snap Oral), AAPM 2017 3 / 15

Purpose Axial Coronal Sagittal ⋄ Robustly, automatically segment CT (CBCT) images of thoracic, abdominal regions with low CNR ⋄ To replace manual delineation and segmentation based on dense registration 4 / 15

Method: deep inverse shape encoding (DISE) network 5 / 15

Inverse shape: basic concept with 2D illustration Gradient : Inverse Shape : ∇ I ( x 0 ) ≈ I ( x 0 +∆ x ) invShape ( I 0 , ∆ I ) = { x | I ( x ) ∈ [ I 0 , I 0 + ∆ I ] } ∆ x local in spatial support non-local in spatial support non-robust to noisy change in intensity robust in shape to noisy change in intensity 6 / 15

Illustration: inverse shape on XCAT phantom Five inverse shapes 1 are shown by their boundaries, each in a unique color 1 XCAT Phantom data from K¨ onik et al. Phys. Med. Biol. 2014 7 / 15

Illustration: sparse samples on a pair of inverse shapes 300 300 250 250 200 200 150 150 100 100 50 50 250 250 200 200 300 300 250 250 150 200 150 200 150 150 100 100 Reference Target Sparse samples on inverse shape containing liver, spleen, diaphragm and aorta 5064 samples (from 1.34M voxels) on reference, 5059 samples (from 1.33M voxels) on target 8 / 15

Shape encoding via shape context descriptor 1 (2D) 2 2 4 4 ) bin index ) bin index 6 6 Angle( Angle( 8 8 10 10 12 12 2 4 2 4 Radii( ) bin index Radii( ) bin index Reference Target ◦ Log-polar histogram of samples on the shape (equally-spaced bins along circumference, larger radii bins at coarser scale) ◦ Capture topology at multiple scales 1 Belongie et al. IEEE PAMI 2002 9 / 15

Inverse shape encoding via shape context descriptors (3D) Geometric depiction data structure for shape descriptor 3D Shape context descriptor in log-spherical histograms 10 / 15

DISE network for inverse shape matching 11 / 15

Illustration: sparse registration Reference Target Correspondence between shape descriptors = ⇒ matching between inverse shapes 12 / 15

Results and Evaluation: liver contour transferring ⋄ XCAT Phantom 1 ⋄ Fast, robust segmentation by DISE network ⋄ DSC on Liver: 0.98 DSC = 2 | V gen ∩ V gt | Ground Truth liver contour on target image | V gen | + | V gt | ⋄ Robust to noise Generated liver contour on target image 1 K¨ onik et al. Phys. Med. Biol. 2014 13 / 15

Results and Evaluation: liver contour transferring ⋄ XCAT Phantom 1 ⋄ Fast, robust segmentation by DISE network ⋄ DSC on Liver: 0.98 Liver contour on reference image DSC = 2 | V gen ∩ V gt | | V gen | + | V gt | ⋄ Robust to noise Generated liver contour on target image 1 K¨ onik et al. Phys. Med. Biol. 2014 13 / 15

Summary of DISE network ⋄ Achieve both robustness and efficiency for anatomical segmentation ◦ Coarse partition of image domain into inverse shapes induced by intensity bins ◦ Sparse representation of the inverse shape via sparse sampling and shape context descriptor ◦ Contour transfer Shape matching via DISE network ◦ Can faciliate finer registration and other analysis tasks 14 / 15

Thank you! Tiancheng Liu – tcliu@cs.duke.edu 15 / 15

References i X. Bai, S. Bai, Z. Zhu, and L. J. Latecki. 3D shape matching via two layer coding. IEEE transactions on pattern analysis and machine intelligence , 37(12):2361–2373, 2015. S. Belongie, J. Malik, and J. Puzicha. Shape matching and object recognition using shape contexts. IEEE transactions on pattern analysis and machine intelligence , 24(4):509–522, 2002.

References ii A. M. Bronstein, M. M. Bronstein, and R. Kimmel. Rock, paper, and scissors: extrinsic vs. intrinsic similarity of non-rigid shapes. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on , pages 1–6. IEEE, 2007. D. Ji, J. Kwon, M. McFarland, and S. Savarese. Deep view morphing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 2155–2163, 2017.

References iii A. K¨ onik, C. M. Connolly, K. L. Johnson, P. Dasari, P. W. Segars, P. H. Pretorius, C. Lindsay, J. Dey, and M. A. King. Digital anthropomorphic phantoms of non-rigid human respiratory and voluntary body motion for investigating motion correction in emission imaging. Physics in Medicine & Biology , 59(14):3669, 2014. T. Liu, D. Floros, N. Pitsianis, X. Sun, F.-f. Yin, and L. Ren. Robust automatic co-segmentation of multiple medical images. SU-K-201-14, presented at AAPM 2017.

References iv O. Nomir and M. Abdel-Mottaleb. Hierarchical contour matching for dental X-ray radiographs. Pattern Recognition , 41(1):130–138, 2008. H. Tabia, H. Laga, D. Picard, and P.-H. Gosselin. Covariance descriptors for 3D shape matching and retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4185–4192, 2014. Y. Zhang, F.-F. Yin, W. P. Segars, and L. Ren. A technique for estimating 4D-CBCT using prior knowledge and limited-angle projections. Medical physics , 40(12), 2013.