Structured Decision Forests For Multi-modal Ultrasound Image - PowerPoint PPT Presentation

Structured Decision Forests For Multi-modal Ultrasound Image Registration Ozan Oktay 1 , Andreas Schuh 1 , Martin Rajchl 1 , Kevin Keraudren 1 , Alberto Gomez 3 , Mattias P. Heinrich 2 , Graeme Penney 3 , Daniel Rueckert 1 1 Department of Computing, Imperial College London, UK 2 Institute of Medical Informatics, University of Lübeck, Germany 3 Division of Imaging Sciences and Biomedical Engineering, King’s College London, UK 18 th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015)

18 th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015) 2 Image Guided Cardiac Interventions Pre-Operative Stage CT and Intra-Operative Image MR Image Acquisitions Guidance with TOE Spatial Alignment of Pre- and Ultrasound Images Intra-Operative Images Spatial Alignment of Pre- and Intra- (for Better Image Guidance) Operative Images

18 th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015) 3 Probabilistic Edge Maps (PEMs) Input Images PEMs Input Images PEMs SSC [1] and GM [2] SSC-US GM-MRI SSC-MRI GM: Intensity Gradient Magnitude – SSC: Self-Similarity Context Descriptor 1. Heinrich et al.: “Towards real-time multimodal fusion for image guided interventions using self-similarities.” MICCAI’13 2. Wein et al.: “Global registration of US to MRI using the LC2 metric for enabling neurosurgical guidance.” MICCAI’13

18 th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015) 4 Advantages of Probabilistic Edge Maps A. Modality independent (e.g. CT, MRI, US) A. Computationally efficient ( 20s per image ) A. Target organ specific image registration B. Accurate and smooth anatomical representation A. Same training and testing configuration is applied to all three modalities. B. It does not require image segmentation.

18 th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015) 5 Structured Decision Forest (SDF) Structured Decision Tree PEM Representation Input Image ψ 1 , θ 1 M a L 1 ψ 2 , θ 2 Input Space: x i ∈ X L 2 L 3 Image features • Each voxel is voted for N t x (M e ) 3 Output Space: y i ∈ Y • N t is the number of trees. M e • All the votes are aggregated by averaging. Output edge patch labels 3. Dollar et al.: “Structured forests for fast edge detection.” ICCV 2013 4. Kontschieder et al.: “Structured class-labels in random forests for semantic image labeling.” ICCV 2011

18 th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015) 6 The Proposed Multi-Modal Registration Framework Global alignment B-spline FFD Input cardiac Initial Alignment PEM with robust block based non-rigid images of the images representation matching [2] registration [1] PEM-CT PEM-US Computation Time ~20s per image ~21s per image ~73s per image (Quad-core 3.0GHz) 5. Rueckert et al.: “Non-rigid registration using free-form deformations: Application to breast MR images.” TMI’99 6. Ourselin et al.: “Reconstructing a 3D structure from serial histological sections.” Image and Vision Computing ’01

18 th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015) 7 US/CT and US/MR Image Alignment Results Subject 1 Subject 1 Subject 3 Subject 3 Subject 2 Subject 2 Subject 4 Subject 4

18 th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015) 8 Structured Regression Forest (SRF) CT image & PEM contours The detected landmark points are used to initialize the multi- ψ 1 , θ 1 modal image registration. L 1 ψ 2 , θ 2 Regression for Septal-wall : Leaf Node Γ 3 , θ 3 L 4 : Classification Node ( y 1 , d n 1 ) 1 , Λ n : Regression Node MR image & PEM contours L 2 L 3 ( y 4 , d n 4 ) 4 , Λ n Regression for Mid-ventricle ( y 2 , d n 2 ) ( y 3 , d n 2 , Λ n 3 , Λ n 3 ) 7. Gall J., et al. ”Class-Specific Hough Forests for Object Detection.” CVPR 2009. 8. Criminisi A., et al. “Regression Forests for Efficient Anatomy Detection and Localization in CT Studies.” MCV 2010.

18 th International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2015) 9 Structured Decision Forests For Multi-modal Ultrasound Image Registration Original Fast x4 § Acknowledgements: • Source code will be available at http://www.doc.ic.ac.uk/~oo2113/