Simultaneous Localization and Mapping for Minimally Invasive Surgery - PowerPoint PPT Presentation

Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery Simultaneous Localization and Mapping for Minimally Invasive Surgery

Introduction � University of Bristol – using particle filters to track football players from a single static camera. � Imperial College - Simultanous Localization and Mapping (SLAM) with by Guang-Zhong Yang and Andrew Davison. � Siemens Corporate Research – Valve modelling Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Current Situation Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Why is Camera Motion Important Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Estimating Camera Motion & Building a 3D Map Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Problem: Noisy Sensor Measurements 1 mm = Many Pixels Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Noise in the Camera Image Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Noisy System How to we get accurate models and estimates from noisy measurements? Structure From Motion V Simultaneous Localization And Mapping (SLAM) Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Structure From Motion and Simultaneous Localization And Mapping Structure From Motion Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Structure From Motion and Simultaneous Localization And Mapping SLAM – Filtering Approach Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Simultaneous Localization And Mapping Extended Kalman Filter Augmented Reality Unscented Kalman Filter Robotic Navigation Particle Filter Wearable Computers Information Filter Autonomous Vehicles/ FastSLAM Helicopters / Planes / Cars RatSLAM RatSLAM / boats / submarines / boats / submarines Submapping Compute Games Hybrid Systems Games Controllers Dense Smart Phones Sparse Space Exploration Large Scale Surgery Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM [1] - State Extended Kalman Filter SLAM – Davison et al [1] MonoSLAM: Real-Time Single Camera SLAM Andrew Davison, Ian Reid, Nicholas Molton and Oliver Strasse. IEEE PAMI June 2007 (vol. 29 no. 6) pp. 1052-1067 Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM – Uncertainty Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM - Prediction Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM - Prediction Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM - Measurement Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM – Active Search Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM - Update Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM - Update Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM Adding new features Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

EKF SLAM Adding new features 1. Feature/Map initialisation 2. Predict motion 3. Feature matching 4. Update Pose and map [1] Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

SLAM for MIS Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery. Peter Mountney, Danail Stoyanov, Andrew Davison, Guang-Zhong Yang. MICCAI (1) 2006: pp. 347-354 Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

SLAM for MIS - Validation CT SLAM A Stereoscopic Fibroscope for Camera Motion and 3D Depth Recovery During Minimally Invasive Surgery. David Noonan, Peter Mountney, Daniel Elson, Ara Darzi and Guang-Zhong Yang. In proc ICRA 2009 , pp. 4463-4468 Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Dynamic View Expansion Dynamic View Expansion for Minimally Invasive Surgery using Simultaneous Localization And Mapping. Peter Mountney and Guang- Zhong Yang. In Proc EMBC 2009: pp. 1184-1187 Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Dense Stereo Real-time Stereo Reconstruction in Robotic Assisted Minimally Invasive Surgery. Danail Stoyanov, Marco Visentini Scarzanella, Philip Pratt and Guang-Zhong Yang. In Proc MICCAI 2010 Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

SLAM Dense Mapping Dense Surface Reconstruction for Enhanced Navigation in Minimally Invasive Surgery. Johannes Totz, Peter Mountney, Danail Stoyanov, and Guang-Zhong Yang. Accepted to MICCAI 2011 § § z § § Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Optical Biopsy Mapping Mauna Kea - www.maunakeatech.com/ Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Optical Biopsy Mapping Optical Biopsy Mapping for Minimally Invasive Cancer Screening. Peter Mountney, Stamatia Giannarou, Daniel Elson and Guang- Zhong Yang. In proc MICCAI(1), 2009, pp. 483-490 Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Optical Biopsy Mapping Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Motion Compensated SLAM in Dynamic Environments Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Motion Compensated SLAM in Dynamic Environments Motion Compensated SLAM for Image Guided Surgery. Peter Mountney and Guang-Zhong Yang. In Proc MICCAI 2010 Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Motion Compensated SLAM in Dynamic Environments Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Motion Compensated SLAM in Dynamic Environments -3 x 10 8 6 4 2 0 -2 -4 PCA -6 -8 -10 0 20 40 60 80 100 120 140 160 180 200 Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Motion Compensated SLAM in Dynamic Environments Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Motion Compensated SLAM in Dynamic Environments EKF state Respiration Model EKF Measurement Model EKF Prediction Model Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Motion Compensated SLAM in Dynamic Environments Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Motion Compensated SLAM in Dynamic Environments Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Motion Compensated SLAM in Dynamic Environments Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Future Work in this Area PTAM DTAM Non rigid structure from motion Incorporation of sensors Incorporation of sensors Incorporating pre operative models Parallel Approaches Biomechanical Models Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.

Questions Thank you for listening Peter Mountney – SLAM for MIS 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.