Augmented Reality for Port Placement and Navigation in Robotically Assisted Minimally Invasive Cardiovascular Surgery Marco Feuerstein Fakultät für Informatik, Technische Universität München Klinik für Herz- und Gefäßchirurgie, Deutsches Herzzentrum München feuerste@cs.tum.edu June 26, 2004
Acknowledgements • Co-Authors: – Jörg Traub – Martin Bauer – Eva Schirmbeck – Hesam Najafi – Robert Bauernschmitt – Gudrun Klinker • A.R.T. GmbH – Marc Schneberger • SFB 453 Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 2/12
Problem Statement • Surgeon on a master console controls a slave teleoperator • Where to place the ports for the teleoperator arms? • Where are the teleoperator arms inside the patient’s body? Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 3/12
Workflow Perception Port Planning Validation/Simulation Patient Registration Augmented Reality Analysis Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 4/12
Perception • Acquisition of cross-sectional image slices (CT) • Image segmentation • Reconstruction of a polygonal triangle mesh, which is used to perform the planning Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 5/12
Planning the Teleoperator Arms • Definition of a region of interest by landmarks • Placing of landmarks for the ports of the endoscope and instrument arms • Visualization of corresponding teleoperator arms in a collision-free way • Possibility to interactively adjust all landmarks Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 6/12
Validation of Planned Data • Simulation of the view of the endoscope arm’s camera by a virtual endoscope view • Collision detection between teleoperator arms and critical organs or bones Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 7/12
Port Placement Movie Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 8/12
Patient Registration • Estimation of a rigid transformation from the virtual model to the real patient • Point based registration to match fiducials • Subsampling to find the correspondences Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 9/12
Augmented Reality • Intra-operative port placement by overlaying virtual planned teleoperator arms • Endosopic view additionally showing the real teleoperator arm position Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 10/12
“Global Positioning System” Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 11/12
Thank you for your attention! Augmented Reality for Port Placement and Navigation in RMICS 26-Jun-2004 Marco Feuerstein 12/12
More recommend