Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery Registration in Computer-Assisted Laparoscopic Surgery Lena Maier-Hein, PhD Division of Medical and Biological Informatics (MBI) German Cancer Research Center (DKFZ)
Laparoscopic surgery • Access to the abdomen through small incisions • Instruments/endoscope are inserted through trocars • Many advantages, but: 2D screen vs. 3D world o Small field-of-view o Reduced mobility o No tactile feedback o requires a lot of skill and experience (Source: D. Stoyanov, PhD thesis 2005 ) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Computer-assisted laparoscopic surgery • Goal: Visualize hidden structures via Augmented Reality (AR) • Requires: Registration of pre-operative data with intra-operative data (Source: M. Baumhauer, IntJ CARS 2008 ) (Source: M. Sugimoto, J Hepatobiliary PancreatSci 2010 ) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Computer-assisted surgery registration registration Planning data Intra-operative patient data Tracking data Modality for Intra-operative Tracking system surgical planning imaging modalities (i.e. optical) (i.e. CT device) (i.e. endoscope) Pre-operative Intra-operative Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Outline • Part I: Basics • Part II: Global Surface Registration • Part III: Fine Surface Registration • Part IV: Registration in practice - state-of-the-art approaches Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Outline • Part I: Basics • Part II: Global Surface Registration • Part III: Fine Surface Registration • Part IV: Registration in practice - state-of-the-art approaches Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Basics – Input shapes • Geometry representation Points o Oriented points (i.e., points with normals) o Surfaces o Skeletons o … o • Surface representation implicit (e.g. level set) o Explicit (e.g. triangle meshes) o Triangle mesh and oriented points (blue) (Source: [Johnson1997]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Basics – Surface descriptor • Shape descriptor : Scalar value or vector that describes a given shape (e.g. moments) • Local surface descriptor : Captures some property of the shape around the neighbourhood of a point (e.g. curvature) Desirable properties: o o Robust o Rotation invariant o Translation invariant o Local o Efficient o …. Requires: Local coordinate system o (Source: [Gelfand2005]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Many types of surface descriptors available • Curvature • Spin images • Integral variants • Shape context • Multi-scale features • Curvature maps • Spherical harmonics and wavelets • Salient geometric features • Part-aware metric • Heat Kernel Signature • Shot • … (cf. [vanKaick2011] for references) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Curvature (on a curve) • Curvature κ in a point on a curve r (Source: Wikipedia) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Principal curvature (on a surface) (Source: Wikipedia) Normal plane : Plane containing the normal vector Normal curvature : Curvature of the curve obtained by intersecting the surface with a normal plane Principal curvature: Maximum ( κ 1 ) and minimum ( κ 2 ) values of the normal curvature at a point. Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Mean/Gaussian curvature (Source: Wikipedia) Mean curvature: Gaussian curvature : Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Many types of surface descriptors available • Curvature (Example of low-dimensional descriptor) • Spin images (Example of high-dimensional descriptor) • Integral variants • Shape context • Multi-scale features • Curvature maps • Spherical harmonics and wavelets • Salient geometric features • Part-aware metric • Heat Kernel Signature • Shot • … Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Spin Images (Source: [Johnson1997]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Spin Images: Coordinate System • Cylinder coordinates α: Distance to axis defined by normal vector n o β : Distance to tangent plane o (Source: [Johnson1997]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Spin Images: Illustration (Source: [Johnson1997]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Spin images: Examples (Source: [Johnson1997]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Spin images: Algorithm • Define size and bin size of spin image • For each vertex v : For each vertex in the support volume of v : o o Compute coordinates ( α , β ) o Increment corresponding bins using bilinear interpolation (Source: [Johnson1997]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Spin images: Image Size • Different image sizes (and thus support volumes ) yield different spin images (Source: [Johnson1997]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Spin images: Bin size • Bin size: Width of bins • Should be set depending on mesh resolution • Larger bin sizes imply: decreasing influence of individual vertices o a greater memory requirement o (Source: [Johnson1997]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Descriptors – Spin images: Support Angle • Support angle θ : n A :normal vector of point of interest o Only consider those points in the support volumes with normal vector n B that fulfill o the condition: The angle enclosed by n B and n A is smaller than or equal to θ o • Useful for range image registration due to occlusion (Source: [Johnson1997]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Many types of surface descriptors available • Curvature • Spin images • Integral variants • Shape context • Multi-scale features • Curvature maps • Spherical harmonics and wavelets • Salient geometric features • Part-aware metric (Source: [Gelfand2005]) • Heat Kernel Signature • Shot • … Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Outline • Part I: Basics • Part II: Global Surface Registration • Part III: Fine Surface Registration • Part IV: Registration in practice - state-of-the-art approaches Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Surface Matching: Challenges Non-rigid surface registration Partial surface registration (Image: Stefanie Speidel et al. , Karlsruhe Institute ofTechnology (KIT)) (Image: Stefanie Speidel et al. , KIT) Noise Real-time registration … Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Outline • Part I: Basics • Part II: Global Surface Registration Global analysis based registration o Local analysis based registration o • Part III: Fine Surface Registration • Part IV: Registration in practice - state-of-the-art approaches Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Surface matching – global analysis • Compute shape properties and use them for an initial alignment • Example: Principal Component Analysis (PCA) based alignment Compute and align centroids and principal axes of both shapes o Not suitable for partial surface matching o • Global surface properties are often used for shape retrieval Example using Manifold Harmonics Transform [Reuter2006] o (Source: [Reuter2006]) Lena Maier-Hein – Registration 22 nd September ,Toronto, Canada Tutorial on 3D Surface Reconstruction in Laparoscopic Surgery.
Recommend
More recommend
