Haptics for Surgical Simulation CPSC 599.86 / 601.86 Sonny Chan - PowerPoint PPT Presentation

Haptics for Surgical Simulation CPSC 599.86 / 601.86 Sonny Chan University of Calgary

Motivation • I taught you all this great, futuristic stuff this semester… - but where would I ever use it? • Few applications can support the high cost of haptic devices • Luckily for us (me), the medical community is well funded - surgery is a high-risk endeavour with very, very costly mistakes!

D E T P E NeuroTouch (NRC Canada) and LAP Mentor (Simbionix) C C A

neuroArm project, University of Calgary

Opportunities for Surgical Simulation What one is the simulator?

My Research Mission Virtual Surgical Environment Operating Room

Volumetric Isosurfaces

Volume Rendering • Implicit representations are now uncommon, but... • 3D medical imaging (CT, MRI, etc.) has resulted in an abundance of volume data • Can be rendered with (almost) the same algorithm for implicit surfaces!

Sampled Volume Data • Image values sampled on rectilinear grid • CT scans measure radiodensity in Hounsfield Units

Interpolation 0.3 0.6 (1,1) What is the value here? F ( x, y ) = (1 � x + b x c )(1 � y + b y c ) I ( b x c , b y c ) + ( x � b x c )(1 � y + b y c ) I ( d x e , b y c ) + (1 � x + b x c )( y � b y c ) I ( b x c , d y e ) + ( x � b x c )( y � b y c ) I ( d x e , d y e ) for x, y, z 2 R (0,0) 0.7 0.9

Isocontours & Isosurfaces Choose a threshold value, T, to determine the surface function T = -600 HU T = 300 HU S ( x, y, z ) = T − F ( x, y, z )

Isosurfaces in 3D

Rendering Algorithm • Our implicit surface rendering algorithm has two specific requirements: - Inside-outside function for S(p) S ( x, y, z ) = T − F ( x, y, z ) - The gradient of S(p) r S ( x, y, z ) = ???

Gradient Estimation • Estimate gradient using central difference: 0 1 S ( x + δ ,y ) − S ( x − δ ,y ) ∂ S ! ∂ x 2 δ r S ( x, y ) = ⇡ @ A ∂ S S ( x,y + δ ) − S ( x,y − δ ) ∂ y 2 δ • What’s the best choice for the δ value? ∂ S/ ∂ x ∂ S/ ∂ y

Isosurface Rendering • The full rendering algorithm: - Detect initial contact when S(p) < 0 - Find surface point with initial seed - Update surface point as the device moves by using tangent plane - Contact breaks when device is moved outside the constraint plane - Repeat from start… • Any issues?

Demo?

Deformable Tissue Simulation

Continuum Mechanics • Governs the stress/strain behaviour of materials • Includes Young’s modulus (elasticity) and Poisson ratio (incompressibility) • Think of it as an “advanced” version of mass-spring systems [from continuummechanics.org]

Finite Element Simulation • Partitions an object into small, discrete elements • Computes continuum mechanics on solid element mesh 1 − ( ) R K R x x − 0 e e e f -f x f plastic plastic R total e ( ) x − x 0 f elastic 1 − R x x e 0 original state plastic rest state deformed state 1 − ( ) K R x x − 1 − 0 e e ( ) R x x − 0 e [from M. Müller & M. Gross, Proc. Graphics Interface, 2004]

Data Representation CT isosurface tetrahedral mesh

Data Processing

Deformation Simulation • Co-rotational linear elastic FEM • Computes nodal displacements from external forces • Implementation from VEGA FEM library (Jernej Barbi č , USC)

Haptic Rendering • Extension of original isosurface rendering algorithm: - sampling deformed volume - force transfer - moving the proxy

Sampling the Deformed Volume p 1 • How do we find the image intensity value p 0 x deformed at a given location in deformed space, x ? tetrahedron p 2 r 1 x m � − 1   �  �  � x m r 0 r 1 r 2 r 3 p 0 p 1 p 2 p 3 x r 2 r 0 = 1 1 1 1 1 1 1 1 1 1 rest vertices barycentric coordinates

Force Transfer • Virtual coupling force is applied to mesh nodes with barycentric weighting F 0 F 1 x p F 2 x F c

Moving the Proxy • To avoid interpenetration, we must keep the proxy outside the isosurface, even when the deformable mesh elements themselves move! • If a tetrahedron’s vertices move, p i → q i • we displace the proxy’s with the tetrahedron: � � 1   x 0 �  �  � q 0 q 1 q 2 q 3 p 0 p 1 p 2 p 3 x p p = 1 1 1 1 1 1 1 1 1 1

forces displacements Haptic Deformation Visual Architecture Rendering Simulation Rendering displacements 1000 Hz ~10-100 Hz 60 Hz

Results

Results

Summary • Surgical simulation is a popular and promising application of high-fidelity computer haptics - can tolerate cost of hardware • Sampled volumetric images (medical image data) can be described as implicit surfaces - adapt the implicit surface rendering algorithm to compute forces