Haptics for Surgical Simulation CPSC 599.86 / 601.86 Sonny Chan University of Calgary
A few announcements • CPSC Showcase / final project deliverables submit one-page project abstract by 9:00 AM , Tuesday April 10 - pick up Novint Falcon devices in MS145 / MS156 at 12:30 PM before showcase - return devices and ball grip at 4:00 PM , after the event - • End of semester logistics last class is Monday, April 9 - CPSC 599.86 final exam is Wednesday, April 25, 12:00 PM - CPSC 601.86 survey paper due Friday, April 20 -
Motivation • I taught you all this great, futuristic stuff this semester… - but where would I ever use it? • Few applications can support the high cost of haptic devices • Luckily for us (me), the medical community is well funded - surgery is a high-risk endeavour with very, very costly mistakes!
ACCEPTED NeuroTouch (NRC Canada) and LAP Mentor (Simbionix)
neuroArm project, University of Calgary
Opportunities for Surgical Simulation What one is the simulator?
My Research Mission Virtual Surgical Environment Operating Room
Volumetric Isosurfaces
Volume Rendering • Implicit representations are now uncommon, but... • 3D medical imaging (CT, MRI, etc.) has resulted in an abundance of volume data • Can be rendered with (almost) the same algorithm for implicit surfaces!
Sampled Volume Data • Image values sampled on rectilinear grid • CT scans measure radiodensity in Hounsfield Units
Interpolation 0.3 0.6 (1,1) What is the value here? F ( x, y ) = (1 � x + b x c )(1 � y + b y c ) I ( b x c , b y c ) + ( x � b x c )(1 � y + b y c ) I ( d x e , b y c ) + (1 � x + b x c )( y � b y c ) I ( b x c , d y e ) + ( x � b x c )( y � b y c ) I ( d x e , d y e ) for x, y, z 2 R (0,0) 0.7 0.9
Isocontours & Isosurfaces Choose a threshold value, T, to determine the surface function T = -600 HU T = 300 HU S ( x, y, z ) = T − F ( x, y, z )
Isosurfaces in 3D
Rendering Algorithm • Our implicit surface rendering algorithm has two specific requirements: - Inside-outside function for S(p) S ( x, y, z ) = T − F ( x, y, z ) - The gradient of S(p) r S ( x, y, z ) = ???
Gradient Estimation • Estimate gradient using central difference: 0 1 S ( x + δ ,y ) − S ( x − δ ,y ) ∂ S ! ∂ x 2 δ r S ( x, y ) = ⇡ @ A ∂ S S ( x,y + δ ) − S ( x,y − δ ) ∂ y 2 δ • What’s the best choice for the δ value? ∂ S/ ∂ x ∂ S/ ∂ y
Isosurface Rendering • The full rendering algorithm: - Detect initial contact when S(p) < 0 - Find surface point with initial seed - Update surface point as the device moves by using tangent plane - Contact breaks when device is moved outside the constraint plane - Repeat from start… • Any issues?
Demo?
Deformable Tissue Simulation
Continuum Mechanics • Governs the stress/strain behaviour of materials • Includes Young’s modulus (elasticity) and Poisson ratio (incompressibility) • Think of it as an “advanced” version of mass-spring systems [from continuummechanics.org]
Finite Element Simulation • Partitions an object into small, discrete elements • Computes continuum mechanics on solid element mesh 1 − ( ) R K R x x − 0 e e e f -f x f plastic plastic R total e ( ) x − x 0 f elastic 1 − R x x e 0 original state plastic rest state deformed state 1 − K ( R x x ) − 1 − 0 e e ( ) R x x − 0 e [from M. Müller & M. Gross, Proc. Graphics Interface, 2004]
Data Representation CT isosurface tetrahedral mesh
Data Processing
Deformation Simulation • Co-rotational linear elastic FEM • Computes nodal displacements from external forces • Implementation from VEGA FEM library (Jernej Barbi č , USC)
Haptic Rendering • Extension of original isosurface rendering algorithm: - sampling deformed volume - force transfer - moving the proxy
Sampling the Deformed Volume p 1 • How do we find the image intensity value p 0 x deformed at a given location in deformed space, x ? tetrahedron p 2 r 1 x m � − 1 � � � x m r 0 r 1 r 2 r 3 p 0 p 1 p 2 p 3 x r 0 r 2 = 1 1 1 1 1 1 1 1 1 1 rest vertices barycentric coordinates
Force Transfer • Virtual coupling force is applied to mesh nodes with barycentric weighting F 0 F 1 x p F 2 x F c
Moving the Proxy • To avoid interpenetration, we must keep the proxy outside the isosurface, even when the deformable mesh elements themselves move! • If a tetrahedron’s vertices move, p i → q i • we displace the proxy’s with the tetrahedron: � � 1 x 0 � � � q 0 q 1 q 2 q 3 p 0 p 1 p 2 p 3 x p p = 1 1 1 1 1 1 1 1 1 1
forces displacements Haptic Deformation Visual Architecture Rendering Simulation Rendering displacements 1000 Hz ~10-100 Hz 60 Hz
Results
Results
Summary • Surgical simulation is a popular and promising application of high-fidelity computer haptics - can tolerate cost of hardware • Sampled volumetric images (medical image data) can be described as implicit surfaces - adapt the implicit surface rendering algorithm to compute forces
Recommend
More recommend
