Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: A - PowerPoint PPT Presentation

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: A Learning-based Method Presented at the Haptics Symposium, Reno, NV, USA, 2008. Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Allison M. Okamura Engineering Research Center for Computer-Integrated Surgical Systems Technology (ERC-CISST), Laboratory for Computational Science and Robotics (LCSR), The Johns Hopkins University Thursday, 13 March 2008 Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Deformation Modeling Simbionix. LAP Mentor Product Brochure Available via web, http://www.simbionix.com/LAP Mentor.html S. Misra, K. T. Ramesh, and A. M. Okamura. Modeling of tool-tissue interactions for computer-based surgical simulation: a literature review . Accepted to Presence: Teleoperators and Virtual Environments , 2008. Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Finite Element Models Idea Model tissue as a set of elements with PDEs defining boundary conditions. Solve matrix equations based on continuum mechanics Characteristics Accurate Slow Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Mass-Spring-Damper Meshes Idea Model tissue as point masses connected by springs and dampers. Dynamics solved in closed form using Hooke’s law. Characteristics Limited to linear deformations No volume conservation Fast Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Our Approach S. Misra, K. T. Ramesh, and A. M. Okamura. Modeling of tool-tissue interactions for computer-based surgical simulation: a literature review . Accepted to Presence: Teleoperators and Virtual Environments , 2008. Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Finite Element Modeling Started with directly-measured porcine brain tissue parameters K. Miller, K. Chinzei, G. Orssengo, and P. Bednarz. Mechanical properties of brain tissue in-vivo: experiment and computer simulation . Journal of Biomechanics , 33(11):1369 1376, 2000. Used ABAQUS with several different loading conditions Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Comparison Model Van Gelder/Mollemans Heuristic k c = E 2 P e area ( T e ) 1 m i = � 4 ρ j V j | c | 2 j A. Van Gelder. Approximate simulation of elastic membranes by triangulated spring meshes . Journal of Graphics Tools , 3(2):2141, 1998. W. Mollemans, F. Schutyser, J. Cleynenbreugel, and P. Suetens. Tetrahedral mass spring model for fast soft tissue deformation . In International Symposium on Surgery Simulation and Soft Tissue Modeling , pages 145–154, 2003. Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Parameter Learning Our approach Optimized all spring stiffnesses for each loading condition Used Simultaneous Perturbation Stochastic Approximation (SPSA) Previous work Simulated Annealing – Deussen et al. (1995), Morris (2006) Genetic Algorithms – Bianchi (2003), (2004) Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion SPSA Pseudocode Variables for k=1:n Perturbation vector δ δ = 2 round ( rand ( p , 1)) − 1 θ ± Positive & negative samples θ ± = θ ± c k δ Loss function at samples y ± y ± = Loss ( θ ± ) ˆ g Gradient estimate g = y + − y − ˆ Perturbation step-size 2 c k δ c k θ = θ + a k ˆ g a k Update step-size end Adapted from Spall, J.C. An Overview of the Simultaneous Perturbation Method for Efficient Optimization , Johns Hopkins APL Technical Digest , vol. 19, pp. 482–492, 1998. Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Results Deformation of a sample node for Sample learning curve all 3 models -3 x 10 10 9 FEM Linear 8 SPSA norm 7 2 6 � || opt 5 f - act 4 f || 3 2 Iterations�( ) k 1 1 1.5 2 2.5 Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Implementation System Phantom Omni SenseAble OpenHaptics Toolkit Boost matrix/vector libraries Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Implementation Method Notation and Data Structures x t Mesh node positions B Damping M Mesh node masses f t Contact force K Spring elasticities Time-step τ Dynamics x t = M − 1 ( K t ∆ x t + B ˙ ¨ x t + f t ) x t = ˙ ˙ x t − 1 + 0 . 5 τ (¨ x t − 1 + ¨ x t ) x t = x t − 1 + 0 . 5 τ (˙ x t − 1 + ˙ x t ) K t = γ K ( f t ) + (1 − γ ) K t − 1 Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Implemented Display Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Conclusions Contributions Method to learn spring stiffness for high-fidelity modeling Enabled fast piece-wise linear approximation of nonlinear deformations Future Work Extension to 3D Learning more mesh parameters Online mesh structure redefinition Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method

Introduction Our Approach FE Modeling Parameter Learning Implementation Conclusion Thanks Thanks for listening. Thanks to NIH Grant R01 EB002004 for continued support. Questions? Zachary Pezzementi, Daniel Ursu, Sarthak Misra, Modeling Realistic Tool-Tissue Interactions with Haptic Feedback: Allison M. Okamura A Learning-based Method