Fusing Non Fusing Non- -Volumetric, Spatially Volumetric, - PowerPoint PPT Presentation

Fusing Non Fusing Non- -Volumetric, Spatially Volumetric, Spatially- - Localized Data with Localized Data with V l V l Volumetric Data Volumetric Data i D i D Robert Weersink, Harsimran Braisch, Greg Bootsma David Jaffray Greg Bootsma, David Jaffray

Comparison of Endoscopy & CT Comparison of Endoscopy & CT Comparison of Endoscopy & CT Comparison of Endoscopy & CT • CT CT CT CT o High Resolution Volumetric High Resolution Volumetric o Anatomic Anatomic Anatomic Anatomic o Post Post- -imaging Analysis imaging Analysis

Comparison of Endoscopy & CT Comparison of Endoscopy & CT Comparison of Endoscopy & CT Comparison of Endoscopy & CT • CT CT CT CT o High Resolution Volumetric High Resolution Volumetric o Anatomic Anatomic Anatomic Anatomic o Post Post- -imaging Analysis imaging Analysis • Optical/endoscopic O O Optical/endoscopic i i l/ l/ d d i i o Topography, Texture, Topography, Texture, o Colour/Contrast Colour/Contrast o Real Real- -time time

Advantages of Combining Advantages of Combining • CT CT & & Optical/endoscopic Optical/endoscopic imaging provide imaging provide complimentary data complimentary data p p y y • Future combined CT/Optical Probes will require “co Future combined CT/Optical Probes will require “co- - visualization” visualization” � Puts molecular info into context of anatomical & functional information Puts molecular info into context of anatomical & functional information

App 1: Head & Neck RT Planning App 1: Head & Neck RT Planning App 1: Head & Neck RT Planning App 1: Head & Neck RT Planning Diagnosis & margins more Diagnosis & margins more Treatment Planning & delivery Treatment Planning & delivery clearly identified and defined in clearly identified and defined in l l l id l id ifi d ifi d d d fi d d fi d i d i based on 3D Data based on 3D Data b b d d 3D D 3D D 2D Optical Domain 2D Optical Domain Glottic Glottic Data Fusion 2D 2D to 3D 3D Floor of Mouth

App 2: Surgical Guidance App 2: Surgical Guidance App 2: Surgical Guidance App 2: Surgical Guidance Laparoscopic surgery requires Laparoscopic surgery requires Diagnosis & margins defined in Diagnosis & margins defined in g g g g accurate registration of scope accurate registration of scope i i i i f f 3D CT Domain 3D CT Domain (& image) to CT (& image) to CT Data Fusion 3D to 2D 3D to 2D

Technology Challenges Technology Challenges Technology Challenges Technology Challenges R Registration i t ti F Fusion i C Co-visualization i li ti

Electro Magnetic Tracking: Electro Magnetic Tracking: While using a magnetic field generator, location of an object can be found using sensor coils that induce a voltage while in the presence of a varying magnetic field. magnetic field. NDI Aurora System Northern Digital. 2008

Registration Registration Registration Registration • Combining Multiple Coordinate Systems Combining Multiple Coordinate Systems CT CS Tip of Endoscope CAMERA CS SENSOR SENSOR WORLD CS WORLD CS CS

Integration & Registration Simultaneous Video Coronal View Rendered View with Axial View Tracking Tip

Integration & Registration Integration & Registration

Spatial Accuracy Tests Spatial Accuracy Tests • Template used to position the 6D sensor at known positions. (every 50 X 50 X 100 mm) iti ( 50 X 50 X 100 ) • Repeated measurements to Assess Precision, R Reproducibility, Accuracy d ibilit A x • • Precision: • Precision: • Precision: ±0 04 mm Precision: ±0.04 mm 0 04 mm 0.04 mm Positioning • Reproducibility: Reproducibility: ± ±0.09 mm 0.09 mm Generator Plate y y z

Accuracy Accuracy Transformed Measurements Original Measurements Systemic position error Systemic position error y y p p 200 200 corrected by applying a corrected by applying a 150 150 polynomial transformation polynomial transformation 100 100 to measured data to measured data to measured data to measured data m) ition (mm 50 0 50 0 0 Position Error Position Error -50 -50 Y Posi Original Original ± ± 8.8 mm 8.8 mm -100 -100 Transformed Transformed ± 3.6 mm ± 3.6 mm -150 -150 -200 -200 -250 -250 0 100 200 300 400 500 0 100 200 300 400 500 Z Position (mm)

Camera Calibration Camera Calibration Camera Calibration Camera Calibration • For focus aberations etc For focus aberations etc For focus, aberations, etc. For focus, aberations, etc. • Necessary for comparison with 3D data Necessary for comparison with 3D data Method: C pt r Capture a series ri of Test Images as Known Targets; Known Targets; Calculate optical parameters

Camera Calibration Camera Calibration • Necessary for Focus and Distortion correction based on • x d = (1 + k a r 2 + k b r 4 )x n ; r 2 = x 2 + y 2 ( b ) n ; y d a Original images: Corners identified, arrows showing distortion Y O O O O Y Y O Y X X X O X Corrected images

Future (Long Term) Steps . . . Future (Long Term) Steps . . . Future (Long Term) Steps . . . Future (Long Term) Steps . . . • Addition of Targeted Probes Addition of Targeted Probes o Combined Fluorescence & CT contrast Combined Fluorescence & CT contrast • Creation of 3D images from multiple 2D images Creation of 3D images from multiple 2D images o Surface contouring from scope images overlayed on to CT Surface contouring from scope images overlayed on to CT • Integrating other data: i.e. database of other clinically Integrating other data: i.e. database of other clinically relevant data relevant data l l t d t t d t o Biopsies, microscope imaging, etc. Biopsies, microscope imaging, etc. • Clinical Testing • Clinical Testing Clinical Testing Clinical Testing o RT planning for H&N; Endoluminal; Lung Surgical RT planning for H&N; Endoluminal; Lung Surgical Guidance Guidance

Tissue Fluorescence Tissue Fluorescence Tissue Fluorescence Tissue Fluorescence • Current: Autofluorescence Current: Autofluorescence • • Future: Fluorescent Future: Fluorescent for disease detection for disease detection molecular contrast agents molecular contrast agents Prescan Prescan Targeting HDL nanocarrier to folate receptors: G. Zhang

Multiple 2D Images to 3D Multiple 2D Images to 3D Round surfaces of tissue may make this difficult Round surfaces of tissue may make this difficult Srikumar Ramalingam Chunxiao Zhou Department of Computer Science Department of Computer Engineering University of California, Santa Cruz

Steps to Clinical Studies Steps to Clinical Studies Steps to Clinical Studies Steps to Clinical Studies • Head & Neck: Head & Neck: Head & Neck: Head & Neck: o Surgery (J. Irish) Surgery (J. Irish) o Radiation Treatment planning (Andrew Hope) Radiation Treatment planning (Andrew Hope) Radiation Treatment planning (Andrew Hope) Radiation Treatment planning (Andrew Hope) • Lung: Lung: o Surgery (A. Pierre) Surgery (A. Pierre) • Evaluation with clinicians on best visualization Evaluation with clinicians on best visualization methods methods o Likely will be different for each indication Likely will be different for each indication

Acknowledgments Acknowledgments Acknowledgments Acknowledgments • Ralph DaCosta Greg Bootsma Harsimran Ralph DaCosta Greg Bootsma Harsimran Ralph DaCosta, Greg Bootsma, Harsimran Ralph DaCosta, Greg Bootsma, Harsimran Braisch, Jeff Siewerdsen, David Jaffray Braisch, Jeff Siewerdsen, David Jaffray • Ontario Institute of Cancer Research • Ontario Institute of Cancer Research Ontario Institute of Cancer Research Ontario Institute of Cancer Research – 1 mm 1 mm 1 mm 1 mm Challenge Challenge • Guided Therapeutics Program G id d Th Guided Therapeutics Program G id d Th i i P P GT X Create Innovate Translate Evaluate