from snakes from snakes to organisms to organisms Ghassan Hamarneh - PowerPoint PPT Presentation

Multimedia and Mathematics 2005 Deformable Models for Biomedical Image Analysis from ‘snakes’ from ‘snakes’ to ‘organisms to ‘organisms’ Ghassan Hamarneh Simon Fraser University

Talk Overview • Multimedia patient records • Medical images • Medical images analysis • Image segmentation and registration • Deformable models: Snakes • Controlling shape deformation • Deformable organisms 2 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Multimedia Patient Record speech audio natural language graphical objects alphanumeric images video bio-signals 3 27Jul’05 B A N F F

Medical Images Need to: Store, Communicate, Visualize, Process, Analyze MR, CT, SPECT, MRA, EIT, MRE, hist., optical, PET, DTMRI, fMRI… Philips Medical University of Bergen - Norway University College London Mayo Clinic Visible Human BrighamRAD www.Brain-Spect.com 4 27Jul’05 Koizumi, Hitachi Chudler, U of Washington Scientific Computing & Imaging, Utah Center for Neural Science at NYU B A N F F

Medical Image Analysis • Medical Images: 2D/3D+Time, scalar/vector/tensor fields, non rigid tissue, patient info • Manual Analysis: Tedious. Time consuming. Inter-, intra-operator variability • General goals: Automation. Quantification. Classification. Data reduction. Visualization • General Methodologies: Image restoration. Image enhancement. Visualization techniques, image segmentation. Image registration. Shape analysis • Mathematics: Inverse problems, PDEs, transforms, optimization, statistics,… • Numerous Applications… Computer-aided diagnosis. Computer assisted intervention. Image guided therapy, therapy evaluation. Surgical simulation, planning, and navigation. Image data fusion. Quantitative & time series analysis. Statistical Structural Shape Analysis Anatomical atlases. Virtual, augmented reality. Instrument, patient localization, tracking. Medical tele-presence and tele-surgery. Functional brain mapping. Screening and functional genomics. 5 hamarneh@cs.sfu.ca 27Jul’05 http://www.uib.no/med/avd/miapr/arvid/matematisk98/index.htm http://www.cs.sfu.ca/~hamarneh B A N F F

Image Segmentation • Partition an image into regions • Assign labels to pixels (binary/fuzzy) • Obtain higher-level representation University Medical Center Utrecht 6 27Jul’05 http://www-dsv.cea.fr Voxel-Man 3D Navigator B A N F F

Thresholding and Clustering pixel Feature 3 classifier background Count o b j e c t Feature 2 Pixel intensity Feature 1 7 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Region-based Methods Growing Courtesy: Tina Kapur Splitting …Merging 8 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Edge Detection and Linking… “Livewire” Laplacian zero-crossing, canny-edge, gradient magnitude and direction 2D � 3D 9 27Jul’05 B A N F F

Image Registration and Atlas-based Segmentation • Registration : Find optimal spatial transformation (warping) of one image to maximize “similarity” to another image • Segmentation via registration register reference image new image warp labels using same transform labelled labelled reference (atlas) image 10 27Jul’05 B A N F F

Registration and deformation analysis Extension and Flexion 11 27Jul’05 B A N F F

Deformable Models • Contours, surfaces, volumes • Originally: 2D semi-automatic tools • Initialized in the image space • Integrate boundary elements, robust to image noise, boundary gaps • Deform according to image data • Implemented on the continuum • … & “shape” constraints achieving sub-pixel accuracy Shape representation Initialization Model deformation 12 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Classical “snakes” s=0 s=1 y Snake or Active Contour Models: v(x(s),y(s)) Deformable contours, initialized in s=0.2 the image, deform according to internal and external constraints x tensile flexural external inflation µ �� + γ � + α + β = + v v F F F F i i i i i i 13 27Jul’05 B A N F F

Deformable Surfaces 2 d V dV + δ = α + β m i i F F int ext 2 dt dt 14 27Jul’05 B A N F F

Some DM Extensions Dimensionality : 2D, 2D+T,3D,3D+T, color, stereo… Framework : Bayesian, wavefront propagation, geodesic computation Shape representations : Wavelets, Splines, Fourier descriptors,... Energy/forces : inflation, distance transform, texture/appearance… Optimization : GA, SA, ANN, DP,… Topological changes: T-snakes, Level-sets 15 McInerney, Ryerson Sethian, Berkley hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

DM Problems • Leaking from weak edges • Low level parameter selection problematic • Lack of high level control (rely on human guidance, user interaction) • Modest prior shape knowledge (amorphous shapes, smoothness constraints) • Sensitivity to initialization 16 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

CC Segmentation for MS 407 images 17 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Global Shape Statistics x   1   y  1  x   2   y   2 alignment   �   x   n     y n − < < 2 k 2 ORL database Training set Labeling corresponding PCA landmarks, aligning shapes Prior shape knowledge: + k λ i Point Distribution Model = + Pb S S Allowable Shape Domain k >> 1 PDM utilized in segmentation: − k λ i 18 Active Shape Models hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

PDM/ASM Segmentation Un-allowable shape Allowable shape domain http://www2.imm.dtu.dk/~aam/datasets/ 4th mode 1 st (too much) mode 19 27Jul’05 B A N F F

“Fourier Snakes” x   1   mean mean y 1 st PC 1 st PC mean mean  1  1 st PC 1 st PC 2 nd PC 2 nd PC x   2 nd PC 2 nd PC 2 PC1 ±1std PC1 ±1std   y   2  �  PC2 ±1std PC2 ±1std   x   n     y n projection allowable spatial on subspace DCT coeff 1 shape domain representaion y 1 PC 1 DCT PC 2 frequency domain DCT coeff N x 1 y n DCT coeff 2 20 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Deformable Organisms Deformable Organisms • Controlling deformation – User interaction – Global to local deformations – Global shape statistics – Setting low-level parameters – New cost/force terms • Utilize high-level knowledge to guide model-fitting • Difficult to encode knowledge in low-level terms • Require intuitive, controlled shape deformation handles • Artificial-Life framework that complements – bottom-up data-driven functionality of deformable models with… – top-down knowledge-driven model-fitting strategies 21 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Deformable Organisms Memory and prior knowledge cognitive Perception Plan or schedule center Interaction with other organisms Cognition perceptual attention muscle actuation mechanism causing shape deformation Behaviours Physics Skeleton Geometry sensors muscles and limbs underlying medial-based shape representation 22 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Controlling Shape Deformation Medial-Based Shape Profiles Geometry Length, Orientation, Left and right thickness 23 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Controlling Shape Deformation Physics Medial-Based Shape Profiles Type: bending stretching thickening HR-PCA     ∑∑ ∑ = + + α  p p M w k      d d dls dls dlst dlst   l s t location scale variation mode operator type amplitude 24 27Jul’05 B A N F F

Controlling Shape Deformation Physics Physics-Based Shape Deformations • Mass-spring model • User interaction • Intuitive deformations • Feasible shapes 25 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Controlling Shape Deformation Deformations: Physics Translation, rotation, scaling, Bending, bulging, tapering,… External forces �� �� D D θ θ s s n n Spring actuation ij ij i i ( ( )( ) ( ) θ d 2 d d old = − − − + r 1 1 K 1 1 r n n ) j j ij ij π R C C R R Statistics-based deformation HR-PCA = + r r M w def loc scl , , def loc scl , , def loc scl , , def loc scl , , 26 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

Controlling Shape Deformation 27 27Jul’05 B A N F F

Controlling Shape Deformation, 3D • In-sheet and out-of-sheet bending values • Upper and lower thickness values • Elongation values 28 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F

3D Deformations Apply operators � Intuitive, controlled shape deformation 29 27Jul’05 B A N F F

Bicipital Groove 30 27Jul’05 B A N F F

Procedural Plan and Behavioral Routines Behaviours 31 hamarneh@cs.sfu.ca 27Jul’05 http://www.cs.sfu.ca/~hamarneh B A N F F