Shape Statistics for Image Segmentation with Prior Guillaume - PowerPoint PPT Presentation

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Shape Statistics for Image Segmentation with Prior Guillaume Charpiat PhD Defense - 2006, December 13 PhD Supervisor: Olivier Faugeras Odyss´ ee Team - ENS INRIA ENPC Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Introduction Image Segmentation ◮ Find a contour in a given image ◮ The best curve for a given segmentation criterion ◮ Criterion based on color homogeneity, texture, edge detectors, etc. − → Image Segmentation Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Introduction Image Segmentation ◮ Find the best contour for a given criterion Variational Method ◮ Energy E to minimize with respect to a curve C ◮ Compute the derivative of the energy ◮ Gradient descent: ∂ t C = −∇ E ( C ) ◮ Initialization → local minimum ◮ Other methods: graph cuts (suitable for few energies) Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Introduction Image Segmentation ◮ Find the best contour for a given criterion Variational Method ◮ Minimize criterion by gradient descent with respect to the contour ◮ Most criteria: no shape information Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Introduction Image Segmentation ◮ Find the best contour for a given criterion Variational Method ◮ Minimize criterion by gradient descent with respect to the contour Shape Statistics ◮ Sample set of contours from already segmented images ◮ Shape variability ? ◮ Shape prior ? Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Introduction Shapes and Shape Metrics Set of Shapes Topological equivalence Variational Shape Warping Gradient Descent Generalized Gradients Approximation of the Hausdorff distance Statistics Mean and Modes of Variation Examples Images Segmentation with prior Shape prior (shape probability) Starfish example Boletus example Summary Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Set of Shapes I - Shapes and Shape Metrics Set of Shapes ◮ A shape: a smooth set of points in R n ◮ C 2 : seen as a function from its parameterization into R n ◮ F ( h 0 ) : distance to its skeleton � h 0 [D&Z] ◮ curvature � κ 0 = 1 / h 0 ◮ no double point: distance between two different parts � h 0 A Skel(A) > h 0 [D&Z]: M.C. Delfour & J.-P. Zol´ esio, Shapes and Geometries , 2000 Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Set of Shapes Shape Metrics Explicit ◮ � � d H (Γ 1 , Γ 2 ) = max sup d Γ 2 ( x ) , sup d Γ 1 ( x ) x ∈ Γ 1 x ∈ Γ 2 Γ 1 Γ with 2 d Γ 1 ( x ) = inf y ∈ Γ 1 d ( x , y ) d ( , ) H Γ Γ 1 2 Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Set of Shapes Shape Metrics Explicit - Implicit ◮ 2 2 � � � � � ˜ − ˜ � ∇ ˜ d Γ 1 −∇ ˜ d W 1 , 2 (Γ 1 , Γ 2 ) 2 = + d Γ 1 d Γ 2 d Γ 2 � � � � � � L 2 (Ω , R ) L 2 (Ω , R n ) Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Set of Shapes Shape Metrics Explicit - Implicit - Path-based [T&Y] ◮ 2 � � � ∂ � � ∂ t v ( t , · ) dt arg min � � � � v , v (0 , · ) = Γ 1 t H 1 (Ω , R n ) v (1 , · ) = Γ 2 [T&Y]: All work by A. Trouv´ e & L. Younes Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Topological equivalence Topological equivalence On the previous set of smooth shapes: ◮ Hausdorff distance ◮ L 2 or W 1 , 2 norm between the signed distance functions ◮ area of the symmetric difference These metrics are topologically equivalent ! Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Topological equivalence Topological equivalence On the previous set of smooth shapes: ◮ Hausdorff distance ◮ L 2 or W 1 , 2 norm between the signed distance functions ◮ area of the symmetric difference These metrics are topologically equivalent ! ◮ Same notion of convergence ◮ Qualitatively different behaviour at greater scales ◮ Hausdorff distance: more geometrical sense Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Gradient Descent II - Variational Shape Warping Shape Gradient E (Γ + ε v ) − E (Γ) � � Directional derivative: G Γ E (Γ) , v = lim ε ε → 0 Curve Γ Deformation field v, in the tangent space of Γ Γ + v Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Gradient Descent II - Variational Shape Warping Shape Gradient E (Γ + ε v ) − E (Γ) � � Directional derivative: G Γ E (Γ) , v = lim ε ε → 0 � � Gradient: field ∇ E , ∀ v ∈ F , G Γ E (Γ) , v = �∇ E | v � F Usual tangent space: F = L 2 : � � f | g � L 2 = f ( x ) · g ( x ) d Γ( x ) Γ Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Gradient Descent Gradient Descent Scheme ◮ Build minimizing path: Γ(0) = Γ 1 ∂ Γ ∂ t = −∇ F Γ E (Γ) Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Gradient Descent Gradient Descent Scheme ◮ Build minimizing path: Γ(0) = Γ 1 ∂ Γ ∂ t = −∇ F Γ E (Γ) ◮ Change the inner product F = ⇒ change the minimizing path [C&P]: G. Charpiat, J.-P. Pons, R. Keriven & O. Faugeras, ICCV 2005 [SYM]: G. Sundaramoorthi, A.J. Yezzi & A. Mennucci, VLSM 2005 [T98]: A. Trouv´ e, IJCV 1998! Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Gradient Descent Gradient Descent Scheme ◮ Build minimizing path: Γ(0) = Γ 1 ∂ Γ ∂ t = −∇ F Γ E (Γ) ◮ Change the inner product F = ⇒ change the minimizing path � + 1 � 2 � v � 2 ◮ −∇ F � � Γ E (Γ) = arg min G Γ E (Γ) , v F v ◮ F as a prior on the minimizing flow Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Generalized Gradients Generalized Gradients: Spatially Coherent Flows ◮ L 2 inner product � � f | g � L 2 = f ( x ) · g ( x ) d Γ( x ) Γ Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Generalized Gradients Generalized Gradients: Spatially Coherent Flows ◮ L 2 inner product ◮ H 1 inner product � � f | g � L 2 = f ( x ) · g ( x ) d Γ( x ) Γ � f | g � H 1 = � f | g � L 2 + � ∂ x f | ∂ x g � L 2 2 � � ∇ H 1 E � u − ∇ L 2 E L 2 + � ∂ x u � 2 = arg inf � � L 2 � u Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior

Introduction Shapes and Shape Metrics Variational Shape Warping Statistics Segmentation with prior Summary Generalized Gradients Generalized Gradients: Spatially Coherent Flows ◮ L 2 inner product ◮ H 1 inner product ◮ Set S of prefered transformations (e.g. rigid motion) Projection on S : P Projection orthogonal to S : Q ( P + Q = Id ) � f | g � S = � P ( f ) | P ( g ) � L 2 + α � Q ( f ) | Q ( g ) � L 2 Guillaume Charpiat PhD Defense Shape Statistics for Image Segmentation with Prior