Variational image segmentation Nicolas Rougon Institut Mines-Tlcom - PowerPoint PPT Presentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 3D membrane 3D membrane energy � � w 10 ( u ) | x u | 2 + w 01 ( u ) | x v | 2 � E ( C ) = d u ◮ Smoothness properties → regularization E is a 1 st -order Tikhonov stabilizer minimizing E over some shape space yields a.e. C 1 -continuous solutions ( w ij ) are regularization functions controlling C 1 -continuity locally smoothness at x ( u ) increases with w ij ( u ) setting w ij ( u 0 ) = 0 allows C to be C 1 -discontinuous at x ( u 0 ) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Thin-plates ◮ A thin-plate (also called shell) is a codimension-1 elastic material which can only deform by bending along its normal n n = 2 n = 3 n n n � x ss Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 2D thin-plate 2D thin-plate energy � β ( u ) | x uu | 2 du E ( C ) = Hyperparameter: nonnegative function β over Λ ◮ Elastic properties → deformation β is a bending stiffness density defining local resistance to flexion setting β ( u 0 ) = 0 enables free deformation at x ( u 0 ) the standard model is the homogeneous thin-plate for which β ( u ) = β > 0 for all u ∈ Λ Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 2D thin-plate 2D thin-plate energy � β ( u ) | x uu | 2 du E ( C ) = ◮ Smoothness properties → regularization E is a 2 nd -order Tikhonov stabilizer minimizing E over some shape space yields a.e. C 2 -continuous solutions β is a regularization function controlling C 2 -continuity locally smoothness at x ( u ) increases with β ( u ) setting β ( u 0 ) = 0 allows C to be C 2 -discontinuous at x ( u 0 ) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 3D thin-plate 3D thin-plate energy � � w 20 ( u ) | x uu | 2 + 2 w 11 ( u ) | x uv | 2 + w 02 ( u ) | x vv | 2 � E ( C ) = d u Hyperparameters: nonnegative functions ( w ij ) i + j =2 over Λ ◮ Elastic properties → deformation ( w ij ) are directional bending stiffness densities defining local resistance to flexion setting w ij ( u 0 ) enables free deformation at x ( u 0 ) the standard model is the homogeneous isotropic thin-plate for which w ij ( u ) = β > 0 for all u ∈ Λ Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 3D thin-plate 3D thin-plate energy � � w 20 ( u ) | x uu | 2 + 2 w 11 ( u ) | x uv | 2 + w 02 ( u ) | x vv | 2 � E ( C ) = d u ◮ Smoothness properties → regularization E is a 2 nd -order Tikhonov stabilizer minimizing E over some shape space yields a.e. C 2 -continuous solutions ( w ij ) are regularization functions controlling C 2 -continuity locally smoothness at x ( u ) increases with w ij ( u ) setting w ij ( u 0 ) = 0 allows C to be C 2 -discontinuous at x ( u 0 ) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Balloons ◮ A balloon is a codimension-1 elastic material which can only deform by inflating / deflating along its normal n n = 2 n = 3 n n n ⊥ x u n � x u ∧ x v Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 2D balloon 2D pressure energy � du � E ( C ) = ± � p ( x ) , x u det Hyperparameter: irrotational vector field p over Ω ⊂ R 2 ◮ Elastic properties → deformation p is a pressure density defining local resistance to inflation setting p ( x ( u 0 )) = 0 enables free deformation at x ( u 0 ) the standard model is the homogeneous isotropic pressure for which p ( x ) = p x ( p > 0) for all x ∈ Ω Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 2D balloon 2D pressure energy � du � E ( C ) = ± � p ( x ) , x u det ◮ Global geometric properties → regularization For a homogeneous isotropic pressure ( p ( x ) = p x ) � du = ± p · Area ( R in ) � E ( C ) = ± p det � x , x u A 2D balloon is a minimal / maximal area segmenting curve Proof: use Green-Stokes theorem to transform the boundary integral into a domain integral Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 2D balloon Theorem (Green-Stokes) Given a C 1 vector field V : Ω → R 2 and a compact domain R ⊂ Ω with smooth boundary ∂ R (unit normal n ) � � ∇ · V ( x ) d x = ( V · n ) ( x ( s )) ds R ∂ R � T , this rewrites as � P , Q Letting V = � � � ( P x + Q y ) dxdy = ( Py u − Qx u ) du = det ( V , x u ) du R ∂ R ∂ R Setting V ( x ) = x , we get � 2 Area ( R ) = det ( x , x u ) du � Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 3D balloon 3D pressure energy � d u � E ( C ) = ± det � p ( x ) , x u , x v Hyperparameter: irrotational vector field p over Ω ⊂ R 3 ◮ Elastic properties → deformation p is a pressure density defining local resistance to inflation setting p ( x ( u 0 )) = 0 enables free deformation at x ( u 0 ) the standard model is the homogeneous isotropic pressure for which p ( x ) = p x ( p > 0) for all x ∈ Ω Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 3D balloon 3D pressure energy � d u � E ( C ) = ± � p ( x ) , x u , x v det ◮ Global geometric properties → regularization For a homogeneous isotropic pressure ( p ( x ) = p x ) � du = ± p · Volume ( R in ) � E ( C ) = ± p det � x , x u , x v A 3D balloon is a minimal / maximal volume segmenting surface Proof: use Green-Ostrogradski theorem to transform the boundary integral into a domain integral Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 3D balloon Theorem (Green-Ostrogradski) Given a C 1 vector field V : Ω → R 3 and a compact domain R ⊂ Ω x u ∧ x v with smooth boundary ∂ R (unit normal n = | x u ∧ x v | ) � � � ∇ · V ( x ) d x = V · d S = ( V · n ) ( x ( u )) | x u ∧ x v | d u R ∂ R ∂ R � T and using a · ( b ∧ c ) = det ( a , b , c ) Letting V = � P , Q , R this rewrites as � � ( P x + Q y + R z ) d x = det ( V , x u , x v ) d u R ∂ R Setting V ( x ) = x , we get � 3 Volume ( R ) = det ( x , x u , x v ) d u � Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Internal energies � d u � � x ( u ) E int ( C ) = V int where: V int = V membrane + V thin − plate + V balloon 2D active contour V int ( x ) = α ( u ) | x u | 2 + β ( u ) | x uu | 2 ± det � p ( x ) , x u ) 3D active contour V int ( x ) = w 10 ( u ) | x u | 2 + w 01 ( u ) | x v | 2 + w 20 ( u ) | x uu | 2 + 2 w 11 ( u ) | x uv | 2 + w 02 ( u ) | x vv | 2 ± det � p ( x ) , x u , x v ) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization External energies ◮ The interactions of C with image content are specified via an external energy E ext ( C , L ) energy additivity provides a flexible mechanism for integrating multiple relevant image information for the problem at hand � λ i E i E ext ( C , L ) = ext ( C , L ) the influence of image energy E i ext in the overall segmentation criterion is controlled by the hyperparameter λ i Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization External energies ◮ Image energies E i ext are derived from potential maps V i ext ( L ) computed from image data over Ω � E i V i ext ( C , L ) = ext ( L )( x ( u )) du contour-based Λ � E i V i ext ( C , L ) = region-based ext ( L )( x ) d x R in ◮ Active contour modeling relies on designing relevant image potentials for the application at hand. V i ext ( L ) are expected to depend on discriminative image features be minimal along C (if contour-based) or over R in (if region-based) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization External energies ◮ Luminance potentials deterministic target V i properties ext ( L ) Dark/bright object dark L contour / region-based bright − L µ in : mean luminance over R in Low-texture object ( L − µ in ) 2 ≡ constant image model region variance (a.k.a. cartoon model) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization External energies ◮ Luminance potentials statistical target V i ext ( L )( x ) properties p R in : pdf of luminance over R in parametric estimator p R in ( L | Θ ) High-texture object e.g. Gaussian Mixture Model − log p R in � L ( x ) � Gaussian pdf ≡ cartoon model region log-likelihood non-parametric estimator e.g. kernel density estimator supervised ( p R in learnt from a training set) | unsupervised setting Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization External energies ◮ Edge potentials V i target properties ext ( L ) High-contrast boundary polynomial −|∇ L | p p ≥ 1 bounded decreasing ϕ ( |∇ L | ) ϕ (0) = 1 , x → + ∞ ϕ ( x ) = 0 lim | ∆ L | p Laplacian zero-crossing p ≥ 1 ◮ medium | large capture range Binary edge map C distance function to C distance-based d C − K σ ⋆ C K σ : kernel with width σ convolutive e.g. Gaussian Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization External energies ◮ Example application Hand tracking using active contours 1 Feature extraction Canny-Deriche edge detection edge map post-processing raw edge map hand silhouette C 2 Potential map generation chamfer distance d C to C V ext ( L ) V ext ( L ) level lines Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Total energy ◮ Summing elastic and image energies yields a total energy E ( C ) ≡ segmentation criterion � d u � � x ( u ) E ( C ) = E int ( C ) + E ext ( C , L ) = V where: V = V int + � λ i V i ext ( L ) ◮ Admissible segmentations C ∗ are minimizers of E ( C ) over the shape space S C ∗ = arg min C ∈ S E ( C ) C ∗ is an equilibrium state between internal / external actions, derived from elastic / image energies ◮ Modeling issues shape space S optimization technique Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Shape spaces ◮ Finite-dimensional shape spaces → parametric estimation The coordinates of x ∈ C can be expressed in some basis ( φ i ) i � x ( u ) = θ i φ i ( u ) C is completely defined by Θ = ( θ i ) i ≡ model parameters E ( Θ ) is computed in closed-form and minimized w.r.t. Θ standard bases: B-spline � B-snakes | e iu � Fourier-snakes | wavelets � wavelet-snakes | ... ◮ Infinite-dimensional shape spaces → nonparametric estimation the location of every point x ∈ C must be estimated E ( C ) is minimized w.r.t. x Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Energy minimization ◮ Minimization techniques iteratively decrease E ( C ) by deforming C until a minimum is reached Direct (a.k.a. gradient-free) methods operate on E ( C ) Variational methods use the derivative of E ( C ) dim S < ∞ � � ∂ Θ E ( Θ ) = ∂ θ i E ( Θ ) i is a standard (vector) derivative dim S = ∞ ∂ x E ( C ) is a functional derivative ≡ force F ( x ) acting on x ∈ C F ( x ) = ∂ x E int ( C ) + ∂ x E ext ( C , L ) = F int ( x ) + F ext ( x ) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Gradient-free minimization ◮ Gradient-free optimizers are search methods applicable to discrete variables and feasible spaces C is sampled → point set / mesh a discrete expression of E ( C ) is derived finite difference | finite element methods candidate deformations of C are hypothetized from a finite set of possible variations for x or Θ for each of them, energy variation is computed the deformation maximizing energy decrease is applied this process is iterared until E ( C ) cannot be ց Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Gradient-free minimization ◮ Efficient algorithms are available dynamic programming Nelder-Mead (a.k.a. nonlinear) simplex . . . ◮ Performances fast accuracy depends on sampling density robust to noise model point clustering around global | “good” local attractors of V ext minimizer → fixed by periodic resampling / remeshing during iterations Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus � V ( f ) be a functional over some Hilbert function ◮ Let E ( f ) = � fg space with standard dot product < f , g > = Gâteaux derivative When it exists, the limit E ( f + εη ) − E ( f ) D η E ( f ) = lim ε ε → 0 defines the Gâteaux derivative of E at f in the direction η The Gâteaux derivative verifies D η E ( f ) = < ∂ f V , η > ∂ f V is referred to as the variational derivative of E ( f ) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus ◮ The extrema of E ( f ) verify D η E ( f ) = < ∂ f V , η > = 0 ∀ η Euler-Lagrange equations The minimizers of E ( f ) are solutions of the PDE ∂ f V = 0 known as Euler-Lagrange equations ◮ Hereafter, we derive general closed-form expressions for ∂ f V when f : Ω ⊂ R p → R q Key results for solving a variety of image understanding issues modeled as deterministic optimization problems Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus � V ( f , f u ) du ◮ We first consider the case of a functional E ( f ) = over a space of univariate scalar functions f ( u ) Performing Taylor expansion � E ( f + εη ) = V ( f + εη, f u + εη u ) du V ( f , f u ) + ε ∂ V ∂ f η + ε ∂ V � � � ≈ η u du ∂ f u � � ∂ V ∂ f η + ∂ V � = E ( f ) + ε η u du ∂ f u � gh ′ ) � b a g ′ h = [ gh ] b a − Integrating by parts ( � b � � ∂ V ∂ f − ∂ � ∂ V � ∂ V � � E ( f + εη ) ≈ E ( f ) + ε η du + ε η ∂ u ∂ f u ∂ f u a Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus Enforcing zero boundary conditions (BCs) for η η ( a ) = η ( b ) = 0 � � ∂ V − ∂ � ∂ V � � E ( f + εη ) ≈ E ( f ) + ε η du ∂ f ∂ u ∂ f u Taking the limit E ( f + εη ) − E ( f ) � ∂ V ∂ f − ∂ � ∂ V � � lim = , η ε ∂ u ∂ f u ε → 0 Hence the variational derivative ∂ f V ∂ f V = ∂ V ∂ f − ∂ � ∂ V � ∂ u ∂ f u Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus � V ( . . . , f u ( i ) , . . . ) du ◮ Consider an arbitrary functional E ( f ) = involving the i th derivative f u ( i ) of f Performing Taylor expansion � E ( f + εη ) = V ( . . . , f u ( i ) + εη u ( i ) , . . . ) du ∂ V � � � ≈ E ( f ) + ε . . . + ∂ f u ( i ) η u ( i ) + . . . du Integrating by parts i times with zero BCs for all η u ( k ) ( k ≤ i ) � � . . . ( − 1) i ∂ i ∂ V � � � E ( f + εη ) ≈ E ( f ) + ε + . . . η du ∂ u i ∂ f u ( i ) Taking the limit � � . . . ( − 1) i ∂ i E ( f + εη ) − E ( f ) ∂ V � � lim = + . . . , η ∂ u i ε ∂ f u ( i ) ε → 0 Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus This proves that ∂ f V has a generating term � � ( − 1) i ∂ i ∂ V ∂ u i ∂ f u ( i ) originating from the occurrence of the derivative f u ( i ) in V (using the convention f u (0) = f ) Variational derivative (univariate scalar problems) � V ( f , f u , . . . , f u ( n ) ) du with f ( u ) ∈ R Given a functional E ( f ) = n � � ( − 1) i ∂ i ∂ V � ∂ f V = ∂ u i ∂ f u ( i ) i =0 Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus � � � ff 2 u + f 2 ◮ Example: Let E ( f ) = du uu E ( f ) depends on a potential of the form V ( f , f u , f uu ). Hence � ∂ V + ∂ 2 ∂ f V = ∂ V − ∂ � ∂ V � � ∂ u 2 ∂ f ∂ u ∂ f u ∂ f uu We get ∂ u (2 ff u ) + ∂ 2 u − ∂ ∂ f V = f 2 ∂ u 2 (2 f uu ) = f 2 u − 2 f 2 u − 2 ff uu + 2 f uuuu = − f 2 u − 2 ff uu + 2 f uuuu � Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus � V ( f , f u ) du over a ◮ We now consider a functional E ( f ) = � ∈ R q � f j ( u ) space of univariate vector functions f ( u ) = u , . . . , f q V ( f , f u ) can be viewed as a function V ( f 1 , . . . , f q , f 1 u ) of 2 q independent scalar variables Performing similar computations coordinatewise readily yields � � ∂ f j V = ∂ V ∂ f j − ∂ ∂ V ∂ f j ∂ u u � � � � ∂ ∂ ∂ ∂ Letting ∂ f = and ∂ f u = leads to the vector form ∂ f j ∂ f j u ∂ f V = ∂ V ∂ f − ∂ � ∂ V � ∈ R q ∂ u ∂ f u → Vector generalization of the scalar case Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus ◮ These conclusions extend to the case of an arbitrary functional � V ( . . . f u ( i ) . . . ) du involving higher-order derivatives f u ( i ) E ( f )= Variational derivative (univariate vector problems) � V ( f , f u , . . . , f u ( n ) ) du with f ( u ) ∈ R q Given a functional E ( f ) = n ( − 1) i ∂ i � � ∂ V � ∂ f V = ∂ u i ∂ f u ( i ) i =0 The generating term of ∂ f V originates from the occurrence of the derivative f u ( i ) in V ◮ Use cases 2D parameterized active contours | minimal paths Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus � V ( f , f u 1 , f u 2 ) du 1 du 2 ◮ We next consider a functional E ( f ) = over a space of bivariate scalar functions f ( u 1 , u 2 ) Performing Taylor expansion � E ( f + εη ) = V ( f + εη, f u 1 + εη u 1 , f u 2 + εη u 2 ) du 1 du 2 � � ∂ V ∂ f η + ∂ V η u 1 + ∂ V � ≈ E ( f ) + ε η u 2 du 1 du 2 ∂ f u 1 ∂ f u 2 Using Green’s theorem ( � Ω ( P u 1 + Q u 2 ) = � ∂ Ω ( Py s − Qx s ) ) � ∂ V � ∂ V � � ∂ V ∂ ∂ � � � E ( f + εη ) ≈ E ( f ) + ε ∂ f − − η du 1 du 2 ∂ u 1 ∂ f u 1 ∂ u 2 ∂ f u 2 � ∂ V y s − ∂ V � � + ε η x s ds ∂ f u 1 ∂ f u 2 ∂ Ω Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus Enforcing zero BCs for η along ∂ Ω � ∂ V � ∂ V � � ∂ V ∂ f − ∂ − ∂ � � � E ( f + εη ) ≈ E ( f ) + ε η du 1 dvu 2 ∂ u 1 ∂ f u 1 ∂ u 2 ∂ f u 2 Hence the variational derivative ∂ f V � ∂ V � ∂ V ∂ f V = ∂ V ∂ ∂ � � ∂ f − − ∂ u 1 ∂ f u 1 ∂ u 2 ∂ f u 2 → Bivariate extension of the univariate case � V ( f , f u 1 , . . . , f u p ) du 1 . . . du p ◮ Extension to functionals E ( f ) = over a space of multivariate scalar functions f ( u 1 , . . . , u p ) � ∂ V p ∂ f V = ∂ V ∂ � � ∂ f − ∂ u k ∂ f u k k =1 Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus ◮ Given a multi-index i = ( i 1 , . . . , i p ) ∈ N p and u = ( u 1 , . . . , u p ) denote | i | = i 1 + · · · + i p ∂ u i = ∂ i 1+ ··· + ip ∂ i denote and f u ( i ) = f u ( i 1) ( ip ) ∂ u i 1 ip ... u 1 ...∂ u p p 1 � V ( . . . f u ( i ) . . . ) d u ◮ Consider an arbitrary functional E ( f ) = involving the partial derivative f u ( i ) of total order | i | of f Performing Taylor expansion and applying Green’s theorem | i | times using zero BCs for all η u ( k ) ( | k | ≤ | i | ) � � . . . ( − 1) | i | ∂ i � � ∂ V � E ( f + εη ) ≈ E ( f ) + ε + . . . η d u ∂ u i ∂ f u ( i ) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus Variational derivative (multivariate scalar problems) � V ( f , f u . . . f u ( n ) ) d u with f ( u ) ∈ R Given a functional E ( f ) = | n | ( − 1) | i | ∂ i � � ∂ V � ∂ f V = ∂ u i ∂ f u ( i ) | i | =0 ◮ Use cases level set active contours gray level image restoration (denoising, deblurring...) shape from X Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Example 1 | Variational 2D image denoising � d x � � � 2 d x + λ � L ( x ) − L n ( x ) � L 2 x + L 2 E ( L ) = y Ω Ω Gaussian noise model + 1 st -order quadratic stabilizer �∇ L � 2 � Since E ( L ) = V ( L , L x , L y ) dxdy � ∂ V � � ∂ L V = ∂ V ∂ L − ∂ − ∂ ∂ V � ∂ x ∂ L x ∂ y ∂ L y We get � ∂ ∂ x (2 L x ) + ∂ � ∂ f V = 2( L − L n ) − λ ∂ y (2 L y ) � L − L n − λ ∆ L � = 2 Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Example 2 | Variational 2D image denoising � d x � � � 2 d x + λ � L ( x ) − L n ( x ) � L 2 xx + 2 L 2 xy + L 2 E ( L ) = yy Ω Ω 2 nd -order quadratic stabilizer � D 2 L � 2 = tr � ( D 2 L ) 2 � � Since E ( L ) = V ( L , L xx , L xy , L yy ) dxdy � ∂ V ∂ L + ∂ 2 ∂ 2 � � + ∂ 2 � � ∂ L V = ∂ V ∂ V ∂ V � + ∂ x 2 ∂ y 2 ∂ L xx ∂ x ∂ y ∂ L xy ∂ L yy We get � ∂ 2 ∂ x ∂ y (4 L xy ) + ∂ 2 ∂ 2 � ∂ f V = 2( L − L n ) + λ ∂ x 2 (2 L xx ) + ∂ y 2 (2 L yy ) L − L n + λ ( L xxxx + 2 L xxyy + 2 L yyyy ) � � = 2 L − L n + λ ∆∆ L � � = 2 ∆∆ is the biharmonic operator Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational calculus ◮ The previous result extends to multivariate vector functions � ∈ R q in a coordinatewise fashion � f j ( u ) f ( u ) = Variational derivative (multivariate vector problems) � V ( f , f u . . . f u ( n ) ) d u with f ( u ) ∈ R q Given a functional E ( f ) = | n | ( − 1) | i | ∂ i � � ∂ V � ∂ f V = ∂ u i ∂ f u ( i ) | i | =0 ◮ Use cases parameterized active surfaces | surface reconstruction optical flow estimation | image registration color image / vector field restoration Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Elastic forces � d u � E int ( C ) = � x ( u ) V int where: V int = V membrane + V thin − plate + V balloon Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 2D internal forces type F int ( x ) V int − ∂ α | x u | 2 � α x u � membrane ∂ u ∂ 2 β | x uu | 2 thin-plate � β x uu � ∂ u 2 � | x u | n � p ( x ) , x u � � ∇ · p det − balloon quadratic potentials yield quasi-linear elastic forces pressure forces induce coordinate coupling in Euler equations Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 2D balloon forces ◮ Proof: Let p = ( p 1 , p 2 ) T � � p 1 ( x ) x u � � � = p 1 ( x ) y u − p 2 ( x ) x u = V ( x , x u ) V balloon = � � p 2 ( x ) y u � � � � ∂ V Computing ∂ x V = ∂ V ∂ x − ∂ � yields ∂ u ∂ x u ∂ p 1 ∂ x y u − ∂ p 2 ∂ x x u + ∂ p 2   ∂ u   F balloon ( x ) = ∂ p 1 ∂ y y u − ∂ p 2 ∂ y x u − ∂ p 1     ∂ u � ∂ ∂ u = x u ∂ ∂ x + y u ∂ ∂ y = x u · ∇ � Using chain rule � ∂ p 1 ∂ x + ∂ p 2 � � � � | x u | n y u = − � ∇· p F balloon ( x ) = � − x u ∂ y Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 3D internal forces type V int F int ( x ) � − ∂ − ∂ w 10 | x u | 2 + w 01 | x v | 2 membrane � w 10 x u � w 01 x v � ∂ u ∂ v ∂ 2 � + ∂ 2 w 20 | x uu | 2 + w 02 | x vv | 2 � w 20 x uu � w 02 x vv � ∂ u 2 ∂ v 2 thin-plate ∂ 2 + 2 w 11 | x uv | 2 � w 11 x uv � + 2 ∂ u ∂ v � | x u ∧ x v | n � p ( x ) , x u , x v � � ∇ · p det − balloon quadratic potentials yield quasi-linear elastic forces pressure forces induce coordinate coupling in Euler equations Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Elastic forces ◮ Owing to their linearity, elastic forces are compactly expressed via a linear differential operator K , known as the stiffness tensor F elastic ( x ) = F membrane ( x ) + F thin − plate ( x ) = K x K ∂ 2 β ∂ 2 − ∂ α ∂ � � � � n = 2 ∂ u 2 ∂ u 2 ∂ u ∂ u ∂ 2 ∂ 2 ∂ 2 ∂ 2 + ∂ 2 ∂ 2 � � � � � � w 20 + 2 w 11 w 02 ∂ u 2 ∂ u 2 ∂ u ∂ v ∂ u ∂ v ∂ v 2 ∂ v 2 n = 3 − ∂ ∂ − ∂ ∂ � � � � w 10 w 01 ∂ u ∂ u ∂ v ∂ v Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Image forces type F ext ( x ) E ext � V ext ( L )( x ( u )) d u ∇ V ext ( L )( x ) contour-based Λ � region-based V in ( L )( x ) d x − V in ( L )( x ) n R in image forces are nonlinear Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Region-based image forces ◮ Proof: Assume V in ( L ) is smooth and does not depend on R in There exists a smooth vector field V : Ω → R n such that V in ( L ) = ∇ · V Using Green’s theorem ( | x s | = | x s 1 ∧ x s 2 | = 1) � � V in ( L )( x ) d x = det ( V ( x ) , x s ) ds n = 2 R in ∂ R in � � n = 3 V in ( L )( x ) d x = det ( V ( x ) , x s 1 , x s 2 ) d s R in ∂ R in → Same expressions as the balloon energy Using the expression of the balloon force � n = − V in ( L )( x ) n F in ( x ) = − � ∇ · V � Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization 2-phase active region segmentation ◮ A generic 2-phase active region segmentation model is obtained by integrating data consistency constraints over the object region R in and the background R out = Ω \R in � � E ext ( C , L ) = λ in V in ( L )( x ) d x + λ out V out ( L )( x ) d x R in R out Since object (background) boundaries C = ∂ R in ( ∂ R out ) have opposite normals, the resulting image force is � � F ext ( x ) = λ out V out ( L )( x ) − λ in V in ( L )( x ) n ◮ The segmenting manifold evolves according to a region competition principle Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Variational optimization ◮ Since image forces are nonlinear, Euler equations cannot be solved in closed-form ◮ This issue is overcome by using numerical optimization methods Starting from an initialization, these techniques iteratively update the variable x to be estimated ( i.e. deform C ) so as to decrease E ( C ) until a minimum is reached A time variable t is introduced � � ( u , t ) ∈ Λ × R + � � C ( t ) = x ( u , t ) The evolution law for x is driven by ∂ x V , and can be mathematically- or physically-based Relevant schemes must ensure fast and stable convergence, and avoid non significant local minima Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Gradient-descent methods ◮ Gradient-descent methods are 1 st -order schemes in which the variable x to be optimized is updated along the steepest slope − ∂ x V of the energy surface They involve a speed law with E prototype deformatjon x t = − ∂ x V ( x ) - δ x V Many schemes are available – fixed | adaptive time step – conjugate gradient C – Levenberg-Marquardt initjalizatjon – Newton | quasi-Newton | BFGS segmentatjon – stochastic gradient – . . . Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Gradient-descent methods ◮ Bibliography A practical overview of gradient-free & gradient-descent methods with C ++ implementations can be found in W. Press - S. Teukolsky - W. Vetterling - B. Flannery Numerical Recipes - The Art of Scientific Computing Cambridge University Press, 2007 (3 rd Edition) ◮ For the sake of simplicty, hereafter we focus our attention on the basic fixed time step gradient-descent Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Physically-based methods ◮ Physically-based methods rely on Lagrangian mechanics, leading to 2 nd -order schemes derived from the least action principle ∂ ∂ t ( M x t ) + C x t + F ( x ) = 0 dissipation applied inertia forces Hyperparameters: mass tensor M , damping tensor C the standard model involve homogeneous isotropic mass and damping distributions ( M = m I , C = γ I) ( m , γ > 0) m x tt + γ x t + F ( x ) = 0 Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Discretization ◮ Discretizing the model requires Sampling C ( t ) (or, equivalently, the parameter space Λ) i = x ( u i , t ) ∈ Ω ⊆ R n ( i ∈ [1 .. N ]) – this yields a point set x t – C ( t ) is then represented as a vector of points X t = ( x t i ) Deriving discrete expressions for the forces F ( x t i ) Discretizing the evolution law for C ( t ) at samples x t i Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Discretizing forces ◮ Image forces An image force map F ext over the pixel grid is readily derived from the potential map V ext – for contour-based image energies defined by smooth potentials, F ext = ∇ V ext is estimated using a simple Sobel filter Image forces F ext ( x i ) at contour points (recall x i ∈ R n ) are then computed by interpolation from the map F ext Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Discretizing forces ◮ Internal forces involve spatial derivatives along C The latter can be discretized using finite differences (FD) u x i = x i − x i − 1 x u ( u i ) ≈ D − δ u u x i = x i +1 − 2 x i + x i − 1 x uu ( u i ) ≈ D 2 δ u 2 � = x i +2 − 4 x i +1 + 6 x i − 4 x i − 1 + x i − 2 x uuuu ( u i ) ≈ D 2 � D 2 u x i u δ u 4 where δ u is the spatial sampling step The stiffness tensor K is discretized into a stiffness matrix K acting on the discrete shape vector X Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Stiffness matrix ◮ Example: 2D closed contour with ◮ Properties of K homogeneous elastic properties arbitrary n / material functions positive   2α+6β −α−4β β β −α−4β symmetric   β  0  −α−4β band, hence sparse       band size depending β     on FD scheme   K =   β   cyclic if C closed     −α−4β   singular   0   β letting 1 = (1) i     −α−4β β β −α−4β 2α+6β one has K1 = 0 Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Discrete evolution law ◮ The evolution law involves time derivatives The latter are discretized using FD i − x t − 1 i = x t x t ( u i , t ) ≈ D − t x t i δ t i − 2 x t − 1 + x t − 2 � = x t x tt ( u i , t ) ≈ D − � D − t x t i i t i δ t 2 where δ t is the time step Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Discrete speed law ◮ For simplicity, we consider a (1 st -order) speed law x t = − F ( x ) Dividing applied forces into linear ( F elastic ) and nonlinear ( F nl = F balloon + F ext ) parts F ( x ) = K x + F nl ( x ) the evolution law splits into a linear l.h.s. and a nonlinear r.h.s. x t + K x = − F nl ( x ) Several discretization options are available, depending on the time at which force terms are evaluated i − x t − 1 x t + K x (?) = − F nl ( x (?) i ) i i δ t leading to various numerical iterative schemes Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Discrete speed law ◮ Explicit scheme All force terms are evaluated at time t − 1 i − x t − 1 x t + K x t − 1 = − F nl ( x t − 1 i ) i i δ t This yields a computationally simple scheme where current estimates are computed directly from previous ones � x t − 1 x t − δ t F nl ( x t − 1 i = � 1 − δ t K ) i i Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Discrete speed law ◮ Implicit scheme All force terms are evaluated at time t i − x t − 1 x t i + K x t i = − F nl ( x t i ) δ t Computing x t i requires inverting the update equation l.h.s � x t i + δ t F nl ( x t i ) = x t − 1 � 1 + δ t K i Intractable due to the nonlinearity of image forces Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Discrete speed law ◮ Semi-implicit scheme Linear (resp. nonlinear) terms are evaluated at time t (resp. t − 1) i − x t − 1 x t + K x t i = − F nl ( x t − 1 i ) i δ t This yields a mildly demanding scheme � − 1 � � x t − 1 − δ t F nl ( x t − 1 x t � 1 + δ t K i = ) i i where the time-independent (sparse, symmetric) matrix � can be pre-inverted efficiently � 1 + δ t K Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Stable convergence ◮ The time step δ t conditions the convergence speed E To avoid divergence / oscillations around minima, the time step must be upper-bounded δ t ≤ ( δ t ) max - δ t F ( x t ) This constraint is known as the Courant-Friedrichs-Lewy (CFL) x t⁺² x t⁺¹ x t C condition for the numerical scheme The CFL condition is (only) necessary for convergence The value of ( δ t ) max depends on the FD scheme Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Stable convergence ◮ Implicit solvers are usually less prone to numerical instability ( δ t ) explicit ≤ ( δ t ) semi − implicit ≪ ( δ t ) implicit max max max Explicit schemes have low computational cost per iteration but require more iterations to converge Semi-implicit schemes have higher computational cost per iteration but converge more rapidly The choice of a numerical scheme should be dictated by the (average) distance of initialization to solution Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Energies Parameterized active contours Energy minimization Level set methods Model discretization Discrete Lagrangian evolution law ◮ All previous observations hold for the 2 nd -order evolution law of Lagragian mechanics m x tt + γ x t + F ( x ) = 0 which lead to more complex but more stable discrete schemes i − 2 x t − 1 + x t − 2 i − x t − 1 m x t + γ x t + K x (?) = − F nl ( x (?) i i i ) i i δ t 2 δ t Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Outline Foundations 1 Parameterized active contours 2 Level set methods 3 Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Limitations of parameterized active contours ◮ Explicit representation ◮ implicit representation iterative reparameterization ( ≡ resampling / remeshing) → computational complexity fixed topology → topological rigidity ◮ Energy properties ◮ energy redefinition non-intrinsic → solutions depend on metric non-convex, hence non-significant local minima → sensitivity to initialization global smoothness → non-preservation of shape discontinuities Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Geodesic active contours ◮ Homogeneous membrane in a contrast potential g 2 � |∇ L | � g bounded, decreasing s.t. g ( x ) + ∞ − − → 0 � du � � α | x u | 2 du + λ g 2 � |∇ L | ( x ) E 0 ( C , L ) = x E 0 can be viewed as the square on the hypotenuse of the right triangle T with edges √ {√ α | x u | , λ g ( x ) } in phase space Span( x , x u ) g ( | ∇ L |( x ) ) Maupertuis principle E 0 has the same minima as the intrinsic energy | x u | x u E ≡ Area ( T ) � | x u | du = � ds � � E ( C , L ) = � |∇ L | ( x ) � |∇ L | ( x ) g g This model is known as geodesic active contours Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Geodesic active contours � ds � � |∇ L | ( x ) E ( C , L ) = g ◮ 2 interpretations by analogy with a homogeneous membrane � E membrane ( C ) = α ds Energy of a membrane with image-dependent stiffness g ( |∇ L | ) + ∞ – since g ( u ) − − → 0, stiffness decreases along image edges. This allows to account for corner points Length of an image curve using the image-dependent � ds Riemannian metric d σ = g � |∇ L | – segmentations are minimal length curves (geodesics) for d σ Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Geodesic active contours ◮ Variational derivative ∂ x V = η ( x ) n where the force magnitude η ( x ) is η ( x ) = ∇ g · n − gk ∂ x V involves shape properties only (normal n , curvature k ) ( i.e. is metric-free). This is because E is intrinsic A more efficient model is obtained by adding a pressure term η ( x ) = ∇ g · n − g ( k ± p ) ◮ Geometric speed law x t = − η ( x ) n Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Geodesic active contours � | x u | and choose u ≡ s ◮ Proof: Let V ( x , x u ) = g � |∇ L | ( x ) Computing ∂ x V = ∂ V ∂ x − ∂ � ∂ V � yields ∂ s ∂ x s ∂ x V = ∇ g | x s | − ∂ � g t � ∂ s = ∇ g − ∂ g ∂ s t − gk n � and � ∂ ∂ ∂ ∂ y = t · ∇ Using chain rule ∂ s = x s ∂ x + y s expressing ∇ g in frame ( t , n ) � ∇ g = ( ∇ g · t ) t + ( ∇ g · n ) n � � n � gk − ∇ g · n ∂ x V = � Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Geodesic active surfaces ◮ Generalization to a surface C ⊆ R 3 in a 3D contrast potential � is achieved by substituting the arclength ds = | x u | du � |∇ L | g with the unit surface element da = | x u ∧ x v | d u � | x u ∧ x v | d u � � |∇ L | ( x ) E ( C , L ) = g ◮ Variational derivative ∂ x V = η ( x ) n η ( x ) = ∇ g · n − gH E is intrinsic, so that ∂ x V involves shape properties only (normal n , mean curvature H ) A more efficient model is obtained by adding a pressure term η ( x ) = ∇ g · n − g ( H ± p ) Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Level set representation ◮ The evolving manifold C ( t ) is represented as the zero level set of a smooth function ϕ over the image grid Ω C ( t ) = � � � ϕ ( x , t ) = 0 � x ∈ Ω ◮ Shape / motion properties of C ( t ) relate to the derivatives of ϕ n = − ∇ ϕ unit normal |∇ ϕ | curv( ϕ ) = ∇ · n (mean) curvature ϕ t = − η |∇ ϕ | speed D ϕ = ϕ t + ∇ ϕ · x t = 0 x t = − η ( x ) n Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Level set representation ◮ Though ϕ is a smooth function, its zero level set can be non-smooth → image discontinuity preservation change its topolopy in time → topological adaptivity φ ( x , t ) φ ( x , t ) C ⁽ t ⁾ C ⁽ t ⁾ φ = 0 φ = 0 fusion | split Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Geodesic active contours ◮ Level set-based evolution law ≡ reaction-diffusion PDE � � ∇ ϕ ϕ t = ± p g |∇ ϕ | + |∇ ϕ | ∇ · g |∇ ϕ | hyperbolic parabolic ϕ is initialized as the normalized signed distance to C (0) � − d C (0) ( x ) if x ∈ R in 1 ϕ ( x , 0) = max Ω d C (0) ( x ) if x ∈ R out + d C (0) ( x ) From the binary image of C (0) d C ⁽⁰⁾ ( x ) d C (0) is computed using a discrete − d C ⁽⁰⁾ ( x ) − d C ⁽⁰⁾ ( x ) distance function C ⁽⁰⁾ sign ( ≡ orientation) is assigned from C ⁽⁰⁾ a connected component labeling Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Chamfer distances ◮ Estimation of geodesic distance map d ij to object O ⊆ Ω ⊂ Z 2 via sequential propagation of local integer approximations m ij of Euclidean distance in pixel neighborhood N Initialization m ⁺ � 0 if x ij ∈ O d ij = d ₂ d ₁ d ₂ + ∞ otherwise d ₁ 0 d ₁ Forward / backward scans using half-masks m ± � � d ₂ d ₁ d ₂ d ij , d i + ı ′ , j +  ′ + m ± d ij = min ı ′  ′ ( ı ′  ′ ) ∈ N ± m - Optimal coefficients m ij N ⁺ d 1 = 3 d 2 = 4 N - 1 Normalization by d 1 Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Chamfer distances ◮ Estimation of geodesic distance map d ijk to object O ⊆ Ω ⊂ Z 3 via sequential propagation of local integer approximations m ijk of Euclidean distance in voxel neighborhood N m ⁺ d ₃ d ₂ d ₃ Initialization � 0 if x ijk ∈ O k ’= -1 d ₂ d ₁ d ₂ d ijk = + ∞ otherwise d ₃ d ₂ d ₃ Forward / backward scans using half-masks m ± d ₂ d ₁ d ₂ � � d ₁ 0 d ₁ d ijk , d i + ı ′ , j +  ′ , k + k ′ + m ± k ’= 0 d ijk = min ı ′  ′ k ′ ( ı ′  ′ k ′ ) ∈ N ± d ₂ d ₁ d ₂ Optimal coefficients m ijk d ₃ d ₂ d ₃ d 1 = 3 d 2 = 4 d 3 = 5 k ’= 1 d ₂ d ₁ d ₂ 1 Normalization by m - d ₃ d ₂ d ₃ d 1 Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Discrete evolution law g ∇ ϕ � � ϕ t = ± p g |∇ ϕ | + |∇ ϕ | ∇ · |∇ ϕ | � is computed analytically from the ◮ The potential map g � |∇ L | (smooth) contrast map |∇ L | ← Canny-Deriche gradient filter � |∇ L | � � |∇ L | ij � g ij = g ◮ The evolution law is discretized using FD distinct schemes are used for the hyperbolic / parabolic terms explicit iterative scheme �� ( t − 1) g ∇ ϕ � � ϕ ( t ) = ϕ ( t − 1) + δ t ± p g |∇ ϕ | + |∇ ϕ | ∇ · |∇ ϕ | Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Discrete evolution law ◮ Parabolic term ≡ smoothing effect → standard FD schemes apply g ∇ ϕ � � |∇ ϕ | ∇ · = g |∇ ϕ | curv( ϕ ) + ∇ g · ∇ ϕ |∇ ϕ | ∇ g is computed using a Sobel gradient filter ( g is smooth) both terms are computed analytically by plugging standard FD estimates of 1 st / 2 nd -order derivatives of ϕ ϕ x ( x ij ) ≈ D x ϕ ij = ϕ i +1 , j − ϕ i − 1 , j 2 δ x x ϕ ij = ϕ i +1 , j − 2 ϕ ij + ϕ i − 1 , j ϕ xx ( x ij ) ≈ D 2 δ 2 x Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Discrete evolution law ◮ Hyperbolic term ≡ transport effect (speed η ( x ) = ± p g ) Singularities arise along non-convex y parts of level sets due to self-collisions η ( x ) C ⁽ t ⁺¹ ⁾ ≡ shocks n - n ⁺ C ⁽ t ⁾ At shocks, ϕ is not differentiable. Yet, sided derivatives of ϕ exist and allow to define a proper normal x motion direction n − or n + Building C ( t +1) geometrically by moving x ∈ C ( t ) according to the speed law does not yield a simply connected curve The proper way for building C ( t +1) is Huygens’ construction → this is performed by monotonic conservative FD schemes Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Discrete evolution law ◮ Hyperbolic term ≡ transport effect Monotonic conservative FD schemes y – lateral FD approximations of sided derivatives η ( x ) C ⁽ t ⁺¹ ⁾ n - n ⁺ x ϕ ij = ϕ i +1 , j − ϕ ij C ⁽ t ⁾ ϕ + x ( x ij ) ≈ D + δ x x ϕ ij = ϕ ij − ϕ i − 1 , j ϕ − x ( x ij ) ≈ D − δ x x – automated selection of relevant sided derivatives based on their sign using Boolean switch functions Nicolas Rougon IMA4509 | Variational image segmentation

Foundations Geodesic active contours Parameterized active contours Level set-based active regions Level set methods Discrete evolution law ◮ Hyperbolic term ≡ transport effect Monotonic conservative FD schemes – forward motion ( p > 0) � � ξ ϕ ij , 0) 2 + max( D − min( D + p g |∇ ϕ ij | ≈ p g ξ ϕ ij , 0) 2 ξ = x , y – backward motion ( p < 0) � � ξ ϕ ij , 0) 2 + max( D + min( D − ξ ϕ ij , 0) 2 − p g |∇ ϕ ij | ≈ − p g ξ = x , y n C ⁽ t ⁺¹ ⁾ n - n ⁺ straightforward 3D extension C ⁽ t ⁾ Nicolas Rougon IMA4509 | Variational image segmentation