geometric constraints and variational approaches to image
play

Geometric Constraints and Variational Approaches to Image Analysis - PowerPoint PPT Presentation

Geometric Constraints and Variational Approaches to Image Analysis Daniel Martins Antunes 1 Supervised by: Jacques-Olivier Lachaud 1 and Hugues Talbot 2 1 LAMA, Universit Savoie Mont Blanc 2 CentraleSuplec, Universit Paris-Saclay Le


  1. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) . min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Geometric Constraints and Variational Approaches to Image Analysis 5 Daniel Martins Antunes

  2. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) . min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Geometric Constraints and Variational Approaches to Image Analysis 5 Daniel Martins Antunes

  3. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) . min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Geometric Constraints and Variational Approaches to Image Analysis 5 Daniel Martins Antunes

  4. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) . min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Geometric Constraints and Variational Approaches to Image Analysis 5 Daniel Martins Antunes

  5. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  6. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  7. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  8. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  9. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  10. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap min X ⊂ Ω Data ( X ) + 1 2 Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  11. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap min X ⊂ Ω Data ( X ) + 1 2 Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  12. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap min X ⊂ Ω Data ( X ) + 1 2 Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  13. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap min X ⊂ Ω Data ( X ) + 1 2 Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  14. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap � ∂X α + βκ 2 ds. min X ∈ Ω − The elastica energy Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  15. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Completion property min X ⊂ Ω Data ( X ) + Perimeter ( ∂X ) + Curvature 2 ( ∂X ) . Larger gap � ∂X α + βκ 2 ds. min X ∈ Ω − The elastica energy Geometric Constraints and Variational Approaches to Image Analysis 6 Daniel Martins Antunes

  16. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation State-of-the-art Continuous setting : Define the energy over the whole domain and minimize the elastica with respect the level-curves ( Chan, S. H. Kang, Kang, and Shen 2002 ). � ∇ f I � � 2 � � α + β ∇ · �∇ f I � d Ω . �∇ f I � Ω Geometric Constraints and Variational Approaches to Image Analysis 7 Daniel Martins Antunes

  17. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation State-of-the-art Continuous setting : Define the energy over the whole domain and minimize the elastica with respect the level-curves ( Chan, S. H. Kang, Kang, and Shen 2002 ). � ∇ f I � � 2 � � α + β ∇ · �∇ f I � d Ω . �∇ f I � Ω ◮ Numerical instability: Fourth-order Euler-Lagrange equation. ◮ Susceptible to bad local minimum. Geometric Constraints and Variational Approaches to Image Analysis 7 Daniel Martins Antunes

  18. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation State-of-the-art Continuous setting : Define the energy over the whole domain and minimize the elastica with respect the level-curves ( Chan, S. H. Kang, Kang, and Shen 2002 ). � ∇ f I � � 2 � � α + β ∇ · �∇ f I � d Ω . �∇ f I � Ω ◮ Numerical instability: Fourth-order Euler-Lagrange equation. ◮ Susceptible to bad local minimum. Discrete setting : T-junctions matching Fast algorithm, but limited to absolute value of Masnou and Morel 1998 curvature (polygonal solutions) and inpainting application. Geometric Constraints and Variational Approaches to Image Analysis 7 Daniel Martins Antunes

  19. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation State-of-the-art Continuous setting : Define the energy over the whole domain and minimize the elastica with respect the level-curves ( Chan, S. H. Kang, Kang, and Shen 2002 ). � ∇ f I � � 2 � � α + β ∇ · �∇ f I � d Ω . �∇ f I � Ω ◮ Numerical instability: Fourth-order Euler-Lagrange equation. ◮ Susceptible to bad local minimum. Discrete setting : T-junctions matching Fast algorithm, but limited to absolute value of Masnou and Morel 1998 curvature (polygonal solutions) and inpainting application. Linear programming Global formulation, but prohibitive running times Schoenemann, Kahl, and Cremers even for small (thus unprecise) neighborhoods. Not 2009 suitable for digital sets. Geometric Constraints and Variational Approaches to Image Analysis 7 Daniel Martins Antunes

  20. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation State-of-the-art Continuous setting : Define the energy over the whole domain and minimize the elastica with respect the level-curves ( Chan, S. H. Kang, Kang, and Shen 2002 ). � ∇ f I � � 2 � � α + β ∇ · �∇ f I � d Ω . �∇ f I � Ω ◮ Numerical instability: Fourth-order Euler-Lagrange equation. ◮ Susceptible to bad local minimum. Discrete setting : T-junctions matching Fast algorithm, but limited to absolute value of Masnou and Morel 1998 curvature (polygonal solutions) and inpainting application. Linear programming Global formulation, but prohibitive running times Schoenemann, Kahl, and Cremers even for small (thus unprecise) neighborhoods. Not 2009 suitable for digital sets. Triple cliques Global formulation, quadratic non-submodular Nieuwenhuis, Toeppe, Gorelick, energy. Limited precision due combinatorial Veksler, and Boykov 2014 explosion. Geometric Constraints and Variational Approaches to Image Analysis 7 Daniel Martins Antunes

  21. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Goals Models based on the minimization of the elastica energy Continuous Discrete Digital Numerical instability Yes No No Suitable for digital sets No No Yes Rounding issues Yes No No Contour completion Partial Partial Extended Global optimum (Free elastica) - - Yes Geometric Constraints and Variational Approaches to Image Analysis 8 Daniel Martins Antunes

  22. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Outline 1. Motivation ◮ Image analysis and geometric priors ◮ Elastica model and completion property ◮ State-of-the-art 2. Contribution ◮ Digital sets and convergent estimators ◮ A combinatorial model for elastica ◮ A quadratic non-submodular formulation for elastica ◮ Elastica minimization via graph-cuts 3. Conclusion and perspectives Geometric Constraints and Variational Approaches to Image Analysis 9 Daniel Martins Antunes

  23. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators ◮ Digital grid particularities and restrictions. ◮ Multigrid convergence of geometric estimators. Geometric Constraints and Variational Approaches to Image Analysis 10 Daniel Martins Antunes

  24. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Digital set peculiarities Where can we do better? ◮ Most of models neglect the digital character of digital images and ignore the fact that geometric measurements (mainly those local as tangent and curvature) in such objects should be done with a definition of convergence that is specific for digital shapes. Geometric Constraints and Variational Approaches to Image Analysis 11 Daniel Martins Antunes

  25. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Digital set peculiarities Where can we do better? ◮ Most of models neglect the digital character of digital images and ignore the fact that geometric measurements (mainly those local as tangent and curvature) in such objects should be done with a definition of convergence that is specific for digital shapes. Exact sampling x digitization Geometric Constraints and Variational Approaches to Image Analysis 11 Daniel Martins Antunes

  26. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Digital set peculiarities Where can we do better? ◮ Most of models neglect the digital character of digital images and ignore the fact that geometric measurements (mainly those local as tangent and curvature) in such objects should be done with a definition of convergence that is specific for digital shapes. Digitization ambiguity Geometric Constraints and Variational Approaches to Image Analysis 11 Daniel Martins Antunes

  27. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Multigrid convergent estimators Definition (Multigrid convergence) Let X be a family of shapes in R n and u a geometric quantity that is defined for every shape X ∈ X . Further, let D h ( X ) denote the digitization of X with grid step h . The estimator ˆ u is multigrid convergent for X if and only if, for any X ∈ X there exists h X > 0 such that for every 0 < h < h X | ˆ u ( D h ( X )) − u ( X ) | ≤ τ ( h ) , with lim h → 0 τ ( h ) = 0 . Geometric Constraints and Variational Approaches to Image Analysis 12 Daniel Martins Antunes

  28. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Multigrid convergent estimators Definition (Multigrid convergence) Let X be a family of shapes in R n and u a geometric quantity that is defined for every shape X ∈ X . Further, let D h ( X ) denote the digitization of X with grid step h . The estimator ˆ u is multigrid convergent for X if and only if, for any X ∈ X there exists h X > 0 such that for every 0 < h < h X | ˆ u ( D h ( X )) − u ( X ) | ≤ τ ( h ) , with lim h → 0 τ ( h ) = 0 . Multigrid convergent estimator of area � Area ( X ) = h 2 | D h ( X ) | . Geometric Constraints and Variational Approaches to Image Analysis 12 Daniel Martins Antunes

  29. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Motivation Multigrid convergent estimators Disk of radius 5( Area ≈ 78 . 54) . h = 1 . 0 , � h = 1 2 , ˆ h = 1 4 , ˆ A = 81 . A = 79 . 25 . A = 78 . 56 . 16 , ˆ 1 32 , ˆ 1 64 , ˆ 1 h = A = 78 . 44 . h = A = 78 . 5 . h = A = 78 . 53 . Geometric Constraints and Variational Approaches to Image Analysis 13 Daniel Martins Antunes

  30. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Multigrid convergent estimators ◮ Minimum Length Polygon (MLP) Sloboda 1998 ◮ Proved multigrid convergent for piecewise 3 -smooth convex shapes. Geometric Constraints and Variational Approaches to Image Analysis 14 Daniel Martins Antunes

  31. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Multigrid convergent estimators ◮ Minimum Length Polygon (MLP) Sloboda 1998 ◮ Proved multigrid convergent for piecewise 3 -smooth convex shapes. Geometric Constraints and Variational Approaches to Image Analysis 14 Daniel Martins Antunes

  32. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Multigrid convergent estimators ◮ Minimum Length Polygon (MLP) Sloboda 1998 ◮ Proved multigrid convergent for piecewise 3 -smooth convex shapes. ◮ Integral Invariant (II) Coeurjolly, Lachaud, and Levallois 2013 ◮ Proved multigrid convergent for C 2 convex shapes with bounded curvature. Geometric Constraints and Variational Approaches to Image Analysis 14 Daniel Martins Antunes

  33. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Multigrid convergent estimators ◮ Minimum Length Polygon (MLP) Sloboda 1998 ◮ Proved multigrid convergent for piecewise 3 -smooth convex shapes. ◮ Integral Invariant (II) Coeurjolly, Lachaud, and Levallois 2013 ◮ Proved multigrid convergent for C 2 convex shapes with bounded curvature. � πr 2 � κ ( p ) = 3 ˆ − | B r ( p ) ∩ X | r 3 2 Geometric Constraints and Variational Approaches to Image Analysis 14 Daniel Martins Antunes

  34. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Multigrid convergent estimators ◮ Minimum Length Polygon (MLP) Sloboda 1998 ◮ Proved multigrid convergent for piecewise 3 -smooth convex shapes. ◮ Integral Invariant (II) Coeurjolly, Lachaud, and Levallois 2013 ◮ Proved multigrid convergent for C 2 convex shapes with bounded curvature. � πr 2 � κ ( p ) = 3 ˆ − | B r ( p ) ∩ X | r 3 2 Geometric Constraints and Variational Approaches to Image Analysis 14 Daniel Martins Antunes

  35. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Conclusion ◮ Digital sets are ambiguous and are constrained to the digital grid. ◮ The multigrid convergence is an adapted definition of convergence for geometric estimation on digital sets. Geometric Constraints and Variational Approaches to Image Analysis 15 Daniel Martins Antunes

  36. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Digital sets and convergent estimators Conclusion ◮ Digital sets are ambiguous and are constrained to the digital grid. ◮ The multigrid convergence is an adapted definition of convergence for geometric estimation on digital sets. Can we construct optimization models using multigrid convergent estimators? Geometric Constraints and Variational Approaches to Image Analysis 15 Daniel Martins Antunes

  37. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References A combinatorial model for elastica ◮ Validate that multigrid convergent estimators can be used in optimization models. ◮ LocalSearch algorithm. ◮ Global optimum for the free elastica. Geometric Constraints and Variational Approaches to Image Analysis 16 Daniel Martins Antunes

  38. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Digital elastica � α + βκ 2 ds. Continuous elastica: ∂S Definition (Digital elastica energy) Let ˆ κ and ˆ s multigrid convergent estimators of curvature and local length. The digital elastica energy of a digital shape D ⊂ Ω ⊂ Z 2 of parameters θ = ( α ≥ 0 , β ≥ 0) is defined as � � ˆ � κ 2 (˙ E θ ( D ) = ˆ s ( ˙ e ) α + β ˆ e ) . e ∈ ∂ h ( D ) ˙ Geometric Constraints and Variational Approaches to Image Analysis 17 Daniel Martins Antunes

  39. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Digital elastica � α + βκ 2 ds. Continuous elastica: ∂S Definition (Digital elastica energy) Let ˆ κ and ˆ s multigrid convergent estimators of curvature and local length. The digital elastica energy of a digital shape D ⊂ Ω ⊂ Z 2 of parameters θ = ( α ≥ 0 , β ≥ 0) is defined as � � ˆ � κ 2 (˙ E θ ( D ) = ˆ s ( ˙ e ) α + β ˆ e ) . e ∈ ∂ h ( D ) ˙ ◮ The digital elastica energy converges (multigrid) to the continuous elastica. Geometric Constraints and Variational Approaches to Image Analysis 17 Daniel Martins Antunes

  40. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Digital elastica � α + βκ 2 ds. Continuous elastica: ∂S Definition (Digital elastica energy) Let ˆ κ and ˆ s multigrid convergent estimators of curvature and local length. The digital elastica energy of a digital shape D ⊂ Ω ⊂ Z 2 of parameters θ = ( α ≥ 0 , β ≥ 0) is defined as � � ˆ � κ 2 (˙ E θ ( D ) = s ( ˙ ˆ e ) α + β ˆ e ) . e ∈ ∂ h ( D ) ˙ ◮ The digital elastica energy converges (multigrid) to the continuous elastica. ◮ Local search : set a local neighborhood W ( D ) of D and pick the shape X ⋆ ∈ W ( D ) among those of minimum digital elastica value. Geometric Constraints and Variational Approaches to Image Analysis 17 Daniel Martins Antunes

  41. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Neighborhood of shapes ◮ Members of W ( D ) are constructed by removing or adding a set of connected pixels to D . Geometric Constraints and Variational Approaches to Image Analysis 18 Daniel Martins Antunes

  42. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Neighborhood of shapes ◮ Members of W ( D ) are constructed by removing or adding a set of connected pixels to D . Geometric Constraints and Variational Approaches to Image Analysis 18 Daniel Martins Antunes

  43. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Neighborhood of shapes ◮ Members of W ( D ) are constructed by removing or adding a set of connected pixels to D . ˆ D ( k +1) ← − arg min E θ ( X ) . X ∈W ( D ( k ) ) ◮ We use the integral invariant estimator (II-r) to estimate the curvature. Geometric Constraints and Variational Approaches to Image Analysis 18 Daniel Martins Antunes

  44. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Free elastica Free elastica: ˆ D ( k +1) ← − arg min E θ ( X ) . X ∈W ( D ( k ) ) Triangle Square D (0) Flower Bean Geometric Constraints and Variational Approaches to Image Analysis 19 Daniel Martins Antunes

  45. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Free elastica evolution � � � ˆ κ 2 ( ˙ E θ ( D ) = s ( ˙ ˆ e ) α + β ˆ e ) . e ∈ ∂ h ( D ) ˙ II- 5 , α = 0 . 01 , β = 1 . Geometric Constraints and Variational Approaches to Image Analysis 20 Daniel Martins Antunes

  46. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Energy evolution � α + βκ 2 ds min E ( X ) = ∂X � 1 / 2 = 4 πβ 1 � α r = 4 πβ , β ∂r 2 π ( αr + β ∂ where r ) = 0 . For α = 0 . 01 , β = 1 , min E ( X ) ≈ 1 . 2566 . Geometric Constraints and Variational Approaches to Image Analysis 21 Daniel Martins Antunes

  47. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Energy evolution � α + βκ 2 ds min E ( X ) = ∂X � 1 / 2 = 4 πβ 1 � α r = 4 πβ , β ∂r 2 π ( αr + β ∂ where r ) = 0 . For α = 0 . 01 , β = 1 , min E ( X ) ≈ 1 . 2566 . ◮ What is the influence of the radius of the estimation disk? Geometric Constraints and Variational Approaches to Image Analysis 21 Daniel Martins Antunes

  48. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Radius and grid resolution Geometric Constraints and Variational Approaches to Image Analysis 22 Daniel Martins Antunes

  49. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Radius and grid resolution Geometric Constraints and Variational Approaches to Image Analysis 22 Daniel Martins Antunes

  50. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Other experiments � � � ˆ κ 2 ( ˙ E θ ( D ) = ˆ s ( ˙ e ) α + β ˆ e ) . e ∈ ∂ h ( D ) ˙ II- 10 , α = 0 . 001 , β = 1 . Free elastica. Constrained elastica. Geometric Constraints and Variational Approaches to Image Analysis 23 Daniel Martins Antunes

  51. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Other experiments � � � ˆ κ 2 ( ˙ E θ ( D ) = s ( ˙ ˆ e ) α + β ˆ e ) . e ∈ ∂ h ( D ) ˙ II- 10 , α = 0 . 001 , β = 1 . Free elastica. Constrained elastica. ◮ What about running time? Geometric Constraints and Variational Approaches to Image Analysis 23 Daniel Martins Antunes

  52. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Running time h = 1 . 0 h = 0 . 5 h = 0 . 25 Pixels Time Pixels Time Pixels Time Triangle 521 2s (0.07s/it) 2080 43s (0.81s/it) 8315 532s(4.8s/it) Square 841 0.9s (0.09s/it) 3249 8s (0.3s/it) 12769 102s (2s/it) Flower 1641 13s (0.24s/it) 6577 209s (1.68s/it) 26321 3534s (12.3s/it) Bean 1574 7s (0.16s/it) 6278 88s (1.08s/it) 25130 1131s (6.4s/it) Ellipse 626 1s (0.14s/it) 2506 16s (0.44s/it) 10038 286s (3.1s/it) Table: Running time for the free elastica problem. Quite high running times. The geometry of the shape influences in the total running time. Geometric Constraints and Variational Approaches to Image Analysis 24 Daniel Martins Antunes

  53. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Combinatorial Elastica Conclusion ◮ Multigrid convergent estimators are suitable for elastica minimization ◮ A simple neighborhood is sufficient to escape bad local minimum. Some solutions are very close to global optimum. ◮ Too slow. It cannot be used in practice. Geometric Constraints and Variational Approaches to Image Analysis 25 Daniel Martins Antunes

  54. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References A quadratic non-submodular formulation for elastica ◮ Global formulation attempt. ◮ Fall back on a local formulation. ◮ FlipFlow algorithm. Up to 10 x faster than LocalSearch. Geometric Constraints and Variational Approaches to Image Analysis 26 Daniel Martins Antunes

  55. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Local models and completion effect The completion effect can be difficult to recover in local formulations. Geometric Constraints and Variational Approaches to Image Analysis 27 Daniel Martins Antunes

  56. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Local models and completion effect The completion effect can be difficult to recover in local formulations. Geometric Constraints and Variational Approaches to Image Analysis 27 Daniel Martins Antunes

  57. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Local models and completion effect The completion effect can be difficult to recover in local formulations. Geometric Constraints and Variational Approaches to Image Analysis 27 Daniel Martins Antunes

  58. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Local models and completion effect The completion effect can be difficult to recover in local formulations. Geometric Constraints and Variational Approaches to Image Analysis 27 Daniel Martins Antunes

  59. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Local models and completion effect The completion effect can be difficult to recover in local formulations. Let’s try a global formulation. Geometric Constraints and Variational Approaches to Image Analysis 27 Daniel Martins Antunes

  60. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Difficulties with a global formulation ◮ m pixels and n edges. Geometric Constraints and Variational Approaches to Image Analysis 28 Daniel Martins Antunes

  61. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Difficulties with a global formulation ◮ m pixels and n edges. ◮ Center of the estimation disk. � κ 2 � � y i α + β ˆ r ( D, ℓ i ) ℓ i ∈L Geometric Constraints and Variational Approaches to Image Analysis 28 Daniel Martins Antunes

  62. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Difficulties with a global formulation ◮ m pixels and n edges. ◮ Center of the estimation disk. ◮ Pixel counting and estimation of curvature squared. � �� α + 9 c 2 − 2 c A T i x + x T A i A T � � r 6 β y i i x ℓ i ∈L x ∈ { 0 , 1 } m , y ∈ { 0 , 1 } n . subject to Geometric Constraints and Variational Approaches to Image Analysis 28 Daniel Martins Antunes

  63. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Difficulties with a global formulation ◮ m pixels and n edges. ◮ Center of the estimation disk. ◮ Pixel counting and estimation of curvature squared. ◮ Linear topological constraints. � �� α + 9 c 2 − 2 c A T i x + x T A i A T � � r 6 β y i i x ℓ i ∈L x ∈ { 0 , 1 } m , y ∈ { 0 , 1 } n , T ( x , y ) . subject to Geometric Constraints and Variational Approaches to Image Analysis 28 Daniel Martins Antunes

  64. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Difficulties with a global formulation ◮ m pixels and n edges. ◮ Center of the estimation disk. ◮ Pixel counting and estimation of curvature squared. ◮ Linear topological constraints. ◮ Third order constrained non-convex binary problem. � �� α + 9 c 2 − 2 c A T i x + x T A i A T � � r 6 β y i i x ℓ i ∈L x ∈ { 0 , 1 } m , y ∈ { 0 , 1 } n , T ( x , y ) . subject to Geometric Constraints and Variational Approaches to Image Analysis 28 Daniel Martins Antunes

  65. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Difficulties with a global formulation ◮ m pixels and n edges. ◮ Center of the estimation disk. ◮ Pixel counting and estimation of curvature squared. ◮ Linear topological constraints. ◮ Third order constrained non-convex binary problem. ◮ Level 1 linearization: non semi-definite positive quadratic problem. � �� α + 9 c 2 − 2 c A T i x + x T A i A T � � r 6 β y i i x ℓ i ∈L x ∈ { 0 , 1 } m , y ∈ { 0 , 1 } n , T ( x , y ) . subject to Geometric Constraints and Variational Approaches to Image Analysis 28 Daniel Martins Antunes

  66. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Difficulties with a global formulation ◮ m pixels and n edges. ◮ Center of the estimation disk. ◮ Pixel counting and estimation of curvature squared. ◮ Linear topological constraints. ◮ Third order constrained non-convex binary problem. ◮ Level 1 linearization: non semi-definite positive quadratic problem. ◮ Level 2 linearization: O ( m 3 ) variables. � �� α + 9 c 2 − 2 c A T i x + x T A i A T � � r 6 β y i i x ℓ i ∈L x ∈ { 0 , 1 } m , y ∈ { 0 , 1 } n , T ( x , y ) . subject to Geometric Constraints and Variational Approaches to Image Analysis 28 Daniel Martins Antunes

  67. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Simplification � � πr 2 3 κ ( p ) = ˆ 2 − | B r ( p ) ∩ X | r 3 ◮ Define the optimization region (yellow) as the inner contour of the shape, denoted I . ◮ Evolve the estimation disks in the current contour. ◮ Set pixels such that the curvature estimation is reduced. Geometric Constraints and Variational Approaches to Image Analysis 29 Daniel Martins Antunes

  68. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Simplification � � πr 2 3 κ ( p ) = ˆ 2 − | B r ( p ) ∩ X | r 3 ◮ Define the optimization region (yellow) as the inner contour of the shape, denoted I . ◮ Evolve the estimation disks in the current contour. ◮ Set pixels such that the curvature estimation is reduced. Geometric Constraints and Variational Approaches to Image Analysis 29 Daniel Martins Antunes

  69. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Simplification � � πr 2 3 κ ( p ) = ˆ 2 − | B r ( p ) ∩ X | r 3 ◮ Optimization identifies zones of shortage (convex) or abundance (concave) of pixels. Geometric Constraints and Variational Approaches to Image Analysis 30 Daniel Martins Antunes

  70. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Simplification � � πr 2 3 κ ( p ) = ˆ 2 − | B r ( p ) ∩ X | r 3 ◮ Optimization identifies zones of shortage (convex) or abundance (concave) of pixels. ◮ x = 1 → Zone of shortage of pixels (convex) → Estimator disk should be shifted towards the interior → This pixel does not belong to the next contour. Geometric Constraints and Variational Approaches to Image Analysis 30 Daniel Martins Antunes

  71. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Simplification � � πr 2 3 κ ( p ) = ˆ 2 − | B r ( p ) ∩ X | r 3 ◮ Optimization identifies zones of shortage (convex) or abundance (concave) of pixels. ◮ x = 1 → Zone of shortage of pixels (convex) → Estimator disk should be shifted towards the interior → This pixel does not belong to the next contour. ◮ Therefore, we invert the optimal labeling. Geometric Constraints and Variational Approaches to Image Analysis 30 Daniel Martins Antunes

  72. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica FlipFlow X ( k ) := { x i ∈ { 0 , 1 } | p i ∈ D ⊂ Ω ⊂ Z 2 , I ( k ) } ���� Inner contour � � E flip ( D ( k ) , X ( k ) ) = κ ( p ) 2 αs ( x j ) + β ˆ θ x j ∈ X ( k ) p ∈ I ( k ) Geometric Constraints and Variational Approaches to Image Analysis 31 Daniel Martins Antunes

  73. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica FlipFlow X ( k ) := { x i ∈ { 0 , 1 } | p i ∈ D ⊂ Ω ⊂ Z 2 , I ( k ) } ���� Inner contour � � E flip ( D ( k ) , X ( k ) ) = κ ( p ) 2 αs ( x j ) + β ˆ θ x j ∈ X ( k ) p ∈ I ( k ) � = αs ( x j ) x j ∈ X ( k ) � � � � � (1 / 2 + | F ( k ) + 2 c 1 β ( p ) | − c 2 ) · x j + x j x l r p ∈ x j ∈ j<l, x j ,x l ∈ I ( k ) X ( k ) ( p ) r X ( k ) ( p ) r Geometric Constraints and Variational Approaches to Image Analysis 31 Daniel Martins Antunes

  74. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica FlipFlow X ( k ) := { x i ∈ { 0 , 1 } | p i ∈ D ⊂ Ω ⊂ Z 2 , I ( k ) } ���� Inner contour � � E flip ( D ( k ) , X ( k ) ) = κ ( p ) 2 αs ( x j ) + β ˆ θ x j ∈ X ( k ) p ∈ I ( k ) � = αs ( x j ) x j ∈ X ( k ) � � � � � (1 / 2 + | F ( k ) + 2 c 1 β ( p ) | − c 2 ) · x j + x j x l r p ∈ x j ∈ j<l, x j ,x l ∈ I ( k ) X ( k ) ( p ) r X ( k ) ( p ) r  ( x j − x i ) 2 , if q i ∈ I ( k )  � ( x j − 1) 2 , if q i ∈ F ( k ) s ( x j ) = t ( q i ) , where t ( q i ) = ( x j − 0) 2 , otherwise.  q i ∈N 4 ( p j ) Geometric Constraints and Variational Approaches to Image Analysis 31 Daniel Martins Antunes

  75. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica FlipFlow X ( k ) := { x i ∈ { 0 , 1 } | p i ∈ D ⊂ Ω ⊂ Z 2 , I ( k ) } ���� Inner contour � � E flip ( D ( k ) , 1 − X ( k ) ) = κ ( p ) 2 αs ( x j ) + β ˆ θ x j ∈ X ( k ) p ∈ I ( k ) � = αs ( x j ) x j ∈ X ( k ) � � � � � (1 / 2 + | F ( k ) + 2 c 1 β ( p ) | − c 2 ) · x j + x j x l r x j ∈ p ∈ I ( k ) j<l, x j ,x l ∈ X ( k ) ( p ) r X ( k ) ( p ) r  ( x j − x i ) 2 , if q i ∈ I ( k )  � ( x j − 0) 2 , if q i ∈ F ( k ) s ( x j ) = t ( q i ) , where t ( q i ) = ( x j − 1) 2 , otherwise.  q i ∈N 4 ( p j ) Geometric Constraints and Variational Approaches to Image Analysis 31 Daniel Martins Antunes

  76. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica FlipFlow X ( k ) := { x i ∈ { 0 , 1 } | p i ∈ D ⊂ Ω ⊂ Z 2 , I ( k ) } ���� Inner contour � � E flip ( D ( k ) , 1 − X ( k ) ) = κ ( p ) 2 αs ( x j ) + β ˆ θ x j ∈ X ( k ) p ∈ I ( k ) � = αs ( x j ) x j ∈ X ( k ) � � � � � (1 / 2 + | F ( k ) + 2 c 1 β ( p ) | − c 2 ) · x j + x j x l r x j ∈ p ∈ I ( k ) j<l, x j ,x l ∈ X ( k ) ( p ) r X ( k ) ( p ) r Shrink mode (convexities) E flip ( D ( k ) , 1 − X ( k ) ); a ( k ) ← arg min θ X ( k ) ← F ( k ) + a ( k ) . D ( k +1) Geometric Constraints and Variational Approaches to Image Analysis 31 Daniel Martins Antunes

  77. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica FlipFlow X ( k ) := { x i ∈ { 0 , 1 } | p i ∈ D ⊂ Ω ⊂ Z 2 , I ( k ) } ���� Inner contour � � E flip ( D ( k ) , 1 − X ( k ) ) = κ ( p ) 2 αs ( x j ) + β ˆ θ x j ∈ X ( k ) p ∈ I ( k ) � = αs ( x j ) x j ∈ X ( k ) � � � � � (1 / 2 + | F ( k ) + 2 c 1 β ( p ) | − c 2 ) · x j + x j x l r x j ∈ p ∈ I ( k ) j<l, x j ,x l ∈ X ( k ) ( p ) r X ( k ) ( p ) r Expansion mode (concavities) Shrink mode (convexities) ( k ) , 1 − X ( k ) ); a ( k ) E flip a ( k ) E flip ( D ( k ) , 1 − X ( k ) ); ← arg min ( D ← arg min θ θ X ( k ) X ( k ) ← F ( k ) + a ( k ) . ( k ) + a ( k ) . D ( k +1) D ( k +1) ← F Geometric Constraints and Variational Approaches to Image Analysis 31 Daniel Martins Antunes

  78. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica FlipFlow r = 3 r = 5 Geometric Constraints and Variational Approaches to Image Analysis 32 Daniel Martins Antunes

  79. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Evaluation on farther rings Geometric Constraints and Variational Approaches to Image Analysis 33 Daniel Martins Antunes

  80. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Evaluation on farther rings � � E flip ( θ ,m ) ( D ( k ) , 1 − X ( k ) ) = κ ( p ) 2 αs ( x j ) + β ˆ x j ∈ X ( k ) p ∈ R m ( D ( k ) ) � = αs ( x j ) x j ∈ X ( k ) � � � � � (1 / 2 + | F ( k ) + 2 c 1 β ( p ) | − c 2 ) · x j + x j x l r p ∈ x j ∈ j<l, R m ( D ( k ) ) x j ,x l ∈ X ( k ) ( p ) r X ( k ) ( p ) r R m ( D ) := { p | m − 1 < d D ( p ) ≤ m } ∪ { p | − m + 1 > d D ( p ) ≥ − m } Geometric Constraints and Variational Approaches to Image Analysis 33 Daniel Martins Antunes

  81. Motivation Digital sets and convergent estimators Combinatorial Elastica Non-submodular elastica Elastica minimization via graph-cuts Conclusion References Non-submodular elastica Evaluation on farther rings r = 5 m = 1 m = 3 m = 4 m = 5 Geometric Constraints and Variational Approaches to Image Analysis 34 Daniel Martins Antunes

Recommend


More recommend