Regularization Theory Nicolas Rougon Institut Mines-Tlcom / Tlcom - PowerPoint PPT Presentation

Direct & Inverse problems Restoring well-posedness Regularization Regularization Theory Nicolas Rougon Institut Mines-Télécom / Télécom SudParis ARTEMIS Department; CNRS UMR 8145 nicolas.rougon@telecom-sudparis.eu May 10, 2020 Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Restoring well-posedness Regularization Course overview ◮ Basic problems in low-level image analysis are hard to solve ◮ The encountered difficulties have common nature and origins, linked to the key notion of ill-posed inverse problem ◮ Generic mathematical approaches for fixing these difficulties have been developed in the deterministic and stochastic frameworks. They are known as regularization techniques ◮ In the next courses, deterministic regularization will be applied to 3 basic image analysis problems Segmentation Denoising Motion estimation Active contours Anisotropic Optical flow Mumford-Shah diffusion Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging Outline Direct & Inverse problems 1 Restoring well-posedness 2 Regularization 3 Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging An example - Surface reconstruction 3D object Problem statement Given a 3D point set, estimate the geometry of the surface ( i.e. the shape) of the underlying 3D object ? 3D sensor ◮ Solving for an interpolation problem over R 3 3D point set Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging An example - Surface reconstruction ◮ A hard problem Topology is lost Geometry is lost → connectivity? homotopy? → metrics? orientation? ր under-constrained problem ⇒ multiple solutions ց ◮ Solution: enforcing prior constraints Topological constraints Geometric constraints connectivity continuity (smoothness) homotopy support (global/patch) boundary (open/close) subspace Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging An example - Surface reconstruction ◮ A hard problem (cont’d) Discretization and noise generate ambiguities Assume subspace constraints ( e.g. circle shape space) sampling reconstruction over-constrained problem ⇒ solutions may not exist noise-sensitive problem ⇒ solutions may vary radically with noise ◮ Solution: defining an approximated problem Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging Direct vs. inverse problems x ◮ Imaging system model H : X → Y continuous (linear | nonlinear) operator H H X , Y Hilbert spaces norm � · � X , dot product < · , · > X y x ◮ Direct problem data Given an input x ∈ X , generate y ∈ Y H such that y = H x solutions → Image synthesis y x ◮ Inverse problem solutions Given an output y ∈ Y , estimate x ∈ X H such that y = H x data → Image analysis y Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging Well-posed vs. ill-posed problems ◮ A (direct/inverse) problem is well-posed iff the following so-called Hadamard conditions hold for any data y ∈ Y , there exists a solution x ∈ X the solution x ∈ X is unique the solution x ∈ X depends continuously on data y ∈ Y X Y x + x δ x y + δ y y ◮ A problem is ill-posed iff. at least one Hadamard condition fails Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging Direct problems ◮ Since H is continuous, classical direct problems in mathematics, physics and engineering are well-posed Physical simulation problems Often defined by basic partial differential equations (PDE) with proper boundary conditions (BC) propagation → elliptic PDE + Dirichlet BC e.g. wave equation diffusion → parabolic PDE + Neumann BC e.g. heat equation transport → hyperbolic PDE + Cauchy BC e.g. eikonal (Burger) equation Image synthesis problems Most models are physically-inspired Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging Inverse problems in image analysis ◮ Inverse problems in image science are usually ill-posed Segmentation | Surface reconstruction | Shape from X Edge detection H is an integral operator � η ( Hx )( η ) = x ( ξ ) d ξ −∞ η : pixel location 400 200 Surface reconstruction 0 0 10 H is a projector onto a local basis 20 30 ( Hx )( η i ) = < x , ϕ i > X 40 50 η i : surface local coordinates 60 70 80 70 60 50 40 30 80 20 10 0 → elliptic PDE Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging Inverse problems in image analysis ◮ Inverse problems in image science are usually ill-posed Matching H is a warping operator ( Hx )( η ) = ( x ◦ φ )( η ) η : pixel location Motion estimation φ = I d + v where v : optical flow Stereovision φ = I d + d where d : disparity η 1 η 2 → Optimal isomorphism Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging Inverse problems in image analysis ◮ Inverse problems in image science are usually ill-posed Deconvolution H is a convolution operator ( Hx )( η ) = ( K ⋆ x )( η ) η : pixel location Deblurring Linear scale-space K : Gaussian kernel → Inverse diffusion Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging Inverse problems in image analysis ◮ Inverse problems in image science are usually ill-posed Transmission tomography reconstruction H is a Radon transform D η � ( Hx )( η ) = x ( l ) dl D η η η : projection angle Computerized Tomography (CT) → Inverse Radon transform Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Well-posedness issues Regularization Direct & Inverse problems in digital imaging Inverse problems in image analysis ◮ Inverse problems in image science are usually ill-posed Emission tomography reconstruction Positon Emission Tomography (PET) Single Photon Emission Computerized Tomography (SPECT) → Inverse Fredholm integral Super-resolution Satellite / Aerial / Astromonical imaging → Interpolation from multiple data Phase unwarping Synthetic Aperture Radar (SAR) Digital Holographic Microscopy (DHM) → Analytic continuation ... Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Pseudo-inverse Regularization Generalized inverses Outline Direct & Inverse problems 1 Restoring well-posedness 2 Regularization 3 Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Pseudo-inverse Regularization Generalized inverses Problem statement Inverse problem (P) Given the data y ∈ Y , estimate x ∈ X such that y = Hx X Y The invertibility of H depends on y x+x 0 x 0 its kernel Ker ( H ) ⊆ X x 0 (closed set) 0 Sp( H ) Ker( H ) its span Sp ( H ) ⊆ Y y’ ◮ The problem (P) is well-posed iff. H is injective (Ker ( H ) = { 0 } ) and onto (Sp ( H ) = Y ). In this case H − 1 exists and is continuous The solution of (P) is x = H − 1 y Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Pseudo-inverse Regularization Generalized inverses Restoring well-posedness X Y Assumptions y x+x 0 x H is not injective 0 x 0 Ker ( H ) � = { 0 } 0 Sp( H ) Ker( H ) H is not onto y’ Sp ( H ) � = Y ◮ How to restore the existence and unicity properties? Key idea Redefine the problem (P) so that Ker ( H ) ∩ X = { 0 } Sp ( H ) = Y Nicolas Rougon IMA4509 - Regularization theory

Direct & Inverse problems Problem statement Restoring well-posedness Pseudo-inverse Regularization Generalized inverses Restoring well-posedness Track #1 Redefine the data space Y and the solution space X X Y y 2 Assumptions y 1 H is not injective 0 Ker ( H ) � = { 0 } 0 Sp( H ) Ker( H ) Sp ( H ) is closed Y = Sp ( H ) ⊕ Sp ⊥ ( H ) ◮ The problem (P) is well-posed over X = Ker ⊥ ( H ) Y = Sp ( H ) Nicolas Rougon IMA4509 - Regularization theory