The Shannon Total Variation Rmy Abergel, joint work with Lionel - PowerPoint PPT Presentation

The Shannon Total Variation Rémy Abergel, joint work with Lionel Moisan. CNRS, MAP5 Laboratory, Paris Descartes University. Conference on variational methods and optimization in imaging, The Mathematics of Imaging, IHP , February 6, 2019. Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 1 / 34

The total variation Given an image U : Ω c ⊂ R 2 → R that belongs to the Sobolev space W 1 , 1 (Ω c ) , we note � TV ( U ) = | D U ( x , y ) | dx dy . Ω c This definition can be extended to the space BV (Ω c ) of functions with bounded variation that are non-necessarily differentiable. Definition (discrete total variation) The total variation of a discrete image u : Ω ⊂ Z 2 → R is generally defined by � TV d ( u ) = |∇ u ( x , y ) | , ( x , y ) ∈ Ω where ∇ denotes a finite-differences scheme. Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 2 / 34

The total variation Given an image U : Ω c ⊂ R 2 → R that belongs to the Sobolev space W 1 , 1 (Ω c ) , we note � TV ( U ) = | D U ( x , y ) | dx dy . Ω c This definition can be extended to the space BV (Ω c ) of functions with bounded variation that are non-necessarily differentiable. Definition (discrete total variation) The total variation of a discrete image u : Ω ⊂ Z 2 → R is generally defined by � TV d ( u ) = |∇ u ( x , y ) | , ( x , y ) ∈ Ω where ∇ denotes a finite-differences scheme. Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 2 / 34

The total variation The use of the total variation in image processing has become popular with the work of Rudin, Osher et Fatemi 1 (ROF), who proposed an image denoising model based on the minimization of the energy ∀ u ∈ R Ω , E ROF ( u ) = � u − u 0 � 2 + λ TV d ( u ) , 2 � �� � � �� � regularity data fidelity (promoting sparsity) where u 0 represents the noisy image and λ ∈ R + a regularity parameter that must be set by the user. 1 L. I. Rudin, S. Osher, and E. Fatemi . “Nonlinear total variation based noise removal algorithms”. Physica D: Nonlinear Phenomena , 1992. Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 3 / 34

Interpolation of an image denoised using TV d We denoise an image u 0 by computing the minimizer of the ROF energy: (a) input image u 0 Shannon zooming of (a) spectrum of (a) (b) denoised image Shannon zooming of (b) spectrum of (b) Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 4 / 34

Interpolation of an image denoised using TV d We denoise an image u 0 by computing the minimizer of the ROF energy: (a) input image u 0 Bicubic zooming of (a) spectrum of (a) (b) denoised image Bicubic zooming of (b) spectrum of (b) Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 5 / 34

Shannon Sampling Theorem This Theorem states that a band-limited function can be recovered exactly from an infinite set of samples. Theorem (Shannon Sampling Theorem) Let U : R d → R be an absolutely integrable function whose Fourier transform � R d U ( x ) e − i � ξ, x � dx , � ∀ ξ ∈ R d , U ( ξ ) = satisfies � U ( ξ ) = 0 si ξ �∈ [ − π, π ] d . Then, U is uniquely determined by its values on Z d since we have � ∀ x ∈ R d , U ( x ) = U ( k ) sinc ( x − k ) , k ∈ Z d � d sin ( π x j ) and sin ( 0 ) where we have set sinc ( x 1 , . . . , x d ) = = 1. π x j 0 j = 1 Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 6 / 34

Shannon interpolation of a discrete image Definition (discrete Shannon interpolation (2D)) Given a discrete domain Ω = { 0 , . . . , M − 1 } × { 0 , . . . , N − 1 } , and an image u : Ω → R , we call discrete Shannon interpolation of u the ( M , N ) -periodical function U : R 2 → R defined by � ∀ ( x , y ) ∈ R 2 , U ( x , y ) = u ( k , ℓ ) sincd M ( x − k ) sincd N ( y − ℓ ) , ( k ,ℓ ) ∈ Ω where  sin ( π x )   if M is odd,  M sin ( π x  M ) sincd M ( x ) =   sin ( π x )   if M is even. M tan ( π x M ) Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 7 / 34

Shannon interpolation of a discrete image We can show that the Shannon interpolate of a discrete image can be evaluated in the Fourier domain. Proposition The Shannon interpolate of a discrete image u : Ω → R satisfies � α x � � M + β y 1 2 i π ε M ( α ) ε N ( β ) � N U ( x , y ) = u ( α, β ) e , MN α,β ∈ Z − M 2 ≤ α ≤ M 2 − N 2 ≤ β ≤ N 2 where ε M et ε N are given 2 by � � 1 si | α | < M / 2 1 si | β | < N / 2 ε M ( α ) = ε N ( β ) = 1 / 2 si | α | = M / 2 1 / 2 si | β | = N / 2 . This interpolation formula is useful to apply precise subpixellic geometric transforms (rotations, translations, zoom) to discrete images. 2 R. Abergel and L. Moisan : “The Shannon Total Variation”, Journal of Mathematical Imaging and Vision, 2017. Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 8 / 34

Shannon interpolation of a discrete image We can show that the Shannon interpolate of a discrete image can be evaluated in the Fourier domain. Proposition The Shannon interpolate of a discrete image u : Ω → R satisfies � α x � � M + β y 1 2 i π ε M ( α ) ε N ( β ) � N U ( x , y ) = u ( α, β ) e , MN α,β ∈ Z − M 2 ≤ α ≤ M 2 − N 2 ≤ β ≤ N 2 where ε M et ε N are given 2 by � � 1 si | α | < M / 2 1 si | β | < N / 2 ε M ( α ) = ε N ( β ) = 1 / 2 si | α | = M / 2 1 / 2 si | β | = N / 2 . This interpolation formula is useful to apply precise subpixellic geometric transforms (rotations, translations, zoom) to discrete images. 2 R. Abergel and L. Moisan : “The Shannon Total Variation”, Journal of Mathematical Imaging and Vision, 2017. Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 8 / 34

The Shannon total variation We call Shannon total variation (STV) of the discrete image u the continuous totale variation of U . Definition ( STV ∞ ) � STV ∞ ( u ) := TV ( U ) = | D U ( x , y ) | dxdy . [ 0 , M ] × [ 0 , N ] For practical implementations, we can estimage STV ∞ ( u ) using a Riemann sum (involving an oversampling factor n = 2 or 3 for the integration domain). Definition ( STV n ) For any integer n ≥ 1, set � � � � k �� STV n ( u ) = 1 � = 1 � D U n , ℓ | D n u ( k , ℓ ) | , n n 2 n 2 ( k ,ℓ ) ∈ Ω n ( k ,ℓ ) ∈ Ω n � k � n , ℓ where D n u ( k , ℓ )= D U , and Ω n = { 0 , ... , nM − 1 }×{ 0 , ... , nN − 1 } . n STV n and TV d share the same structure since TV d ( u ) = � ( k ,ℓ ) ∈ Ω |∇ u ( k , ℓ ) | Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 9 / 34

The Shannon total variation We call Shannon total variation (STV) of the discrete image u the continuous totale variation of U . Definition ( STV ∞ ) � STV ∞ ( u ) := TV ( U ) = | D U ( x , y ) | dxdy . [ 0 , M ] × [ 0 , N ] For practical implementations, we can estimage STV ∞ ( u ) using a Riemann sum (involving an oversampling factor n = 2 or 3 for the integration domain). Definition ( STV n ) For any integer n ≥ 1, set � � � � k �� STV n ( u ) = 1 � = 1 � D U n , ℓ | D n u ( k , ℓ ) | , n n 2 n 2 ( k ,ℓ ) ∈ Ω n ( k ,ℓ ) ∈ Ω n � k � n , ℓ where D n u ( k , ℓ )= D U , and Ω n = { 0 , ... , nM − 1 }×{ 0 , ... , nN − 1 } . n STV n and TV d share the same structure since TV d ( u ) = � ( k ,ℓ ) ∈ Ω |∇ u ( k , ℓ ) | Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 9 / 34

The Shannon total variation We call Shannon total variation (STV) of the discrete image u the continuous totale variation of U . Definition ( STV ∞ ) � STV ∞ ( u ) := TV ( U ) = | D U ( x , y ) | dxdy . [ 0 , M ] × [ 0 , N ] For practical implementations, we can estimage STV ∞ ( u ) using a Riemann sum (involving an oversampling factor n = 2 or 3 for the integration domain). Definition ( STV n ) For any integer n ≥ 1, set � � � � k �� STV n ( u ) = 1 � = 1 � D U n , ℓ | D n u ( k , ℓ ) | , n n 2 n 2 ( k ,ℓ ) ∈ Ω n ( k ,ℓ ) ∈ Ω n � k � n , ℓ where D n u ( k , ℓ )= D U , and Ω n = { 0 , ... , nM − 1 }×{ 0 , ... , nN − 1 } . n STV n and TV d share the same structure since TV d ( u ) = � ( k ,ℓ ) ∈ Ω |∇ u ( k , ℓ ) | Rémy Abergel, conference on variational methods and optimization in imaging, IHP , February 6, 2019. 9 / 34