Image Denoising Using Mean Curvature of Image Surface Tony Chan - PowerPoint PPT Presentation

Image Denoising Using Mean Curvature of Image Surface Tony Chan (HKUST) Joint work with Wei ZHU (U. Alabama) & Xue-Cheng TAI (U. Bergen) In honor of Bob Plemmons’ 75 birthday CUHK Nov 18, 2013 1

How I Got to Know Bob Plemmons • Berman-Plemmons (mid 70’s?) • Stanford Serra House (late 70’s?) • SIAM Conferences • Collaboration with Curt Vogel (90’s) • HK (2010’s)! 2

Plemmons 60, Jan 1999, WFU 3

Near Bozeman,Montana (with Curt Vogel) 4

Plemmons Family Reunion, Asheville, NC? 1995 5

Trip to Grass Island Aug 2010 Hike to Lei Yue Mun Nov 2011 6

Outline • Problem • Related Work • Our Model • Fast Algorithm Using Augmented Lagrangian Method • Numerical Experiments • Summary and Future Work 7

Our related publications: • Zhu and Chan, Image denoising using mean curvature of image surface, SIIMS 2012. • Zhu, Tai and Chan, Augmented Lagrangian method for a mean curvature based image denoising model, Inverse Probl Imag, In Press , 2013. • Zhu, Tai and Chan, Image Segmentation Using Euler’s Elastica as the Regularization, J. Scientific Computing, 2013. • Zhu, Tai and Chan, A fast algorithm for a mean curvature based image denoising model using augmented Lagrangian method, To appear in LNCS 2014, in “Efficient Algorithms for Global Optimisation Problems in Computer Vision”. 8

Typical Methods of Image Denoising • Variational method, PDE-based method, statistical method and many other ones • Variational method = u + f n Given image Desired clean image Noise → 1 f : Ω R How to decompose the given noisy image using appropriate regularizers? 9

Classical Variational Models • Mumford-Shah (89) ∫ ∫ 2 = − + ∇ + 2 1 E(u, K) (f u) λ u μH (K) Ω Ω \ K goal boundary positive parameters goal true image • Rudin-Osher-Fatemi (92) ∫ ∫ = ∇ + − > 2 E(u) λ u (f u) , λ 0 Ω Ω – Powerful & popular, excellent analytical properties – Preserve edges and sweep noise very efficiently – Cannot preserve corner & image contrast – Suffers from the staircase effect 10

Related high-order models for image denoising • Euler’s Elastica: C-Kang-Shen (2002), Ambrosio-Masnou-Morel (2003)   2   ∇ u 1 ∫   ∫   = + ∇ ⋅ ∇ + − 2 E ( u ) a b u ( f u )   ∇   u 2 Ω Ω     – Originally proposed for the disocclusion problem – Noise removal efficiently, no staircase effect – Need to solve a fourth-order PDE • Lysaker-Lundervold-Tai (LLT)(2003) 1 ∫ ∫ λ = λ + + + + − 2 2 2 2 2 L ( u , ) u u u u ( f u ) xx xy yx yy 2 Ω Ω – Excellent noise suppression, no staircase effect – Need to solve a fourth-order PDE 11

Mean curvature of image surface • Give an image : → ⊂ 1 2 f : Ω R , Ω R • Consider the function : = − ∈ Φ(x, z) z f(x), x Ω (x, f(x)) Its zero level set corresponds to the image surface , whose mean curvature reads:     ( )   ∇ Φ ∇ − ∇   1 1 f, 1 1 f     ∇ ⋅ (x, z) = ∇ ⋅ = ∇ ⋅ = x x H   ( )     (x, z) (x, z) x f ∇ − ∇ 2 2 f, 1 2 Φ   2   + ∇ 1 f   x (x, z)   x 12

Our Model ( Zhu, Chan SIIMS 2012 ) • Energy: 1 2 ∫ ∫ = + − E(u) λ | H | (f u) u Ω Ω 2    ∇  λ u 1 2 ∫ ∫ = ∇ ⋅ + − | | (f u)   Ω Ω 2 2  2  + ∇ 1 | u |   • Gradient Descent Equation:   ∂ u 1   ' = − ∇ ⋅ − ∇ + − λ (I P) (Φ (H )) (f u) u   ∂ t 2 + ∇ 1 u     2 2 → I, P : R R The two operators are defined as   ∇ ∇   u u     = = ⋅ = I( ν ) ν , P( ν ) ν , Φ(x) | x |     2 2 + ∇ + ∇ 1 u 1 u   ∂ u 2 ∇ << ≈ − + − | u | 1, λΔ u (f u) • If , the bi-harmonic equation, explaining why ∂ t small oscillation part can be removed effectively. 13

Our model preserves contrast with small regularization  We can prove that: ∫ = f = | H | P(E, Ω) 2 C hχ E If is an open set with boundary, and , then , f E Ω E Ω the perimeter of set inside the domain (independent of h).  These results suggest that the proposed model is able to preserve image contrasts, as the regularizer doesn ’ t rely on the height of signal. • Property of our model (contrast preservation): = χ f h Ω = × ( - 2 R , 2R ) (-2R, 2R) Let be an image defined on . Define B(0, R) = ∈ = + 2 2 2 2 S { u C (R ) : u(x, y) g( x y ), g takes the same type of profile as shown. } , then C > λ < 0 C there exists a constant , such that if , then the following holds: = ∈ E(f) inf{E(u) : u S} λ f This property shows that the model attains a minimum at if is small f enough, i.e. the model restores exactly and thus preserves contrast. 14

Corner Preservation = χ f h Ω = × ( -R , R ) (-R, R) Let be an image defined on . Define × (0, R) (0, R) = = Q { u : the surface of z u(x, y) is obtained by rotating the generatrix along the orbit. } , then C > λ < 0 C there exists a constant , such that if , then the following holds = ∈ E(f) inf{E(u) : u Q} For small enough regularization (e.g. low noise level), our model can preserve corners. Summary of our model: •Using L1 norm of mean curvature of image surface as regularization •Regularization does not penalize contrast or discontinuties •For small regularization, can preserve contrast, edges and corners. •Complete theory still lacking 15

Augmented Lagrangian Method • Related functionals ∫ ∫ = ∇ + − 2 E(u) λ u (f u) • non-differentiable Ω Ω • nonlinear   2   ∇ u 1 ∫   ∫   = + ∇ ⋅ ∇ + − 2 • high order E ( u ) a b u ( f u )   ∇   u 2 Ω Ω • non-differentiable     • nonlinear • Augmented Lagrangian method (ALM) has been successfully applied to the minimization of the above functionals by Tai et al. ( SIIMS 2010 & 2011 )  convert the original minimization of those functionals to be constrained optimization problems  search for saddle points of the resulting problem by solving several associated subproblems • Key of ALM: whether the subproblems can be solved efficiently 16

Review of ALM for Euler’s Elastica Denoising ( Tai,Hahn,Chung, SIIMS ,2011 ) • Tai et al. applied ALM to minimize the following functional for image denoising through minimization of Euler’s Elastica   2   ∇ u 1 ∫   ∫   = + ∇ ⋅ ∇ + − 2 E ( u ) a b u ( f u )   ∇   u 2 Ω Ω     • Introducing new variables for the gradient and the unit normal vector • The problem can be casted as a constrained minimization problem with new variables • The last constraint is difficult to handle. Needed a new idea. 17 17

A new constraint • In ( Tai et al SIIMS11) • A new constraint problem is to solve: • The minimization variables are: u, p, n. When two of them are fixed and we just need to minimize with one of them, the problem is convex 18 18

Fast Augmented Lagrangian • Augmented Lagrangian method (ALM) has been used to solve: Features of the ALM in Tai et al SIIMS11: • ALM with L2 penalization is used to handle: • ALM with L1 penelization is used to handle: • All the subproblems either has closed form solutions or can be solved by fast solvers like FFT. • Need few iterations (total). Around 100-200. Makes this algorithm very fast. 19 19

Mean curvature minization ( Zhu, Tai, Chan IPI 2013 ) • How to obtain fast algorithm to minimize:   ∇ u 1   ∫ ∫ = λ ∇ ⋅ + − 2 E(u) (f u)   2 + ∇ 2 Ω 1 | u | Ω   • Can introduce new variables and consider: 1 + ∫ ∫ λ − 2 min q ( f u ) u , p , q , n 2 Ω Ω 2 = ∇ = ∇ + ∇ = ∇ ⋅ subject to p u , n u / 1 u , q n • It is very difficult to handle: . 20

A new constraint • We introduce the following new variables = ∇ = ∇ ∇ = ∇ ⋅ p u , 1 , n u , 1 / u , 1 , q n • The original minimization problem is reformulated as 1 2 ∫ ∫ λ + − min q ( f u ) u , p , q , n Ω Ω 2 = ∇ ⋅ = = = ∇ subject to q n , n , n n , n , n p / p , p u , 1 1 2 1 2 3 • Same idea: the following two are equivalent: • All ALM subproblems can be solved using FFT or thresholding 21