Image Denoising Using Two-stage Non-local Means Enming Luo - PowerPoint PPT Presentation

Image Denoising Using Two-stage Non-local Means Enming Luo advisor: Truong Nguyen March 15, 2012

Outline 2 Introduction 1 Non-local Means 2 Two-stage Non-local Means 3 Simulation Results 4 Conclusion and Future Work 5

Introduction 3 A noisy image is modelled as Z = U + V U : clean image V : noise Z : noisy image Z i : ith pixel Z i : patch centered at the ith pixel Most denoising papers assume: V ∼ N (0 , σ 2 I ) Denoising process: Z → ˆ Z ≈ U

Introduction-short review 4 Local methods: small search window Bilateral filter Directional filters Non-local methods: large search window NLMeans (Non-local Means)-2005 BM3D (Block Matching and 3D Filtering)-2007 LPG-PCA (Local Pixel Grouping-Principal Component Analysis)-2010

Introduction 1 Non-local Means 2 Two-stage Non-local Means 3 Simulation Results 4 Conclusion and Future Work 5

NLMeans filtering 6 Weighted averaging: ˆ � Z i = W ij Z j j ∈ N i NLMeans: find enough similar pixels and assign them large weights. enough: search window similar: similarity measure weight: weight calculation

NLMeans - search window 7 there is no reason that similar pixels should be close not restricted to a local neighbourhood

NLMeans - similarity measure 8 similarity decreases in distance. An accurate way: squared Euclidean distance ( U i − U j ) 2 . We only have the noisy data. ( Z i − Z j ) 2 ≈ ( U i − U j ) 2 (not accurate) E [( Z i − Z j ) 2 ] = E [( U i − U j + V i − V j ) 2 ] = ( U i − U j ) 2 + 2 σ 2 ( Z i − Z j ) 2 − 2 σ 2 ≈ ( U i − U j ) 2 We could use patch-based distance instead of pixel-based distance. Similar patches most likely have similar center pixels. d d ( Z i ( k ) − Z j ( k )) 2 − 2 dσ 2 ≈ Use � � ( U i ( k ) − U j ( k )) 2 k =1 k =1 d is the number of pixels in the patch

NLMeans - weight calculation 9 weight is an increasing function of similarity, or decreasing function of distance. Exponential kernel e − x 2 h 2 is used to compute the weight. x is the Euclidean distance h is the decay parameter Specifically in NLMeans � k =1 ( Z i ( k ) − Z j ( k )) 2 − 2 dσ 2 � � d W ij = exp − dσ 2 T 2 d ( Z i ( k ) − Z j ( k )) 2 − 2 dσ 2 is the approximation of squared � k =1 Euclidean distance dσ 2 is for normalization, T is the decay parameter

NLMeans - performance 10 NLMeans: Good: remove noises but keep details (edges, textures).

Introduction 1 Non-local Means 2 Two-stage Non-local Means 3 Simulation Results 4 Conclusion and Future Work 5

Iterative NLMeans 12 Can we apply NLMeans again to the denoised image? NLMeans assume IID noises: Z i : IID noise , ˆ � Z i = W ij Z j : non-IID noise j ∈ N i Non-identical variance: V ar ( ˆ W 2 ij ) σ 2 Z i ) = ( � j ∈ N i Z i and ˆ ˆ j Z j become dependent. i Answer: No (not directly)

Modification of NLMeans 13 NLMeans weight(patch size 1x1): − ( Z i − Z j ) 2 − 2 σ 2 � � W ij = exp σ 2 T 2 E [( Z i − Z j ) 2 ] − V ar ( Z i − Z j ) = E [ Z i − Z j ] 2 ( Z i − Z j ) 2 − V ar ( Z i − Z j ) ≈ E [ Z i − Z j ] 2 = ( U i − U j ) 2 Replace 2 σ 2 with V ar ( Z i − Z j ) : − ( Z i − Z j ) 2 − V ar ( Z i − Z j ) � � W ij = exp V ar ( Z i − Z j ) T 2 2 ( Z i − Z j ) 2   V ar ( Z i − Z j ) − 1 = exp  − 1  2 T 2

Generalized NLMeans 14 Extending to a patch, we would get the new weight:   d � ( Z i ( k ) − Z j ( k )) 2 � � V ar ( Z i ( k ) − Z j ( k )) − 1   k =1 W G  − ij = exp   1 2 dT 2    Generalized: noises in Z i ( k ) and Z j ( k ) not necessarily independent or identical. For IID noise, W G ij ≡ W ij

Iterative denoising 15 Back to original problem: Can we apply NLMeans again on the denoised image? NLMeans has: ˆ � Z i = W ij Z j j ∈ N i To further denoise ˆ Z : � W ij ˆ ˜ Z i = Z j × j ∈ N i ˜ � ij ˆ W G Z i = Z j j ∈ N i

Variance calculation 16   d ( ˆ Z i ( k ) − ˆ � Z j ( k )) 2 � � Z j ( k )) − 1 V ar ( ˆ Z i ( k ) − ˆ   k =1 W G ij = exp  −   1 2 dT 2    New problem: how to compute V ar ( ˆ Z i ( k ) − ˆ Z j ( k )) ? For better explanation, again assume the patch is 1x1 and Z i − ˆ ˆ Z j = ˆ Z i − ˆ Z j

Variance calculation 17 Z i and ˆ ˆ Z j are from NLMeans: ˆ � � Z i = W iℓ Z ℓ = W iℓ ( U ℓ + V ℓ ) ℓ ∈ N i ℓ ∈ N i ˆ � � Z j = W jℓ Z ℓ = W jℓ ( U ℓ + V ℓ ) ℓ ∈ N j ℓ ∈ N i Thus we have � � � V ar ( ˆ Z i − ˆ � W iℓ V ℓ − Z j ) = V ar W jℓ V ℓ ℓ ∈ N i ℓ ∈ N j

Variance calculation - no overlapping 18 � � � V ar ( ˆ Z i − ˆ Z j ) = V ar ℓ ∈ N i W iℓ V ℓ − � ℓ ∈ N j W jℓ V ℓ j N j N i i � � � V ar ( ˆ Z i − ˆ W 2 � W 2 σ 2 Z j ) = iℓ + jℓ ℓ ∈ N i ℓ ∈ N j

Variance calculation - overlapped 19 � � � V ar ( ˆ Z i − ˆ Z j ) = V ar ℓ ∈ N i W iℓ V ℓ − � ℓ ∈ N j W jℓ V ℓ j N j N i i � � � � � W iℓ V ℓ − W jℓ V ℓ = W iℓ V ℓ − W jℓ V ℓ + ( W iℓ − W jℓ ) V ℓ ℓ ∈ N i ℓ ∈ N j ℓ ∈ N i ∩ N j ℓ ∈ N i \ N j ℓ ∈ N j \ N i � � � V ar ( ˆ Z i − ˆ � � W 2 W 2 σ 2 Z j ) = iℓ + jℓ − 2 W iℓ W jℓ ℓ ∈ N i ℓ ∈ N j ℓ ∈ N i ∩ N j

Algorithm 20 center:to be denoised, window size:5x5 weight depends on the patch distance. pixel-wise variance is required. dependence due to overlapping

Introduction 1 Non-local Means 2 Two-stage Non-local Means 3 Simulation Results 4 Conclusion and Future Work 5

Simulation setup 22 Search window size small: can’t provide enough similar pixels large: bring more bias Patch size small: not robust to noise large: can’t find enough similar pixels Decay parameter T It controls trade-off between bias and variance These parameters are chosen empirically in NLMeans and we did the same way. for example, if the noise variance is 10 2 , search window size is 21x21, patch size is 3x3 and T is 0.4

Simulation setup 23 Methods for comparison NLMeans BM3D LPG-PCA Test images Standard test images like House, Cameraman, Monarch, Peppers. Noise added Noises with std being 10,20 and 30 are pre-generated and then added in order to have a fair comparison between different methods. Comparison metric PSNR SSIM

Simulation results - PSNR 24

Simulation results - PSNR 25

Visual Quality 26

Introduction 1 Non-local Means 2 Two-stage Non-local Means 3 Simulation Results 4 Conclusion and Future Work 5

Conclusion 28 X ? s n a e M L N NLMeans G - N L M e a n s

Future work 29 Improve weight calculation Different kernel functions, different distances. Adaptive search window size, patch size and decay parameter. Extend to denoising real images Estimate the noise variance for each pixel. Extend to stereo or multi-view denoising More views will help the denoising.

Reference 30 T., M., ”Bilateral fitering for gray and color images,” 1998 B., C., M., ”A review of image denoising algorithms, with a new one,” 2005 Z.,D.,Z.,S.,”Two-stage image denoising by principal component analysis with local pixel grouping,” 2010 D., F., K., E.,”Image denoising by sparse 3d transform-domain collaborative filtering,” 2007 B., C., M., ”Self-similarity-based image denoising,” 2011

Thank you!