1 Image Denoising and Enhancement Karen Egiazarian (TUT , NI) Department of Signal Processing
2 Image denoising: motivating example • Images are inevitably corrupted by various degradations and particularly by noise. • Megapixels race: Pixels are getting smaller, and images even noisier image noise denoised image _ = Canon Powershot A590IS ISO 800 Department of Signal Processing
3 Imaging Sensors: Exposure-time/noise trade-off Digital imaging sensors can have very different performance Different acquisition settings result in different noise levels in the image “Exposure-time/noise trade-off “ Department of Signal Processing
4 Outline • Intro • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing
5 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing
6 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing
7 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations equipped with ICI rule (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing
8 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations equipped with ICI rule (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing
9 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing
10 Outline • Signal-dependent noise modeling and removal for digital imaging sensors • Local polynomial approximations (LPA-ICI) • Advanced image processing techniques: - shape-adaptive methods - nonlocal transform-based methods • Applications: - denoising - deblurring - deblocking - super-resolution/zooming Department of Signal Processing
11 Intro Department of Signal Processing
12 Intro Department of Signal Processing
13 Intro Department of Signal Processing
14 Intro Department of Signal Processing
15 Intro Department of Signal Processing
16 Intro Department of Signal Processing Karen Egiazarian
17 Intro Department of Signal Processing Karen Egiazarian
18 Intro Department of Signal Processing Karen Egiazarian
19 Intro Department of Signal Processing Karen Egiazarian
20 Intro Department of Signal Processing Karen Egiazarian
21 Intro Department of Signal Processing Karen Egiazarian
22 Intro Department of Signal Processing Karen Egiazarian
23 Statistical analysis of raw data Department of Signal Processing
24 Statistical analysis of raw data Department of Signal Processing 8.4.2016
25 Statistical analysis of raw data Department of Signal Processing
26 Statistical analysis of raw data Department of Signal Processing
27 Statistical analysis of raw data Department of Signal Processing
28 Statistical analysis of raw data Department of Signal Processing
29 Statistical analysis of raw data Department of Signal Processing
30 Statistical analysis of raw data The analysis of experimental data demonstrates that: 1. The model of noise is close to the Poissonian one 2. Model parameters depend neither on the color channel nor on the exposure time Department of Signal Processing
31 Parametric signal-dependent noise-modelling: Poissonian-Gaussian with clipping Department of Signal Processing
Parametric signal-dependent noise-modelling: 32 automatic estimation from single-image raw- data ( http://www.cs.tut.fi/~foi/sensornoise.html ) Department of Signal Processing
33 Practical modeling for raw data: idea • Model photon-to-electron conversion using Poisson distributions (signal dependent); • Model the other noise sources as signal-independent and Gaussian (central- limit theorem); • Exploit normal approximation of Poisson distributions; • The acquisition/dynamic range is limited: too dark or too bright signals are clipped; • There can be a pedestal; • Spatial dependencies can be ignored for normal operating conditions (go for independent noise). Eventually, only two parameters are sufficient to describe the noise model where the raw data is described as clipped signal-dependent observations. Department of Signal Processing
34 Variance stabilization Department of Signal Processing Karen Egiazarian
35 Variance stabilization Department of Signal Processing Karen Egiazarian
36 Inversion for Poisson stabilized by Anscombe Mäkitalo, Foi (TIP, 2011) Department of Signal Processing Karen Egiazarian
37 Experiment: clipped noisy data Original image : y (x1, x2) = 0.7 sin (2 π x1/512)+ 0.5 Department of Signal Processing
38 Experiment: Noise Estimation st.dev.-function . σ estimation and fitting a = 0.0038, b = 0.022 Department of Signal Processing
39 Experiment: denoised estimate after variance stabilization before declipping Department of Signal Processing
40 Experiment: declipped estimate Department of Signal Processing
41 Experiment: declipped estimate (crosssection) Department of Signal Processing
42 Real experiment: (Raw-data from Fujiflm FinePix S9600, ISO 1600) Department of Signal Processing
43 Real experiment: Denoising before declipping Department of Signal Processing
44 Real experiment: Denoising after declipping Department of Signal Processing
45 Real experiment: Denoising after declipping (crossection) Department of Signal Processing
46 LASIP www.cs.tut.fi/~lasip/ • Local Approximation Signal and Image Processing (LASIP) Project LASIP project is dedicated to investigations in a wide class of novel efficient adaptive signal processing techniques. Department of Signal Processing
47 LASIP LPA estimates, bias and variance, and asymptotic MSE Department of Signal Processing
48 LASIP : Intersection of Confidence Intervals ( ICI ) rule Goldenshluger & Nemirovski, 1997 Department of Signal Processing
49 Department of Signal Processing Karen Egiazarian
50 Anisotropy: motivation Department of Signal Processing Karen Egiazarian
51 Anisotropic estimator based on directional adaptive-scale: idea Department of Signal Processing Karen Egiazarian
52 Directional LPA Department of Signal Processing Karen Egiazarian
53 LASIP : HOW LPA-ICI WORKS Department of Signal Processing
54 Anisotropic LPA-ICI: Kernels used in practice Department of Signal Processing Karen Egiazarian
55 Department of Signal Processing Karen Egiazarian
56 Department of Signal Processing Karen Egiazarian
57 Department of Signal Processing Karen Egiazarian
Sliding DCT denoising DCT KxK DCT -1 x i B x i B y i B x i B y i B y i B K. Egiazarian, J. Astola, M. Helsingius, and P. Kuosmanen (1999) “Adaptive denoising and lossy compression of images in transform domain”, J. Electronic Imaging Department of Signal Processing
59 Shape-adaptive DCT image filtering By demanding the local fit of a polynomial model, we are able to avoid the presence of singularities or discontinuities within the transform support. In this way, we ensure that data are represented sparsely in the transform domain, significantly improving the effectiveness of shrinkage (e.g., thresholding). noisy image and noisy data after hard-thresholding adaptive-shape extracted from in SA-DCT domain neighborhood the neighborhood Department of Signal Processing
60 Shape-adaptation: use directional LPA-ICI Department of Signal Processing
61 Shape-adaptive DCT image filtering Pointwise SA-DCT: anisotropic neighborhoods Department of Signal Processing
Recommend
More recommend
