Leica - CNIC 1st Practical School in Super-Resolution Microscopy, Centro Nacional de Investigaciones Cardiovasculares Carlos III (CNIC) Image Restoration by Deconvolution: Concepts and Applications Chong Zhang SIMBioSys, Depertment of Information and Communication Technologies Universitat Pompeu Fabra 15th March, 2016
What is deconvolution in your mind? Convolution Spatial resolution Deconvolution Pixel size Point Spread Function Rayleigh Criterion Noise Airy disk Fourier Transform Numerial Aperture Refractive Index Wavelength How do you perform deconvolution? Fiji plugins Huygens Professional MC AutoquantX …
What is deconvolution (in microscopy)? Deconvolution is a computational technique allowing to partly compensate for the image distortion caused by a microscope. The betterment can be signi fj cant both in terms of attenuation of the out of focus light and increase of the spatial resolution . It was fj rst devised at the MIT for seismology (Robinson, Wiener, early 50’), then applied to astronomy and fj nally found its way to 3D optical fm uorescence microscopy (Agard 1984). It should not be seen as a “black box” to enhance image quality since it can introduce artifacts or further degrade low quality images. It is compatible with quantitative measurements (should even improve). It works best for thin (<50 um), optically transparent, fj xed, bright samples. Challenging for live microscopy: short exposure (limit motion blur), objective adapted to medium (limit spherical aberrations). Courtesy of S. Tosi
Ideal case Image Detector Ideal ¡ case ¡ Objective Specimen Courtesy of P. Bankhead
Reality Image Detector - Blurred More ¡ realis-c ¡ Objective Specimen Courtesy of P. Bankhead
Reality Image Detector - Blurred - Noisy More ¡ realis-c ¡ Objective Specimen Courtesy of P. Bankhead
Noises along the optical train in digital microscopy Courtesy of S. Tosi
Convolution Convolution consists of replacing each point in the original object with its blurred image in all dimensions and summing together overlapping contributions from adjacent points to generate the resulting image Filter kernel � � Original image Original image Filtered image Filtered image “stamping” the kernel on each pixel of the image the kernel is scaled (multiplied) by the intensity of the central pixel Accumulated (summed) in the output image. Courtesy of S. Tosi
Convolution Convolution consists of replacing each point in the original object with its blurred image in all dimensions and summing together overlapping contributions from adjacent points to generate the resulting image = = A 3D kernel is a stack holding the fj lter coe ffi cients Courtesy of S. Tosi
Fourier Transform (FT) vs convolution If we take the FT of the Spatial = equation, the is replaced by Domain multiplication, thus image restoration might be achievable by: Fourier dividing the FT of the image Domain = X by the FT of the kernel and then taking the inverse Fourier transform. Courtesy of S. Tosi
Point Spread Function (PSF) A record of how much the image created by a microscope spreads/blurs an object of a single point (thus determines the way in which images of objects blur into each other in the fj nal image). Wide fj eld PSF Cannell 2006 Bankhead 2014
Theoretical PSF vs Measured PSF Measured PSF : an image resulting from a single small spherical fm uorescent bead (smaller than the optical resolution, thus forms e ff ectively a point source of light) Theoretical PSF : generated by a computer program, after input of values describing the optical system – the magni fj cation , NA of the objective, the illuminating and emitted wavelengths , and the refractive index of the objective lens immersion medium. It gives an indication of the best possible resolution for a given objective but these limits are not achievable. Real PSFs are typically >20% bigger than calculated versions. Parton 2006 Measured PSF Theoretical PSF (experimental PSF) Courtesy of S. Tosi Bankhead 2014
3D ¡ Gated ¡ /GSD STED ¡ STED ¡ <50 ¡ <130 ¡ 70 ¡ 560 ¡ <130 ¡ 70 ¡ Courtesy of N. Garin
PSF in the focal plane (where most things are measured) Airy disk ¡ Spatial resolution : radius of the smallest point source in the image, i.e. fj rst minimum of Airy disk Parton 2006 Bankhead 2014 Wide fj eld PSF: 100nm beads, excitation 520nm, emission 617nm Bankhead 2014 Rayleigh criterion Notes: 1. Rayleigh criterion has not taken λ : fm uorophore emission wavelength into account the e ff ects of: NA : objective numerical aperture brightness, pixel size, noise n : refractive index of the objective lens immersion medium 2. High NAs are possible when the NA = n sin θ immersion refractive index is NA can never exceed n , which itself has high fj xed values (e.g. 1.0 for air, 1.33 for water, or 1.52 for oil)
What else does measured PSF tell us? Asymmetry radial (x-y): commonly misalignment of optical components about the z-axis, either as tilt or decentration along the optical axis (z-axis): commonly due to spherical aberration , which may result from refractive index mismatches between the objective, immersion medium, and sample or tube length/coverslip thickness errors. Pawley 2006 Notes: 1. The immersion refractive index should match the refractive index of the medium surrounding the sample, to avoid spherical aberration 2. Item 1 is often strongly preferable to using the highest NA objective available, as it is usually better to have a larger PSF than a highly irregular one .
Deconvolution principle Noise Digital fj lter Microscope PSF Deconvolved image Object (sample) Courtesy of S. Tosi = R = D The deconvolution fj lter F should “undo” the e ff ect of the microscope PSF H by processing the sampled image R , ideally D = S .
Deconvolution principle Noise Digital fj lter Microscope PSF Deconvolved image Object (sample) Courtesy of S. Tosi = R = D The deconvolution fj lter F should “undo” the e ff ect of the microscope PSF H by processing the sampled image R , ideally D = S .
Classi fj cation of deconvolution methods Not for Fast Nearest neighbors 2D only quantitative Deblurring Software intensity subtractive No need PSF No neighbors available measures Do not Not for Tikhonov-Miller fj lter count super for noise resolution Fast 3D Linear Software inverse Needs PSF Trade-o ff available Regularized between Wiener fj lter inverse sharpness & noise Jansson van Cittert Also for 3D Use noise super models Needs PSF resolution Richardson-Lucy Slow Constrained iterative Good 3D May not 3D results be always Adaptive blind Estimates quantitative PSF Lifting the Fog: Image Restoration by Deconvolution, Parton 2006
Linear deconvolution: inverse fj lter deconvolution For example, assuming H known, F linear (convolution) and no noise ( N = 0) leads to: But in practice… Noise enhancement ruins our e ff orts! 4,1·√10 -‑8 ¡ 2,4·√10 7 ¡ 1 ¡ 1 ¡ A very simple model for the PSF H (Gaussian std = 1 pixel) H -1 power spectrum (log display) H power spectrum (log display) overlaid with raw values overlaid with raw values Courtesy of S. Tosi
Inverse fj lter deconvolution H is a Gaussian with std = 2 pixels + N H H -1 Original image S S after convolution by H No noise Noise std = 10 -4 Noise std = 10 -12 Courtesy of S. Tosi
Second try: regularized inverse 4,1·√10 -‑8 ¡ 2,4·√10 7 ¡ 100 ¡ 1 ¡ 1 ¡ 1 ¡ A very simple model for the PSF H (Gaussian std = 1 pixel) ( H -1 ) reg (1% clipping) H power spectrum (log display) power spectrum (log display) overlaid with raw values overlaid with raw values Courtesy of S. Tosi
Regularized Inverse Filter Deconvolution + N H Original image S S after convolution by H ( H -1 ) trunc Courtesy of S. Tosi Noise std = 10 -4
Third Try: Wiener Filter The Golden Linear Deconvolution Trade-o ff Coming back to: Minimizing the expectation of ||E|| over all possible noise realizations assuming a white Gaussian noise: Wiener fj lter Bands free of noise: | N(u,v)| = 0 � F(u,v) = H(u,v) -1 (inverse fj lter) Strong noise bands: |N(u,v)| � ∞ � F(u,v) � 0 (cut-o ff ) Intermediate bands: best trade-o ff As ¡noise ¡at ¡certain ¡frequencies ¡increases, ¡the ¡signal-‑to-‑noise ¡ra-o ¡drops, ¡so ¡ F ¡also ¡ drops. ¡This ¡means ¡that ¡the ¡ Wiener fj lter attenuates frequencies dependent on their signal-to-noise ratio. Courtesy of S. Tosi
Wiener Deconvolution Regularized inverse fj lter result Wiener fj lter result Noise std = 10 -4 Noise std = 10 -4 Courtesy of S. Tosi
Recommend
More recommend
