Master ISTI / PARI / IV Introduction to Astronomical Image Processing 8. Mid-level processing (data preparation) André Jalobeanu LSIIT / MIV / PASEO group Jan. 2006 lsiit-miv.u-strasbg.fr/paseo PASEO
Mid-level processing: data preparation Data presentation Enhanced visualization Color display of multidimensional data Multiresolution vision model Dimensionality reduction Dimensionality reduction methods, Band fusion Object extraction and signal/noise separation SNR maximization via binning Significant information detection Combining multiple observations: data fusion Frame co-addition, mosaicing Super-resolution, multiframe restoration
Data presentation Learn how to better display grayscale or RGB color images Get acquainted to problems related to multispectral image visualization See how nonlinear transforms and multiscale representations can help visualize lots of information at once
� � � � Enhanced visualization single band (grayscale) images High dynamic range, difficult to visualize � linear pixelwise transforms ( e.g. histogram clipping ) � nonlinear pixelwise transforms ( e.g. log-scale ) � multiple pixel operations ( e.g. opacity filtering, unsharp mask ) � grayscale to RGB mapping: false color display M51 in false color (C. Buil) histogram clipping
Enhanced visualization single band (grayscale) images C. Buil X+k.(X-X*g) amplify details (image - blurred image) g = Gaussian blur Unsharp masking (Sun spot) C. Buil X.f(X*g) reduce high pixel intensities , opacity mask= blurred image Opacity masking (M51)
Enhanced visualization 3-band (RGB) images ๏ Color space manipulations (e.g. HSV space) ‣ Saturation adjustment (S) Compensate for overlapping filters (poor color image) ‣ Color cycling (H) Try different colors for a better visual segmentation HSV space ‣ Nonlinear intensity transform with color preservation (V) Useful for high dynamic range images C. Buil C. Buil Saturation-enhanced Before and after saturation adjustment “true colors” (400, 560, 910nm)
Color display of multidimensional data High dimensionality of multi/hyperspectral images: Impossible to visualize them as they are! Reduction to 3 bands (RGB) using PCA, ICA... insufficient (stationary model) ๏ Multiband image reduction & display [Petremand 04] ‣ PCA or ICA reduction ( keep at least 6 bands ) ‣ Markovian segmentation : label map ‣ Discriminant Factor Analysis : color (H,S) Work in the HSV (Hue Saturation Value) space ‣ PCA for each class : V ‣ HSV to RGB conversion 6 reduced bands (from 48) Label map and color display
Multiscale Vision Model ๏ Use a multiresolution transform (wavelets) [Bijaoui 95] ‣ Select significant coefficients (above noise) ‣ Grow regions defined by the significant coefficient location ‣ Keep regions according to the inter-scale, region-based connectivity (image segmentation) Galaxy MVM segmentation Connectivity
Dimensionality reduction Grasp the ideas behind classical dimensionality reduction methods Understand why such methods are not well-suited to astronomy Get to know recent techniques used to merge multispectral data into a single band image
� � Dimensionality reduction methods ๏ Principal Component Analysis (PCA) methods Maximum variance subspace, Gaussian assumptions ‣ Simple PCA - Gaussian cluster, no noise, one step linear ‣ Probabilistic PCA, Factor Analysis - add some noise (indep.) embedding subspace ‣ Mixture of PCA - mixture of PCA models: multiple clusters, nonlinearity ๏ Independent Component Analysis (ICA) Non-Gaussian components assumption ‣ Simple ICA - maximize the nongaussianity linear ‣ Probabilistic ICA - add some noise embedding subspace ‣ Projection pursuit - more general: maximize projection index ‣ Mixture of ICA - multiple clusters, nonlinearity iterative algorithms, EM used in probabilistic approaches Major drawbacks: ◆ Stationarity assumption ◆ No image formation model (e.g. blur) Linear vs. nonlinear embedding subspace
� � � � Dimensionality reduction of deep field images ๏ Problems specific to astronomical images ‣ Spatially adaptive behavior ‣ Dark background & photon noise ‣ Sources multiplied by spatially variable density (e.g. nebulae) ‣ Compound sources (e.g. star clusters, galaxies) Synthetic image ‣ Complex sources (e.g. stars) ๏ Problems related to Integral Field Spectroscopy ‣ Very large number of bands ( thousands ) 2-band histogram One spectrum for each spaxel, contains lots of information that needs to be preserved! ‣ Very small number of spatial samples ( until Muse, 2012 ) Better goal: Source separation or decorrelation � Find minimum number of independent sources Color histogram (Hue,Saturation) To preserve information, may be larger than the number of bands! � Determine their spatial distribution ( spatial consistency )
Band fusion ๏ Merge multidimensional data into a single band SNR maximization & detection purposes ‣ Simple averaging (naive, inaccurate) ‣ Weighted averaging Optimal averaging, stationary noise ‣ Spatially adaptive averaging • Use image space information • Use a multiscale approach based on (e.g. saturation, bad pixels) wavelet transforms Wavelet domain band fusion using Hidden Markov Trees (combined with Van Cittert deconvolution) [Flitti 05]
Object extraction and signal/noise separation Grasp the principles of SNR enhancement using spatial grouping Become familiar with the significant information extraction problem and the related challenges
SNR maximization via spatial binning Maximize the SNR by averaging samples (Central Limit Theorem: σ →σ / √ n) ๏ Uniform binning ‣ Sum or average pixel values within predefined super-pixels ๏ Spatially adaptive binning ‣ Find image space subdivision such that SNR=constant ( prescribed SNR ) • Quadtree plane partition • Voronoi tessellation Quadtree partition Voronoi tessellation X-ray data binning, by S. Diehl & T. Statler
Significant information detection ๏ Nonparametric approaches ‣ Strong denoising (ensure a minimum false alarm rate) Use existing methods and minimize the residual noise ‣ Enhanced visualization of significant transform coefficients ( e.g. Multiscale Vision Model ) ๏ Parametric methods ‣ Supervised object detection • Use template matching to detect predefined patterns • Recursive object detection/subtraction ( e.g. CLEAN ) • Feature extraction ( e.g. local maxima ) • Statistical fitting (parametric objects), decision theory ‣ Unsupervised object extraction & optimal representation • Pixon approach [Pina-Puetter 93]: Set of parametric shapes (dictionary), simplest representation • Fully Bayesian approaches, to be developed → Keep the parameters (shape, scale, location) • Both spatial and spectral methods (for IFS), to be developed
Combining multiple observations: data fusion Understand how to combine multiple images into a single one See how super-resolution can help preserve information and extrapolate a limited bandwidth Get acquainted to some techniques used to increase SNR, FOV and bandwidth in astronomy
� � � � Data fusion: introduction Problem: lots of data, same object! Usually, images are recorded with various: ◆ pose parameters (position, orientation) ◆ sensors (resolution, noise, bad pixels) ◆ observing conditions (transparency, seeing) ◆ telescopes (PSF, distortions) ๏ Multisource data fusion objectives ‣ Optimally combine all observations into a single image Co-add or build a mosaic, depending on the overlap ‣ Preserve all the information from the original data set Increase resolution if needed, compute the uncertainties ‣ Optional: enhance the image quality Denoise or deblur depending on the degradation ๏ Possible approaches ‣ Forward methods Shift-and-add, register-and-add, simple resampling schemes, drizzle ‣ Inverse methods Bayesian super-resolution, probabilistic fusion, multiframe deconvolution
Frame co-addition in deep-field imaging increase SNR & dynamic range ๏ Register-and-add ‣ Apply geometric corrections: resample input images potential problems: Avoid simple interpolation schemes! noise correlation, ‣ Add resulting images image blurring, sampling... Use weighted sum to account for noise variations Apply rank filters to remove cosmic rays & bad pixels ๏ Drizzle ‣ Designed to handle undersampled HST images [Fruchter 96] ‣ Shrink input pixels, project onto finer output grid (scale ≈ 0.5-0.7) Perform geometric corrections, including distortions ‣ Add: Image update iteration ( add a drop ) X ← (awY+WX)/(aw+W), W ← aw+W Y Y = drop value (pixel value in the input image) a = intersection btw. drop and fine output grid X w = user-defined pixel weight ( significance ) Prerequisite: accurate camera calibration (estimate the registration parameters)
Image mosaicing increase the FOV ๏ Same as image co-addition, with some specificities ‣ In all cases: • Accurate camera calibration using overlapping areas • Image normalization (flat, transparency, exposure) to avoid boundary effects ‣ Register-and-add Apply geometric corrections (registration), add images in overlapping areas ‣ Drizzle Shrink pixels, project (geometric transform), add drops to finer pixels Output pixel size < input PSF FWHM / 2 O’Dell / NASA Image mosaicing in planetary and wide field astronomical imaging
Recommend
More recommend
