IMA 4103 Extraction d’Information Multimédia Histogram-based methods Nicolas ROUGON ARTEMIS Department Nicolas.Rougon@telecom-sudparis.eu Institut Mines-Télécom
Overview ■ Thresholding & Noise ■ Histogram-based methods ● Histogram-based segmentation ● Histogram-based image enhancement Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Problem statement ■ We hereafter investigate the possibility of segmenting images based on color features only ■ For simplicity, we deal with gray level still images L ● Defined over the digital grid Ω Z n denoting |Ω| the # of pixels ● Encoded on b bits L ( x ) Λ = {0..2 b -1} ( x Ω ) ■ The presented notions readily generalize to ● Multichannel / multi-view images ● Image sequences Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Thresholding & Noise Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
T 100 ( L ) Thresholding ■ Image thresholding L T λ : upper-threshold operator at level λ L x L x λ if ( ) ( ) T λ L x ( )( ) 0 otherwise T λ : lower-threshold operator at level λ L x if L x λ ( ) ( ) T 100 ( L ) T λ L x ( )( ) 0 otherwise ● Thresholding is the basic operator for retaining pixels in a luminance range Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
B 100 ( L ) Binarization ■ Image binarization L B λ : upper-binarization operator at level λ 1 L x λ if ( ) B λ L x ( )( ) 0 otherwise B λ : lower-binarization operator at level λ 1 if L x λ ( ) B 100 ( L ) B λ L x ( )( ) 0 otherwise ● Whereas threshold operators output images, image binarization yields sets Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Level sets ■ Image level sets L λ : image level set at level λ L x Ω L ( x λ ) λ ● The collection ( L λ ) λ Λ of image level sets defines the image topographic map ● It provides an algebraic image decomposition L L λ λ Λ ● This allows for region-based geometric image processing ► mathematical morphology Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Level lines ■ Image level lines L λ : image level lines at level λ L x Ω L ( x λ ) λ ● Ω R n : level lines comprise constant luminance regions (plateau) ► non generic (noise) L 47 non-intersecting closed (except along ∂Ω ) codimension-1 surfaces ● Ω Z n : connectivity does not hold sub-pixel image interpolation is required Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Level lines ■ Image level lines L λ : image level lines at level λ L x Ω L ( x λ ) λ ● Level lines provide a geometric image decomposition Description of local image geometry Basic representation for contour-based geometric image processing Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Intensity-based image segmentation ■ Basic definition Intensity-based image segmentation consists of estimating a family of thresholds λ 1 < µ 1 ... λ i < µ i ... λ N < µ N allowing to partition the luminance space Λ into non-overlapping intervals [ λ i , µ i ] defining the segmentation regions R i λ R i x Ω B T ( L )( x ) 1 i μ i R Ω / R 0 i 1 i N ● More generally, segmentation regions can be defined as unions of intensity ranges [ λ i , µ i ] Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Intensity-based image segmentation ■ In practice Accurately segmenting natural images based on intensity / color features only is rarely possible ● Natural images are always noisy ● Natural objects are generally textured ● Sensor and scene (object & source) properties induce variability in object appearance low contrast cast shadows transparency non-uniform lighting multiple reflections … Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Intensity-based image segmentation ■ In practice ● Preprocessing is often necessary to reduce appearance variability and improve signal quality Restoration Photometric calibration (denoising, deblurring…) Contrast-enhancement ► This statement applies to any segmentation approach ● Statistics and machine learning provide tools for consistently modeling/dealing with intensityvariability Hard assignment to disjoint intervals is replaced by probabilistic /statistical assignment to overlapping clusters ► This framework allows for intensity mixtures Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Intensity-based image segmentation ■ Performance and limitations ● Intensity-based segmentation approaches are mostly applicable to simple images ► Cartoon model ● Integrating spatial / spatiotemporal context information is usually necessary to disambiguate local intensity information ► Improved accuracy / robustness ● Nonetheless, intensity-based approaches are often relevant for efficiently deriving a coarse segmentation (pre-segmentation) ► Meaningful for real-time applications e.g. object tracking in live video sequences Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image noise ■ Noise models ● Optical imaging Additive ● Thermal imaging ► IR L = L 0 + n ● Quantization noise ● Speckle noise ► US ► OCT Multiplicative ► SAR ► PC microscopy (interferometric imaging) L = L 0 n ► PC X-Ray ● Shot noise ► X-Ray (photon counting imaging) ► TEP / SPECT Complex ● Impulse noise models (salt-and-pepper) Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image noise ■ Noise distributions ● Optical imaging Gaussian ● Thermal imaging ► IR ● Speckle noise ► US ► OCT Heavy-tailed ► SAR ► PC microscopy (interferometric imaging) Laplace | negative- ► PC X-Ray exponential | α -stable ● Shot noise ► X-Ray Poisson (photon counting imaging) ► TEP / SPECT ● Thermal noise ► MRI Rician | Rayleigh ● Quantization noise Uniform ● Salt-and-pepper noise Impulse Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image noise ■ Noise distributions ● Gaussian ● Rician |Rayleigh Optical Infrared MRI Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image noise ■ Noise distributions ● Negative exponential US OCT PC X-Ray PC microscopy SAR LIDAR Sonar Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image noise ■ Noise distributions ● Poisson medical industrial X-Ray X-Ray PET SPECT Hyperspectral Astronomical Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image noise ■ Noise distributions ● Impulse ● Uniform Optical Color quantization Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image noise ■ Modeling issues A consistent noise model for a sensor/ imaging chain is a key point ● For methodology design Integrating realistic noise priors leads to optimal denoising schemes and robust image analysis estimators ● For performance assessment Adequate noise generators allows for synthesizing simulated test images with controlled SNR Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Generic image segmentation framework Filtering & Restoration Preprocessing Segmentation Postprocessing ● Denoising ● Artifacts removal ● Deblurring Contour-based approaches ● Enhancement ● Linking ● … ● Thinning ● … Region-based approaches ► Preprocessing can be integrated ● Hole filling into segmentation, yielding ● Split & Merge robust segmentation schemes ● … Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Histogram techniques Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image histogram ■ Image histogram The histogram H L of an image L is the array counting the number of gray level occurrences in L k Λ H L ( k ) = # { x Ω | L ( x ) = k } modes intensity range Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image histogram ■ Image histogram The histogram H L of an image L is the array counting the number of gray level occurrences in L k Λ H L ( k ) = # { x Ω | L ( x ) B k } ● Histograms can be quantified ► Histogram binning ● Optimal (uniform) bin size selection rules Sturges | Scott | Freedman-Diaconis Knuth | Wand Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image histogram ■ Generalization The previous definition generalizes to ● Multichannel / multi-view images ► Multidimensional histogram / co-occurrence matrix ● Arbitrary domains Ω e.g. image region / pixel neighborhood ► Regional / local histogram ● Arbitrary (quantified) image features e.g. local orientation / motion ► Feature histogram Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Image histogram ■ Properties ● H L accounts for luminance distribution over Ω Empirical density estimator 1 L p k H ( k ) L Ω ► H L summarizes luminance global / regional / local statistical properties Institut Mines-Télécom IMA 4103 - Nicolas ROUGON
Recommend
More recommend
