ECG782: Multidimensional Digital Signal Processing Spatial Domain - PowerPoint PPT Presentation

Professor Brendan Morris, SEB 3216, brendan.morris@unlv.edu ECG782: Multidimensional Digital Signal Processing Spatial Domain Filtering http://www.ee.unlv.edu/~b1morris/ecg782/

2 Outline • Background • Intensity Transformations • Histogram Equalization • Linear Image Filtering • Morphology • Connected Components

3 Outline • Background • Intensity Transformations • Histogram Equalization • Linear Image Filtering • Morphology • Connected Components

4 Spatial Domain Processing • Spatial domain = the image plane ▫ Image processing through direct manipulation of image pixels ▫ Generally are more computationally efficient and require less resources than transform methods • Two categories of spatial processing ▫ Intensity transformations – operate on single pixels ▫ Spatial filtering – operations that work in a neighborhood of each pixel

5 Image Processing Basics • Input an image to a system  • Basic spatial filtering get a processed image as output implementation 𝑔(𝑦, 𝑧) 𝑈 𝑕(𝑦, 𝑧) • 𝑕 𝑦, 𝑧 = 𝑈 𝑔 𝑦, 𝑧 ▫ 𝑔(𝑦, 𝑧) – input image ▫ 𝑕(𝑦, 𝑧) – output image ▫ 𝑈 – operator defined over a neighborhood around (𝑦, 𝑧) • Apply operator 𝑈 to pixels in the neighborhood to yield output at (𝑦, 𝑧) ▫ Typically the neighborhood is rectangular and much smaller size than image

6 Outline • Background • Intensity Transformations • Histogram Equalization • Linear Image Filtering • Morphology • Connected Components

7 Intensity Transformations • Spatial filtering with smallest • Contrast stretching 1 × 1 neighborhood ▫ Increase dark/light pixels ▫ 𝑕 only depends on 𝑔 at a single point (𝑦, 𝑧) • Intensity transformation function (gray-level mapping) ▫ 𝑡 = 𝑈(𝑠)  𝑠 – input intensity  𝑡 – output intensity • Thresholding ▫ Produce binary (two-level) image

8 Pixel Transforms • Gain and bias (Multiplication and addition of constant) ▫ 𝑕 𝑦, 𝑧 = 𝑏(𝑦, 𝑧)𝑔 𝑦, 𝑧 + 𝑐(𝑦, 𝑧) ▫ 𝑏 (gain) controls contrast ▫ 𝑐 (bias) controls brightness  Notice parameters can vary spatially (think gradients) • Linear blend ▫ 𝑕 𝑦 = 1 − 𝛽 𝑔 0 𝑦 + 𝛽𝑔 1 (𝑦) ▫ We will see this used later for motion detection in video processing

9 Image Negatives • Given image with intensity range [0, 𝑀 − 1] • Negative image transformation ▫ 𝑡 = 𝑀 − 1 − 𝑠 • Reverse intensity levels of image ▫ Well suited for enhancing white or gray detail embedded in a dark image

10 Compositing and Matting • Techniques to remove an object and place it in a new scene ▫ E.g. blue/green screen • Matting – extracting an object from an original image • Compositing – inserting object into another image (without visible artifacts) • A fourth alpha channel is added to an RGB image 𝛽 describes the opacity (opposite of transparency) of a pixel ▫ • Over operator – a linear blend ▫ 𝐷 = 1 − 𝛽 𝐶 + 𝛽𝐺

11 Outline • Background • Intensity Transformations • Histogram Equalization • Linear Image Filtering • Morphology • Connected Components

12 Histogram Processing • Digital image histogram is the count of pixels in an image having a particular value in range [0, 𝑀 − 1] ▫ ℎ 𝑠 𝑙 = 𝑜 𝑙  𝑠 𝑙 - the kth gray level value  Set of 𝑠 𝑙 are known as the bins of the histogram  𝑜 𝑙 - the numbers of pixels with kth gray level • Empirical probability of gray level occurrence is obtained by normalizing the histogram ▫ 𝑞 𝑠 𝑙 = 𝑜 𝑙 /𝑜  𝑜 – total number of pixels

13 Histogram Example • x-axis – intensity value ▫ Bins [0, 255] • y-axis – count of pixels • Dark image ▫ Concentration in lower values • Bright image ▫ Concentration in higher values • Low-contrast image ▫ Narrow band of values • High-contrast image ▫ Intensity values in wide band

14 Histogram Equalization • Let 𝑡 by the cumulative • Assume continuous functions (rather than discrete images) distribution function (CDF) 𝑠 • Define a transformation of the ▫ 𝑡 = 𝑈 𝑠 = 𝑞 𝑠 𝑥 𝑒𝑥 0 intensity values to “equalize” • Then each pixel in the image 𝑒𝑡 𝑒𝑠 = 𝑞 𝑠 (𝑠) ▫ ▫ 𝑡 = 𝑈 𝑠 0 ≤ 𝑠 ≤ 1 • Which results in a uniform ▫ Notice: intensity values are PDF for the output intensity normalized between 0 and 1 ▫ 𝑞 𝑡 𝑡 = 1 • The inverse transformation is given as ▫ 𝑠 = 𝑈 −1 𝑡 0 ≤ 𝑡 ≤ 1 • Hence, using the CDF of a histogram will “equalize” an • Viewing the gray level of an image image as a random variable ▫ Make the resulting histogram 𝑒𝑠 ▫ 𝑞 𝑡 (𝑡) = 𝑞 𝑠 (𝑠) 𝑒𝑡 flat across all intensity levels

15 Discrete Histogram Equalization • The probability density is approximated by the normalized histogram 𝑜 𝑙 ▫ 𝑞 𝑠 𝑠 𝑙 = 𝑜 𝑙 = 0, … , 𝑀 − 1 • The discrete CDF transformation is 𝑙 𝑙 = ▫ 𝑡 𝑙 = 𝑈 𝑠 𝑞 𝑠 (𝑠 𝑘 ) 𝑘=0 𝑜 𝑙 𝑙 ▫ 𝑡 𝑙 = 𝑘=0 𝑜 • This transformation does not guarantee a uniform histogram in the discrete case ▫ It has the tendency to spread the intensity values to span a larger range

16 Histogram Equalization Example • Histograms have wider spread of intensity levels • Notice the equalized images all have similar visual appearance ▫ Even though histograms are different ▫ Contrast enhancement Original histogram original image histogram equalized equalized image

17 Local Histogram Enhancement • Global methods (like • Original image histogram equalization as presented) may not always make sense ▫ What happens when properties of image regions are different? • Block histogram equalization • Compute histogram over smaller windows ▫ Break image into “blocks” ▫ Process each block separately • Notice the blocking effects that cause noticeable boundary effects

18 Local Enhancement • Compute histogram over a block (neighborhood) for every pixel in a moving window • Adaptive histogram equalization (AHE) is a computationally efficient method to combine block based computations through interpolation Figure 3.8 Locally adaptive histogram equalization: (a) original image; (b) block histogram equalization; (c) full locally adaptive equalization.

19 Outline • Background • Intensity Transformations • Histogram Equalization • Linear Image Filtering • Morphology • Connected Components

20 Image Processing Motivation • Image processing is useful for the reduction of noise • Common types of noise ▫ Salt and pepper – random occurrences of black and white pixels ▫ Impulse – random occurrences of white pixels ▫ Gaussian – variations in intensity drawn from normal distribution Adapted from S. Seitz

21 Ideal Noise Reduction • How can we reduce noise given a single camera and a still scene? ▫ Take lots of images and average them • What about if you only have a single image? Adapted from S. Seitz

22 Image Filtering • Filtering is a neighborhood operation ▫ Use the pixels values in the vicinity of a given pixel to determine its final output value • Motivation: noise reduction ▫ Replace a pixel by the average value in a neighborhood ▫ Assumptions:  Expect pixels to be similar to their neighbors (local consistency)  Expect noise processes to be independent from pixel to pixel (i.i.d.)

23 Linear Filtering • Most common type of neighborhood operator • Output pixel is determined as a weighted sum of input pixel values ▫ 𝑕 𝑦, 𝑧 = 𝑔 𝑦 + 𝑙, 𝑧 + 𝑚 𝑥(𝑙, 𝑚) 𝑙,𝑚  𝑥 – is known as the kernel, mask, filter, template, or window  𝑥(𝑙, 𝑚) – entry is known as a kernel weight or filter coefficient • This is also known as the correlation operator ▫ 𝑕 = 𝑔⨂𝑥

24 Filtering Operation • 𝑕 𝑦, 𝑧 = 𝑔 𝑦 + 𝑙, 𝑧 + 𝑚 𝑥(𝑙, 𝑚) 𝑙,𝑚 • The filter mask is moved from point to point in an image • The response is computed based on the sum of products of the mask coefficients and image • Notice the mask is centered at 𝑥 0,0 • Usually we use odd sized masks so that the computation is symmetrically defined • Matlab commands ▫ imfilter.m, filter2.m, conv2.m

25 Filtering Raster Scan • Zig-zag scan through of image ▫ Process image row-wise

26 Connection to Signal Processing • General system notation 𝑦 𝑧 𝑔 • LTI system ▫ Convolution relationship • Discrete 1D LTI system • Discrete 2D LTI system 𝑦[𝑜] ℎ 𝑧[𝑜] 𝑔(𝑦, 𝑧) 𝑕(𝑦, 𝑧) 𝑥 ∞ ∞ ∞ 𝑧 𝑜 = 𝑦 𝑙 ℎ[𝑜 − 𝑙] 𝑕(𝑦, 𝑧) = 𝑔 𝑡, 𝑢 𝑥(𝑦 − 𝑡, 𝑧 − 𝑢) 𝑙=−∞ 𝑡=−∞ 𝑢=−∞ ▫ Linear filtering is the same as convolution without flipping