Pixels Measure Brightness over an Area Digital Images Sample the 2D - PowerPoint PPT Presentation

9/21/16 A Digital Image is a Matrix of Pixels Image Filtering 0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99 0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91 0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92 0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95 0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85 0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33 0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74 0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93 0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99 0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97 0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93 Reading: Sections 3.1, 3.2 and 10.3.1 in Szeliski book Source: D. Hoiem Pixels Measure Brightness over an Area Digital Images • Sample the 2D space on a regular grid (the “pixels”) • Quan9ze each sample (oHen, 8 or 24 bits per pixel) (the “brightness”, “intensity” or “gray level”) • Image represented as a (row, column) matrix of integer values (in Matlab) 2D 1D Source: S. Seitz 1

9/21/16 Percep9on of Intensity Percep9on of Intensity Simultaneous Contrast Effect Color Image R G B Source: D. Hoiem Source: E. Adelson 2

9/21/16 Digital Color Images Interpolate Values to Create the 3 RGB Channels Source: D. Hoiem Images in Matlab Image Filtering • Image represented as a matrix Except returns type • Suppose we have an n x m RGB image called “im” • Image filtering : computes a funcZon of a local neighborhood logical if 1 bpp – im(1,1,1) = top-leH pixel value in R-channel at each pixel posiZon – im(y, x, b) = y pixels down, x pixels to right in the b th channel • Called “ Local operator ,” “ Neighborhood operator ,” or – im( n , m , 3) = boWom-right pixel in B-channel “ Window operator ” • imread(filename) returns a uint8 image (values 0 to 255 for each color) • h : image è image – Convert to double format (values 0 … 1) with im2double row column • Uses: R 0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99 – Enhance images 0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91 G 0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92 • Noise reducZon, smooth, resize, increase contrast, 0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99 0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95 0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91 B recolor, arZsZc effects, etc. 0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85 0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92 0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.99 0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33 0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95 – Extract features from images 0.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.91 0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74 0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85 0.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.92 0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93 0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33 • Texture, edges, disZncZve points, etc. 0.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.95 0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99 0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74 0.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.85 0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97 0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93 – Detect paHerns 0.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.33 0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93 0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99 0.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.74 0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97 • Template matching, e.g., eye template 0.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.93 0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93 0.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.99 0.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.97 0.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93 Source: D. Hoiem 3

9/21/16 Instagram Filters Point Opera9ons • How do they make those Instagram filters? • Map each pixel’s value to a new value • Neighborhood is 1 x 1 • g ( i , j ) = h ( f ( i , j )) where f is the input image, g is the output (i.e., transformed) image, and h is the point operator / transformaZon • Examples – g ( i , j ) = af ( i , j ) + b where a > 0 is a gain parameter and b controls the brightness – Mapping one color space to another, e.g., RGB ⇒ HSV “It's really a combination of a bunch of different methods. In some cases – Image rotaZon, translaZon, scale change, … we draw on top of images, in others we do pixel math. It really depends on the effect we're going for.” --- Kevin Systrom, co-founder of Instagram Example Instagram Steps Example Instagram Steps 1. Perform an independent RGB color point 2. Overlay a circle background image to create a transformaZon on the original image to vigneWe effect increase contrast or make a color cast 4

9/21/16 Example Instagram Steps Example Instagram Steps 3. Overlay a background image as decoraZve 4. Add a border or frame grain Result Histogram Equaliza9on / FlaHening • Transform image by modifying its histogram – create a lookup table defining h • Why? Javascript library for – Image normalizaZon creating Instagram-like effects, see: – Comparing images http://alexmic.net/filtrr/ – Contrast enhancement • When an image’s histogram is transformed so that all gray levels occur about equally oHen, the result tends to produce an image with higher contrast. Why? 5

9/21/16 Histogram Equaliza9on Algorithm Histogram EqualizaZon Goal: Given an m x n image, f , with 8 bpp (gray levels 0 .. 255), create a new image, g , that has about mn /256 pixels with each gray level value 1. Compute f ’s histogram: h ( i ) = number of pixels with gray level i for i = 0..255 2. Compute f ’s cumulaZve histogram: c ( i ) = ∑ j =0.. i h ( j ) Ideally, image g has c ( k ) = ( mn /256)* k 3. Compute transformaZon k = t ( i ) = (256/ mn )* c ( i ) for i = 0..255 4. Create output image: g ( i , j ) = t ( f ( i , j )) Note: Max value in Step 3 above is 256, but legal values are 0..255, so subtract 1 Source: Wikipedia Color Transfer “Lab” Color Representa9on • Goal: Change the colors of a given “source” image to match the color “paleWe” of another L A transformation image in order to transfer the “mood” or of the colors into a “style” of one image to another color space that is • One simple method: ShiH and scale the pixel more perceptually meaningful: a values of the target image to match the mean L: luminance, and standard deviaZon of the source image a: red − green, • Color space representaZon of images affects b: blue − yellow quality of results. One good one: “Lab” color b space. 6

9/21/16 Color Transfer Color Transfer Let I = Source image, i.e., one being recolored Let J = PaleWe image, i.e., one colors taken from 1. Convert I and J to Lab color space; call result images S and P, respecZvely 2. Compute L channel output image matrix: Palette σ = − µ + µ L P ( L ) out S S P σ S 3. Similarly, compute a and b channels 4. Combine L , a and b , and convert back to RGB Local / Neighborhood Opera9ons Linear Filtering • Value of pixel in output image is a funcZon of the corresponding pixel in the input image plus other nearby pixels (usually defined by a square or rectangular “window” centered on the given pixel) 7

9/21/16 Linear Filtering Linear Filtering Linear Filtering Linear Filtering 8

9/21/16 Linear Filtering Linear Filtering Original image Linear Func9ons Image Filtering • Modify the pixels in an image based on some • Simplest: linear filtering funcZon of a local neighborhood of the pixels – Replace each pixel by a linear combinaEon of its neighbors • Simplest: linear filtering – Replace each pixel by a linear combination of its neighbors 10 5 3 Some function 10 5 3 0 0 0 4 5 1 7 4 5 1 0 0.5 0 7 1 1 7 1 1 7 0 1 0.5 Original, local image data Output image data Local image data kernel Output image data 9