2/1/2011 Previously • Filters allow local image neighborhood to influence our description and features – Smoothing to reduce noise Edges and Binary Image Analysis – Derivatives to locate contrast, gradient • Seam carving application: Mon, Jan 31 – use image gradients to measure “interestingness” or Prof. Kristen Grauman “energy” UT-Austin – remove 8-connected seams so as to preserve image’s energy. Edge detection Today • Goal : map image from 2d array of pixels to a set of • Edge detection and matching curves or line segments or contours. – process the image gradient to find curves/contours • Why? – comparing contours • Binary image analysis – blobs and regions Figure from J. Shotton et al., PAMI 2007 Figure from D. Lowe • Main idea : look for strong gradients , post-process Gradients -> edges Thresholding • Choose a threshold value t Primary edge detection steps: • Set any pixels less than t to zero (off) 1. Smoothing: suppress noise • Set any pixels greater than or equal to t to one 2. Edge enhancement: filter for contrast (on) 3. Edge localization Determine which local maxima from filter output are actually edges vs. noise • Threshold, Thin Kristen Grauman, UT-Austin 1
2/1/2011 Original image Gradient magnitude image Thresholding gradient with a lower threshold Thresholding gradient with a higher threshold Canny edge detector The Canny edge detector • Filter image with derivative of Gaussian • Find magnitude and orientation of gradient • Non-maximum suppression : – Thin wide “ridges” down to single pixel width • Linking and thresholding ( hysteresis ): Linking and thresholding ( hysteresis ): – Define two thresholds: low and high – Use the high threshold to start edge curves and the low threshold to continue them • MATLAB: edge(image, ‘canny’); original image (Lena) >>help edge • Source: D. Lowe, L. Fei-Fei Slide credit: Steve Seitz 2
2/1/2011 The Canny edge detector The Canny edge detector norm of the gradient thresholding The Canny edge detector Non-maximum suppression How to turn these thick regions of the gradient into curves? Check if pixel is local maximum along gradient direction, select single max across width of the edge thresholding • requires checking interpolated pixels p and r The Canny edge detector Hysteresis thresholding • Use a high threshold to start edge curves, and a low threshold to continue them. Problem: Problem: pixels along this edge didn’t survive the thresholding thinning (non-maximum suppression) Source: Steve Seitz 3
2/1/2011 Hysteresis thresholding Hysteresis thresholding original image original image original image high threshold low threshold hysteresis threshold (strong edges) (weak edges) high threshold low threshold hysteresis threshold (strong edges) (weak edges) Source: L. Fei-Fei Source: L. Fei-Fei Recap: Canny edge detector Low-level edges vs. perceived contours • Filter image with derivative of Gaussian • Find magnitude and orientation of gradient • Non-maximum suppression : – Thin wide “ridges” down to single pixel width • Linking and thresholding ( hysteresis ): Linking and thresholding ( hysteresis ): – Define two thresholds: low and high – Use the high threshold to start edge curves and the low threshold to continue them • MATLAB: edge(image, ‘canny’); Shadows Background Texture >>help edge • Source: D. Lowe, L. Fei-Fei Kristen Grauman, UT-Austin Low-level edges vs. perceived contours Learn from humans which combination of image human segmentation gradient magnitude features is most indicative of a “good” contour? Berkeley segmentation database: http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/ Human-marked segment boundaries [D. Martin et al. PAMI 2004] Source: L. Lazebnik 4
2/1/2011 What features are responsible for What features are responsible for perceived edges? perceived edges? Feature profiles Feature profiles (oriented energy, (oriented energy, brightness, color, brightness, color, and texture and texture gradients) along gradients) along the patch s the patch’s the patch’s the patch s horizontal horizontal diameter diameter [D. Martin et al. PAMI 2004] [D. Martin et al. PAMI 2004] Kristen Grauman, UT-Austin Kristen Grauman, UT-Austin State-of-the-Art in Contour Detection Canny+opt thresholds Human agreement Canny Prewitt, Learned Sobel, with Roberts combined features UC Berkeley Source: Jitendra Malik: Computer Vision Group [D. Martin et al. PAMI 2004] http://www.cs.berkeley.edu/~malik/malik-talks-ptrs.html Kristen Grauman, UT-Austin Today • Edge detection and matching – process the image gradient to find curves/contours – comparing contours • Binary image analysis – blobs and regions Figure from Belongie et al. 5
2/1/2011 Chamfer distance Chamfer distance • Average distance to nearest feature = I Set of points in image = T Set of points on (shifted) template = ( t ) Minimum distance between point t d I and some point in I Kristen Grauman, UT-Austin Kristen Grauman, UT-Austin Chamfer distance Distance transform • Average distance to nearest feature Distance Transform Image features (2D) 1 0 1 2 3 4 3 2 How is the measure 1 0 1 2 3 3 2 1 different than just 1 0 1 2 3 2 1 0 1 0 0 1 2 1 0 1 filtering with a mask 2 1 1 2 1 0 1 2 having the shape 3 3 2 2 2 2 2 2 1 1 0 0 1 1 2 2 points? points? 4 3 3 2 1 0 1 2 5 4 4 3 2 1 0 1 How expensive is a naïve implementation? ( ⋅ Distance Transform is a function that for each image D ) pixel p assigns a non-negative number corresponding to D ( p ) distance from p to the nearest feature in the image I Edge image Features could be edge points, foreground points,… Source: Yuri Boykov Kristen Grauman, UT-Austin Distance transform (1D) Distance transform // 0 if j is in P , infinity otherwise edges d original distance transform Value at (x,y) tells how far that position is from the nearest edge point (or other binary mage structure) >> help bwdist Adapted from D. Huttenlocher Kristen Grauman, UT-Austin 6
2/1/2011 Chamfer distance Distance Transform (2D) • Average distance to nearest feature Edge image Distance transform image Adapted from D. Huttenlocher Kristen Grauman, UT-Austin Chamfer distance Chamfer distance: properties • Sensitive to scale and rotation • Tolerant of small shape changes, clutter • Need large number of template shapes g p p • Inexpensive way to match shapes Edge image Distance transform image Fig from D. Gavrila, DAGM 1999 Kristen Grauman, UT-Austin Binary images Today • Edge detection and matching – process the image gradient to find curves/contours – comparing contours • Binary image analysis – blobs and regions Kristen Grauman, UT-Austin 7
2/1/2011 Binary image analysis: Binary images basic steps • Two pixel values • Convert the image into binary form – Foreground and background – Thresholding – Mark region(s) of interest • Clean up the thresholded image – Morphological operators M h l i l t • Extract separate blobs – Connected components • Describe the blobs with region properties Kristen Grauman, UT-Austin Thresholding Thresholding • Given a grayscale image or an intermediate matrix � • Grayscale -> binary mask threshold to create a binary output. • Useful if object of interest’s intensity distribution is distinct from background Example: edge detection Gradient magnitude fg_pix = find(gradient_mag > t); • Example http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FITZGIBBON/ Looking for pixels where gradient is strong. simplebinary.html Kristen Grauman, UT-Austin Thresholding Thresholding • Given a grayscale image or an intermediate matrix � • Given a grayscale image or an intermediate matrix � threshold to create a binary output. threshold to create a binary output. Example: background subtraction Example: intensity-based detection - = Looking for pixels that differ significantly fg_pix = find(im < 65); from the “empty” background. Looking for dark pixels fg_pix = find(diff > t); Kristen Grauman, UT-Austin Kristen Grauman, UT-Austin 8
2/1/2011 A nice case: bimodal intensity Thresholding histograms • Given a grayscale image or an intermediate matrix � threshold to create a binary output. Example: color-based detection Ideal histogram, light object on dark background Actual observed histogram with noise fg_pix = find(hue > t1 & hue < t2); Looking for pixels within a certain hue range. Images: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT2/node3.html Kristen Grauman, UT-Austin Issues Not so nice cases • What to do with “noisy” binary outputs? – Holes – Extra small fragments • How to demarcate multiple regions of interest? – Count objects – Compute further features per object Shapiro and Stockman Kristen Grauman, UT-Austin Dilation Morphological operators • Expands connected components • Change the shape of the foreground regions via • Grow features intersection/union operations between a • Fill holes scanning structuring element and binary image. • Useful to clean up result from thresholding • Useful to clean up result from thresholding • Basic operators are: – Dilation – Erosion Before dilation After dilation Kristen Grauman, UT-Austin 9
Recommend
More recommend
