Administrivia Homework 2 due today CMPSCI 370: Intro to Computer - PowerPoint PPT Presentation

Administrivia • Homework 2 due today CMPSCI 370: Intro to Computer Vision • Homework 3 will be posted later this week Image processing: #4 • will be due March 10 Corner detection • No class or honors section next Tuesday, 3/1 University of Massachusetts, Amherst February 22/24, 2015 • I am out of town to attend a CVPR program committee meeting Instructor: Subhransu Maji • Honors section will meet today 2 1 2 Overview Feature extraction: Corners • Edge detection 9300 Harris Corners Pkwy, Charlotte, NC • Derivative filters q image E = G 2 x + G 2 y • Corner detection [today] • What are corners? • Why detect corners? • Harris corner detector 3 Slide credit: L. Lazebnik 4 3 4

Why extract features? Why extract features? • Motivation: panorama stitching • Motivation: panorama stitching • We have two images – how do we combine them? • We have two images – how do we combine them? Step 1: extract features Step 2: match features Slide credit: L. Lazebnik 5 Slide credit: L. Lazebnik 6 5 6 Why extract features? Characteristics of good features • Motivation: panorama stitching • We have two images – how do we combine them? • Repeatability • The same feature can be found in several images despite geometric and photometric transformations • Saliency • Each feature is distinctive • Compactness and efficiency Step 1: extract features • Many fewer features than image pixels • Locality Step 2: match features • A feature occupies a relatively small area of the image; robust to clutter and occlusion Step 3: align images Slide credit: L. Lazebnik 7 Slide credit: L. Lazebnik 8 7 8

Applications A hard feature matching problem Feature points are used for: • Image alignment • 3D reconstruction • Motion tracking • Robot navigation • Indexing and database retrieval • Object recognition NASA Mars Rover images Slide credit: L. Lazebnik 9 10 9 10 Answer below (look for tiny colored squares…) Corner detection: Attempt one • A corner is the intersection of two edges • We know how to detect edges • Corner detector v1 • Detect edges in images (G x and G y ) • Find places where both G x and G y are high • Problem: this also finds slanted edges NASA Mars Rover images with SIFT feature matches 
 Figure by Noah Snavely 11 12 11 12

Corner detection: Attempt two Corner detection: Attempt two • We should easily recognize the corners by looking through • We should easily recognize the corners by looking through a small window a small window • Shifting a window in any direction should give a large • Shifting a window in any direction should give a large change in intensity at a corner change in intensity at a corner “flat” region:
 “edge”:
 “corner”:
 “flat” region:
 “edge”:
 “corner”:
 no change in no change along significant no change in no change along significant all directions the edge change in all all directions the edge change in all direction directions direction directions 13 14 13 14 Corner detection: Attempt two Corner detection: details • Implementing a corner detector • One way to measure change is to consider the sum of squared-differences. Thus, the change for a shift u, v • Fix the size of the patch (window size) and for an image window W is • What happens if the window is too small? or too large? [ I ( x + u, y + v ) − I ( x, y )] 2 X E ( u, v ) = • Consider eight directions, and measure how much does a patch change in each direction for each location of the image ( x,y ) ∈ W W ( u, v ) W I ( x, y ) 15 16 15 16

Corner detection: details Corner detection: details • Assume for now that the window size W is one pixel • Generalize to any window size W • How can we compute E(u,v) for every pixel in the image? • Given D(1,0) for each pixel, how can you compute the sum- of-squared-differences within a window? [ I ( x + u, y + v ) − I ( x, y )] 2 X E ( u, v ) = • Answer: filter the output with a box kernel ( x,y ) ∈ W EdgeScore(1 , 0) = f W box ∗ D (1 , 0) 2 • As an example consider E(1, 0), i.e. image shifted by one pixel to the right. This can be computed as convolution with • Better version: Convolve with a Gaussian kernel f and squaring the result, • More emphasis on the center than the boundary for each window   0 0 0 D (1 , 0) = I ∗ 0 − 1 1 Matlab: imfilter(I, f, ‘same’)   0 0 0 f 17 18 17 18 Corner detection: algorithm Corner score example image Box filter • Corner detector v2 • Input: image I , window size W • Output : corner score for each pixel • Initialize : score = zeros(size( I )) • for u = -1:1 % horizontal shifts • for v = -1:1 % vertical shifts W = 5 W = 10 Gaussian filter • imdiff = imfilter( I , f(u,v) , ‘same’); % compute difference • score = score + imfilter(imdiff.^2, ones( W ), ‘same’); % weighted squared difference • Here, f(u,v) is the filter to compute difference between a pixel and another shifted by (u,v) σ = 2 σ = 5 19 20 19 20

Corner detection: remaining steps Corner detection: complete algorithm • Corner detector v2 • Threshold the score (user specified) • Input: image I , window size W , threshold T • Find the peaks of responses: a pixel is a peak if it has a higher value than all the pixels in its neighborhood • Output : corner score for each pixel • 4 connectivity, 8 connectivity, etc • Initialize : score = zeros(size( I )) • The Matlab command imregionalmax computes a binary • for u = -1:1 % horizontal shifts image which is 1 at the peak and 0 elsewhere • for v = -1:1 % vertical shifts • Note that there can be multiple pixels at each peak. We can • imdiff = imfilter( I, f(u,v), ‘same’); % compute difference simply pick one for each connected component. • score = score + imfilter(imdiff.^2, ones( W ), ‘same’); % weighted squared difference • score = nms (score > T ); % perform non-maximum suppression peak • locs = find(score > 0); % indices of the corners • [cy, cx] = ind2sub(size(score), lots)); % their x, y locations 21 22 21 22 Corner detection: summary Corner detection: Mathematics • The algorithm presented is the basis for the Harris corner Recall that the change in appearance of window W for the detector shift [ u,v ]: • One weakness is that we only considered only 8 directions [ I ( x + u, y + v ) − I ( x, y )] 2 X E ( u, v ) = for computing the differences. Why not 16? or 32? ( x,y ) ∈ W • The Harris detector generalizes this to “all” directions • More directions improve performance I ( x, y ) E ( u, v ) • Can be thought of the limit of the earlier algorithm as the number of directions approach infinity! • However, it does this without explicitly enumerating directions. To E (3,2) understand how we need some math. 23 24 23 24

Corner detection: Mathematics Corner detection: Mathematics Recall that the change in appearance of window W for the Recall that the change in appearance of window W for the shift [ u,v ]: shift [ u,v ]: [ I ( x + u, y + v ) − I ( x, y )] 2 [ I ( x + u, y + v ) − I ( x, y )] 2 X X E ( u, v ) = E ( u, v ) = ( x,y ) ∈ W ( x,y ) ∈ W We want to find out how this function behaves for I ( x, y ) small shifts E ( u, v ) E ( u, v ) E (0,0) 25 26 25 26 Corner detection: Mathematics Corner Detection: Mathematics • First-order Taylor approximation for small motions [ u , v ]: The quadratic approximation can be written as u I ( x + u, y + v ) = I ( x, y ) + I x u + I y v & # [ ] E ( u , v ) u v M ≈ $ ! v • Let’s plug this into E ( u , v ) % " X [ I ( x + u, y + v ) � I ( x, y )] 2 E ( u, v ) = where M is a second moment matrix computed from image ( x,y ) ∈ W derivatives: X [ I ( x, y ) + I x u + I y v � I ( x, y )] 2 ' 2 I I I ∑ ∑ & # ( x,y ) ∈ W x x y $ ! x , y x , y M [ I x u + I y v ] 2 = X = $ 2 ! I I I ∑ ∑ x y y $ ! ( x,y ) ∈ W % " x , y x , y x u 2 + I x I y uv + I y I x uv + I 2 X I 2 y v 2 ⇤ ⇥ = (the sums are over all the pixels in the window W ) ( x,y ) ∈ W 27 28 27 28