Local features: detection and description detection and description - PowerPoint PPT Presentation

Local features: detection and description detection and description Kristen Grauman UT ‐ Austin UT Austin

Local invariant features – Detection of interest points – Detection of interest points • Harris corner detection • Scale invariant blob detection: LoG • Scale invariant blob detection: LoG – Description of local patches • SIFT : Histograms of oriented gradients SIFT Hi t f i t d di t

Today

Local features: main components 1) Detection: Identify the interest points p 2) Description :Extract vector 2) D i ti E t t t  ( 1 ) ( 1 )  x [ x , , x ] feature descriptor 1 1 d surrounding each interest g point.  ( ( 2 2 ) ) ( ( 2 2 ) )  x [ x , , x ] 2 1 d 3) Matching: Determine correspondence between p descriptors in two views Kristen Grauman

Local features: desired properties • Repeatability p y – The same feature can be found in several images despite geometric and photometric transformations • Saliency S li – Each feature has a distinctive description • Compactness and efficiency • Compactness and efficiency – Many fewer features than image pixels • Locality oca ty – A feature occupies a relatively small area of the image; robust to clutter and occlusion Kristen Grauman

Goal: interest operator repeatability p p y • We want to detect (at least some of) the same points in both images. i t i b th i No chance to find true matches! N h t fi d t t h ! • Yet we have to be able to run the detection procedure independently per image.

Goal: descriptor distinctiveness p • We want to be able to reliably determine which point goes with which. hi h i t ith hi h ? • Must provide some invariance to geometric and photometric differences between the two p views.

Local features: main components 1) Detection: Identify the interest points p 2) Description :Extract vector 2) D i ti E t t t feature descriptor surrounding each interest g point. 3) Matching: Determine correspondence between p descriptors in two views Kristen Grauman

• What points would you choose? • What points would you choose? Kristen Grauman

Corners as distinctive interest points We should easily recognize the point by looking through a small window Shifti Shifting a window in any direction should give i d i h ld i di ti a large change in intensity “flat” region: “edge”: “corner”: no change in no change in no change no change significant significant all directions along the edge change in all direction directions Slide credit: Alyosha Efros, Darya Frolova, Denis Simakov

Corners as distinctive interest points   I I I I    x x x y     M w ( ( x , , y y ) ) I I I I I I I I   x y y y 2 x 2 matrix of image derivatives (averaged in 2 2 t i f i d i ti ( d i neighborhood of a point).     I I I I       Notation: Notation: I I x I y I I I I I     x y x y x y Kristen Grauman

What does this matrix reveal? First, consider an axis-aligned corner:

What does this matrix reveal? First, consider an axis-aligned corner:      2 0 I I I     x x y 1       M   2 2         I I I I I I 0 0     x y y 2 This means dominant gradient directions align with This means dominant gradient directions align with x or y axis Look for locations where both λ ’s are large Look for locations where both λ s are large. If either λ is close to 0, then this is not corner-like. What if we have a corner that is not aligned with the image axes?

What does this matrix reveal?    0  Since M is symmetric, we have 1 T   M X X      0 2 2   Mx x i i i The eigenvalues of M reveal the amount of intensity change in the two principal orthogonal gradient directions in the window gradient directions in the window.

Corner response function “edge”: edge : “corner”: corner : “flat” region flat region  1 >>  2  1 and  2 are  1 and  2 are large,  1 ~  2 ; small;  2 >>  1 2 1         ' 2 f ( ) 1 2 1 2    det( M ) trace ( M

Harris corner detector 1) Compute M matrix for each image window to get their cornerness scores. 2) Find points whose surrounding window gave large corner response ( f > threshold) 3) Take the points of local maxima, i.e., perform ) f f non-maximum suppression

Example of Harris application

Example of Harris application C Compute corner response f t f

Example of Harris application Find points with large corner response: f > threshold f > threshold Fi d i t ith l

Example of Harris application T k Take only the points of local maxima of f l th i t f l l i f f

Example of Harris application

Example of Harris application Kristen Grauman

Example of Harris application Compute corner response at every pixel. Kristen Grauman

Example of Harris application Kristen Grauman

Properties of the Harris corner detector Rotation invariant? Yes    0  1 T   M X X      0 2 2 Scale invariant?

Properties of the Harris corner detector Rotation invariant? Yes No Scale invariant? All points will be Corner ! classified as edges

Scale invariant interest points How can we independently select interest points in each image, such that the detections are repeatable across different scales? across different scales? Kristen Grauman

Automatic Scale Selection How to find corresponding patch sizes independently ?      f ( I ( x , )) f ( I ( x , ))   i i i i 1 m 1 m Intuition: • Find scale that gives local maxima of some function g f in both position and scale. K. Grauman, B. Leibe

Automatic Scale Selection • Function responses for increasing scale (scale signature)    f ( I ( x , )) f ( I ( x , ))  i i i  i 1 m 1 m K. Grauman, B. Leibe

Automatic Scale Selection • Function responses for increasing scale (scale signature)    f ( I ( x , )) f ( I ( x , ))  i i i  i 1 m 1 m K. Grauman, B. Leibe

Automatic Scale Selection • Function responses for increasing scale (scale signature)    f ( I ( x , )) f ( I ( x , ))  i i i  i 1 m 1 m K. Grauman, B. Leibe

Automatic Scale Selection • Function responses for increasing scale (scale signature)    f ( I ( x , )) f ( I ( x , ))  i i i  i 1 m 1 m K. Grauman, B. Leibe

Automatic Scale Selection • Function responses for increasing scale (scale signature)    f ( I ( x , )) f ( I ( x , ))  i i i  i 1 m 1 m K. Grauman, B. Leibe

Automatic Scale Selection • Function responses for increasing scale (scale signature)     f ( I ( x , )) f ( I ( x , )) i  i i  i 1 m 1 m K. Grauman, B. Leibe

What can be the “signature” function?

Blob detection in 2D Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D   2 2 g g       2 g g   2 2 x y

Blob detection in 2D: scale selection   2 2 g g    Laplacian-of-Gaussian = “blob” detector 2 g   2 2 x y s scales filter img2 img1 img3 Bastian Leibe

Blob detection in 2D We define the characteristic scale as the scale that produces peak of Laplacian response characteristic scale characteristic scale Slide credit: Lana Lazebnik

Example Original image at ¾ the size Kristen Grauman

Original image at ¾ the size Kristen Grauman

Kristen Grauman

Kristen Grauman

Kristen Grauman

Kristen Grauman

Kristen Grauman

Scale invariant interest points Interest points are local maxima in both position and scale.     scale       L L ( ( ) ) L L ( ( ) )   xx yy    List of (x, y, σ )   Squared filter response maps Kristen Grauman

Scale-space blob detector: Example Image credit: Lana Lazebnik

Technical detail We can approximate the Laplacian with a difference of Gaussians; more efficient to i implement. l t              2 L L G G ( , , ( , , x y x y ) ) G G ( , , ( , , x y x y ) ) xx yy (Laplacian)     DoG D G G G x y k ( ( , , k ) ) G x y G ( , , ( ) ) (Difference of Gaussians)