Overview Overview Introduction to local features Introduction to - PowerPoint PPT Presentation

Overview Overview • Introduction to local features Introduction to local features • Harris interest points + SSD, ZNCC, SIFT H i i t t i t + SSD ZNCC SIFT • Scale & affine invariant interest point detectors S l & ffi i i t i t t i t d t t • Evaluation and comparison of different detectors E l i d i f diff d • Region descriptors and their performance

Scale invariance - motivation Scale invariance motivation • Description regions have to be adapted to scale changes • Interest points have to be repeatable for scale changes I t t i t h t b t bl f l h

Harris detector + scale changes Harris detector + scale changes Repeatability rate     | | {( {( , ) ) | | ( ( ( ( ), ) ) ) } } | | a a b b a a b b dist dist H H   ( ) i i i i R max(| |, | |) a b i i

Scale adaptation Scale adaptation Scale change between two images Scale change bet een t o images             x x sx         1 2 1 I I I       1 2 2       y y sy 1 2 1 Scale adapted derivative calculation

Scale adaptation Scale adaptation Scale change between two images Scale change bet een t o images             x x sx         1 2 1 I I I       1 2 2       y y sy 1 2 1 Scale adapted derivative calculation         x x           1 2 ( ) ( ) n n I G s s I G s s     1 2   i i i i         y 1 y 1 n n 1 2

Scale adaptation Scale adaptation     2     ( ( ) ) ( ( ) ) L L L L L L ~   ( ) x x y   G 2   ( ) ( )   L L L   x y y (  ) where are the derivatives with Gaussian convolution L i

Scale adaptation Scale adaptation     2     ( ( ) ) ( ( ) ) L L L L L L ~   ( ) x x y   G 2   ( ) ( )   L L L   x y y (  ) where are the derivatives with Gaussian convolution L i Scale adapted auto correlation matrix Scale adapted auto-correlation matrix     2 ( ) ( ) L s L L s ~    2 2 ( ( ) ) x x x x y y     s G G s   2 ( ) ( ) L L s L s   x y y

Harris detector – adaptation to scale Harris detector adaptation to scale     ( ) {( , ) | ( ( ), ) } R a b dist H a b i i i i

Multi-scale matching algorithm Multi scale matching algorithm  1 s  3 s  5 s

Multi-scale matching algorithm Multi scale matching algorithm  1 s 8 matches 8 matches

Multi-scale matching algorithm Multi scale matching algorithm Robust estimation of a global  1 s affine transformation 3 matches 3 matches

Multi-scale matching algorithm Multi scale matching algorithm  1 s 3 matches 3 matches  3 s 4 4 matches t h

Multi-scale matching algorithm Multi scale matching algorithm  1 s 3 matches 3 matches  3 s 4 4 matches t h highest number of matches  5 correct scale s 16 matches

Matching results Matching results Scale change of 5 7 Scale change of 5.7

Matching results Matching results 100% 100% correct matches (13 matches) t t h (13 t h )

Scale selection Scale selection • We want to find the characteristic scale of the blob by convolving it with Laplacians at several scales and looking for the maximum response • However, Laplacian response decays as scale H L l i d l increases: original signal increasing σ (radius=8) Why does this happen?

Scale normalization Scale normalization • The response of a derivative of Gaussian filter to a perfect The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases 1 1   2

Scale normalization Scale normalization • The response of a derivative of Gaussian filter to a perfect The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases • To keep response the same (scale invariant) must • To keep response the same (scale-invariant), must multiply Gaussian derivative by σ • Laplacian is the second Gaussian derivative so it must be Laplacian is the second Gaussian derivative, so it must be multiplied by σ 2

Effect of scale normalization Effect of scale normalization Original signal Unnormalized Laplacian response Scale-normalized Laplacian response maximum

Blob detection in 2D Blob detection in 2D • Laplacian of Gaussian: Circularly symmetric operator for Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D   2 2 g g    2 g g   2 2 2 2 x y

Blob detection in 2D Blob detection in 2D • Laplacian of Gaussian: Circularly symmetric operator for Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D       2 2 g g       2 2 g g Scale-normalized:     norm norm     2 2 2 2     x y

Scale selection Scale selection • The 2D Laplacian is given by The 2D Laplacian is given by 2 2 2       2 2 2 ( ) / 2 (up to scale) ( 2 ) x y x y e • For a binary circle of radius r, the Laplacian achieves a y , p maximum at   / 2 r e n respons Laplacian r / 2 scale ( σ ) r image

Characteristic scale Characteristic scale • We define the characteristic scale as the scale that We define the characteristic scale as the scale that produces peak of Laplacian response characteristic scale T. Lindeberg (1998). Feature detection with automatic scale selection. International Journal of Computer Vision 30 (2): pp 77--116.

Scale selection Scale selection • For a point compute a value (gradient, Laplacian etc.) at For a point compute a value (gradient Laplacian etc ) at several scales • Normalization of the values with the scale factor Normali ation of the al es ith the scale factor  e.g. Laplacian | 2 ( ) | s L L xx yy • Select scale at the maximum → characteristic scale  s  | 2 ( ) | s L L xx yy scale • Exp. results show that the Laplacian gives best results E lt h th t th L l i i b t lt

Scale selection Scale selection • Scale invariance of the characteristic scale Scale invariance of the characteristic scale s p. norm. La scale

Scale selection Scale selection • Scale invariance of the characteristic scale Scale invariance of the characteristic scale s p. p. norm. La norm. La scale scale       • Relation between characteristic scales s s s 1 2

Scale-invariant detectors Scale invariant detectors • Harris-Laplace (Mikolajczyk & Schmid’01) Harris Laplace (Mikolajczyk & Schmid’01) • Laplacian detector (Lindeberg’98) • Difference of Gaussian (Lowe’99) Harris-Laplace Laplacian

Harris-Laplace Harris Laplace multi-scale Harris points selection of points at maximum of Laplacian invariant points + associated regions [Mikolajczyk & Schmid’01]

Matching results Matching results 213 / 190 detected interest points 213 / 190 detected interest points

Matching results Matching results 58 points are initially matched 58 points are initially matched

Matching results Matching results 32 points are matched after verification – all correct 32 points are matched after verification all correct

LOG detector LOG detector Convolve image with scale Convolve image with scale- normalized Laplacian at several scales several scales     2 ( ( ) ( )) LOG s G G xx yy Detection of maxima and minima of Laplacian in scale space

Hessian detector Hessian detector   L L  xx xy ( ) Hessian matrix   H x L L L L     xy yy   2 Determinant of Hessian matrix DET L L L xx yy xy Penalizes/eliminates long structures g  with small derivative in a single direction

Efficient implementation Efficient implementation • Difference of Gaussian (DOG) approximates the Difference of Gaussian (DOG) approximates the     Laplacian ( ) ( ) DOG G k G • Error due to the approximation

DOG detector DOG detector • Fast computation scale space processed one octave at a • Fast computation, scale space processed one octave at a time David G. Lowe. "Distinctive image features from scale-invariant keypoints.”I JCV 60 (2).

Local features - overview Local features overview • Scale invariant interest points • Affine invariant interest points • Evaluation of interest points • Descriptors and their evaluation