Feature Point Feature-based approach: Detect and match feature - PowerPoint PPT Presentation

Wide Baseline Matching • Images taken by cameras that are far apart make the correspondence problem very difficult Feature Point • Feature-based approach: Detect and match feature Detec.on and Matching points in pairs of images Matching with Features Matching with Features • Problem 1: • Detect feature points – Detect the same point independently • Find corresponding pairs in both images no chance to match! We need a repeatable detector 1

Matching with Features Proper7es of an Ideal Feature • Local: features are local, so robust to occlusion and clu?er (no • Problem 2: prior segmenta.on) – For each point, correctly recognize the • Invariant (or covariant) to many kinds of geometric and corresponding point photometric transforma.ons • Robust: noise, blur, discre.za.on, compression, etc. do not have ? a big impact on the feature • Dis7nc7ve: individual features can be matched to a large database of objects • Quan7ty: many features can be generated for even small objects • Accurate: precise localiza.on We need a reliable and distinctive descriptor • Efficient: close to real-.me performance Problem 1: Detec7ng Good Feature Feature Detectors Points • Hessian • Harris • Lowe: SIFT (DoG) • Mikolajczyk & Schmid: Hessian/Harris-Laplacian/Affine • Tuytelaars & Van Gool: EBR and IBR • Matas: MSER • Kadir & Brady: Salient Regions • and many others 2

Harris Detector: Basic Idea Harris Corner Point Detector • We should recognize the point by looking through a small window • ShiXing a window in any direc&on should give a large change in response C. Harris, M. Stephens, “A Combined Corner and Edge Detector,” 1988 Harris Detector: Basic Idea Harris Detector: Deriva7on Change of intensity for a (small) shift by [ u,v ] in image I : ∑ = + + − 2 E ( u , v ) w ( x , y )[ I ( x u , y v ) I ( x , y )] x , y 0 0 ( x , y ) ∈ N ( x , y ) 0 0 Shifted Weighting Intensity intensity function “flat” region: “edge”: “corner”: no change in no change along significant change all directions the edge direction in all directions Weighting function w ( x,y ) = or 1 in window, 0 outside Gaussian 3

Harris Detector Harris Corner Detector Approximate using 1 st order Taylor series expansion of I : + + ≈ + + I ( x u , y v ) I ( x , y ) I u I v In summary, expanding E ( u , v ) in a Taylor series, we have, for x y small shifts, [ u,v ], a bilinear approximation: [ ] ⎡ ⎤ u ≈ + I ( x , y ) I I ⎢ ⎥ x y v ⎣ ⎦ ⎡ u ⎤ [ ] ≅ E ( u , v ) u v M ⎢ ⎥ Plugging this into previous formula, we get: v ⎣ ⎦ = + + E ( u , v ) Au 2 2 Cuv Bv 2 where M is a 2 x 2 matrix computed from image derivatives: ∑ ⎡ ⎤ ⎡ ⎤ = 2 A C u A w ( x , y ) I ( x , y ) [ ] = x E u v ( , ) u v ⎢ ⎥ ⎢ ⎥ ⎡ ⎤ 2 I I I = ∂ ∂ I x I ( x , y ) / x x , y ∑ C B v ⎣ ⎦ ⎣ ⎦ = x x y M w ( x , y ) ⎢ ⎥ ∑ = ∂ ∂ = 2 I y I ( x , y ) / y B w ( x , y ) I ( x , y ) 2 I I I ⎢ ⎥ ⎣ ⎦ y x , y x y y x , y = ∂ ∂ I x I ( x , y ) / x where ∑ = C w ( x , y ) I ( x , y ) I ( x , y ) Note: Sum computed over small neighborhood around given pixel = ∂ ∂ x y I y I ( x , y ) / y x , y Selec7ng Good Features Harris Corner Detector Image patch Intensity change in shifting window: eigenvalue analysis ⎡ ⎤ u λ max , λ min – eigenvalues of M [ ] ≅ E ( u , v ) u v M ⎢ ⎥ v ⎣ ⎦ SSD surface Eigenvector associated with λ max = direction of the fastest change in E Ellipse E ( u , v ) = const Eigenvector = direction of the ( λ max ) -1/2 slowest change in ( λ min ) -1/2 E λ 1 and λ 2 both large 4

Selec7ng Good Features Selec7ng Good Features SSD surface SSD surface large λ 1 , small λ 2 small λ 1 , small λ 2 Harris Corner Detector Harris Corner Detector Measure of corner response: λ 2 “Edge” λ 2 >> λ 1 “Corner” Classification of = − 2 R det M k ( trace M ) λ 1 and λ 2 both large, image points using λ 1 ~ λ 2 ; eigenvalues of M : E increases in all = λ λ det M directions 1 2 = λ + λ trace M 1 2 λ 1 and λ 2 both small; “Edge” E is almost constant in “Flat” λ 1 >> λ 2 all directions region k is an empirically-determined constant; e.g., k = 0.05 λ 1 5

Harris Corner Detector Harris Corner Detector: Algorithm 1. Find points with large corner response λ 2 “Edge” “Corner” func.on R • R depends only on R < 0 eigenvalues of M (i.e., R > threshold) • R is large for a corner R > 0 • R is negative with large 2. Keep the points that are local maxima of R magnitude for an edge (i.e., value of R at a pixel is greater than • | R | is small for a flat the values of R at all neighboring region “Flat” “Edge” pixels) |R| small R < 0 λ 1 Harris Detector: Example Harris Detector: Example Corner response R = λ 1 λ 2 – k ( λ 1 + λ 2 ) 2 6

Harris Detector: Example Harris Detector: Example Points with large corner response: R > threshold Keep only the points that are local maxima of R Harris Detector: Example Harris Detector: Example Interest points extracted with Harris (~ 500 points) 7

Harris Detector: Example Harris Detector: Summary • Average intensity change in direc.on [ u,v ] can be expressed (approximately) in bilinear form: ⎡ u ⎤ [ ] ≅ E ( u , v ) u v M ⎢ ⎥ v ⎣ ⎦ • Describe a point in terms of the eigenvalues of M : measure of “cornerness”: ( ) 2 = λ λ − λ + λ R k 1 2 1 2 • A good (corner) point should have a large intensity change in most direc&ons , i.e., R should be a large posi.ve value Harris Detector Proper7es Harris Detector Proper7es But not invariant to image scale Rota7on invariance Ellipse rotates but its shape (i.e., eigenvalues) remains the same Fine scale: All points will Coarse scale: Corner Corner response R is invariant to image rotation be classified as edges 8

Harris Detector: Scale Harris Detector Proper7es Quality of Harris detector for different scale changes Repeatability rate: # correct correspondences # possible correspondences R min = 0 R min = 1500 C. Schmid et al., “Evaluation of Interest Point Detectors,” IJCV 2000 Invariant Local Features Models of Image Change Goal: Detect the same interest points regardless of • Geometry image changes due to transla7on , rota7on , scale , and viewpoint – Rota.on – Similarity (rota.on + uniform scale) – Affine (scale dependent on direc.on) valid for: orthographic camera, locally planar object • Photometry – Affine intensity change ( I → a I + b ) 40 9

Scale Invariant Detec7on Scale Invariant Detec7on • Consider regions (e.g., circles) of different sizes Problem: How do we choose corresponding circles around a point independently in each image? • Regions of corresponding sizes will look the same in both images Scale Invariant Detec7on Scale Invariant Detec7on Solu.on: Use the local maximum of this function Key idea : – Design a func.on on the region (circle) that is “scale invariant,” i.e., the same for corresponding Observation : The region size, for which the maximum is regions, even if they are at different scales achieved, should be invariant to image scale Example : Average intensity. For corresponding Important: This scale invariant region size regions (even of different sizes) it will be the same is found in each image independently ! – For a point in one image, we can consider it as a function of region size (i.e., circle radius) Image 1 Image 1 f f f f Image 2 Image 2 scale = 1/2 scale = 1/2 s 1 s 2 region size region size region size region size 10

Automa7c Scale Selec7on Automa7c Scale Selec7on Function responses for increasing scale Scale trace (signature) Lindeberg et al ., 1996 σ σ f ( I ( x , )) f ( I ( x , )) i … i i … i 1 m 1 m One possible definition of the function f : Absolute value of the Laplacian of Gaussian ( ∇ 2 G) (or DoG) Automa7c Scale Selec7on Automa7c Scale Selec7on Function responses for increasing scale Function responses for increasing scale Scale trace (signature) Scale trace (signature) σ σ f ( I ( x , )) f ( I ( x , )) i … i i … i 1 m 1 m 11

Automa7c Scale Selec7on Automa7c Scale Selec7on Function responses for increasing scale Function responses for increasing scale Scale trace (signature) Scale trace (signature) σ f ( I ( x , )) σ f ( I ( x , )) i … i i … i 1 m 1 m Automa7c Scale Selec7on Automa7c Scale Selec7on Function responses for increasing scale Function responses for increasing scale Scale trace (signature) Scale trace (signature) σ f ( I ( x , )) i … i 1 m σ f ( I ( x , )) i … i 1 m ′ σ f ( I ( x , )) i … i 1 m 12