CS4495/6495 Introduction to Computer Vision 4A-L2 Finding corners - PowerPoint PPT Presentation

CS4495/6495 Introduction to Computer Vision 4A-L2 Finding corners

Feature points

Characteristics of good features Repeatability/Precision • The same feature can be found in several images despite geometric and photometric transformations

Characteristics of good features Saliency/Matchability • Each feature has a distinctive description

Characteristics of good features Compactness and efficiency • Many fewer features than image pixels

Characteristics of good features Locality • A feature occupies a relatively small area of the image; robust to clutter and occlusion

Corner Detection: Basic Idea “corner”: “edge”: “flat” region: significant change no change no change in in all directions along the edge all directions with small shift direction Source: A. Efros

Finding Corners • Key property: in the region around a corner, image gradient has two or more dominant directions

Finding Corners C. Harris and M. Stephens. "A Combined Corner and Edge Detector,” Proceedings of the 4th Alvey Vision Conference : 1988

Corner Detection: Mathematics Change in appearance for the shift [ u,v ]:    2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , Window Shifted Intensity function intensity Window function w(x,y) = or 1 in window, Gaussian 0 outside Source: R. Szeliski

Corner Detection: Mathematics Change in appearance for the shift [ u,v ]:    2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , I ( x , y ) E ( u , v ) E (3,2) E (0,0)

Corner Detection: Mathematics Change in appearance for the shift [ u,v ]:    2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , We want to find out how this function behaves for small shifts (u,v near 0,0)

Corner Detection: Mathematics Change in appearance for the shift [ u,v ]:    2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , Second-order Taylor expansion of E ( u , v ) about (0,0) (local quadratic approximation for small u,v ):

Corner Detection: Mathematics Change in appearance for the shift [ u,v ]:    2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , 2 d F d F (0 ) 1 (0 )       2 F x F x x ( ) (0 ) · · 2 d x d x 2

Corner Detection: Mathematics Change in appearance for the shift [ u,v ]:    2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , 2 d F d F (0 ) 1 (0 )       2 F x F x x ( ) (0 ) · · 2 d x d x 2       E E E u (0, 0 ) (0, 0 ) (0, 0 ) 1 u u u u v    E u v E u v u v ( , ) (0, 0 ) [ ] [ ]       E E E v (0, 0 ) (0, 0 ) (0, 0 )     2   v u v vv

   2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , Second-order Taylor expansion of E ( u , v ) about (0,0):      1  E E E (0, 0 ) (0, 0 ) (0, 0 ) u    E u v E u v u v ( , ) (0, 0 ) [ ] [ ] u u u u v       E E E v (0, 0 ) (0, 0 ) (0, 0 )       2 v u v vv

   2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , Second-order Taylor expansion of E ( u , v ) about (0,0):      1  E E E (0, 0 ) (0, 0 ) (0, 0 ) u    E u v E u v u v ( , ) (0, 0 ) [ ] [ ] u u u u v       E E E v (0, 0 ) (0, 0 ) (0, 0 )       2 v u v vv Need these derivatives…

   2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , Second-order Taylor expansion of E ( u , v ) about (0,0):          I E u u v w x y I x u y v I x y x u y v ( , ) 2 ( , ) ( , ) ( , ) ( , ) x x , y

   2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , Second-order Taylor expansion of E ( u , v ) about (0,0):       I I E u u u v w x y x u y v x u y v ( , ) 2 ( , ) ( , ) ( , ) x x x , y          I w x y I x u y v I x y x u y v 2 ( , ) ( , ) ( , ) ( , ) xx x , y

   2     E u v w x y I x u y v I x y ( , ) ( , ) ( , ) ( , ) x y , Second-order Taylor expansion of E ( u , v ) about (0,0):       I I E u v w x y x u y v x u y v ( , ) 2 ( , ) ( , ) ( , ) u v y x x y ,          I w x y I x u y v I x y x u y v 2 ( , ) ( , ) ( , ) ( , ) xy x y ,

Second-order Taylor expansion of E ( u , v ) about (0,0):       1 E E E u (0, 0 ) (0, 0 ) (0, 0 )    E u v E u v u v ( , ) (0, 0 ) [ ] u [ ] u u u v       E E E v (0, 0 ) (0, 0 ) (0, 0 )       2 v u v vv          E u v w x y I x u y v I x y I x u y v ( , ) 2 ( , ) ( , ) ( , ) ( , ) u x x , y       E u v w x y I x u y v I x u y v ( , ) 2 ( , ) ( , ) ( , ) u u x x x y ,          w x y I x u y v I x y I x u y v 2 ( , ) ( , ) ( , ) ( , ) xx x , y       E u v w x y I x u y v I x u y v ( , ) 2 ( , ) ( , ) ( , ) u v y x x y ,          w x y I x u y v I x y I x u y v 2 ( , ) ( , ) ( , ) ( , ) xy x y ,

Evaluate E and its derivatives at (0,0) : = 0       1 E E E u (0, 0 ) (0, 0 ) (0, 0 )    E u v E u v u v ( , ) (0, 0 ) [ ] [ ] u u u u v       E E E v (0, 0 ) (0, 0 ) (0, 0 )       2 v u v vv      E w x y I x y I x y I x y (0, 0 ) 2 ( , ) ( , ) ( , ) ( , ) u x = 0 x , y   E w x y I x y I x y (0, 0 ) 2 ( , ) ( , ) ( , ) u u x x x y = 0 ,      w x y I x y I x y I x y 2 ( , ) ( , ) ( , ) ( , ) xx x , y   E w x y I x y I x y (0, 0 ) 2 ( , ) ( , ) ( , ) u v y x = 0 x y ,      w x y I x y I x y I x y 2 ( , ) ( , ) ( , ) ( , ) xy x y ,

Second-order Taylor expansion of E ( u , v ) about (0,0):   1     E E E u (0, 0 ) (0, 0 ) (0, 0 )    E u v E u v u v ( , ) (0, 0 ) [ ] [ ] u u u u v       E E E v    ( 0, 0 )   (0, 0 ) (0, 0 )  2 v u v vv    E E w x y I x y I x y (0, 0 ) 0 (0, 0 ) 2 ( , ) ( , ) ( , ) u u x x x , y    E E w x y I x y I x y (0, 0 ) 0 (0, 0 ) 2 ( , ) ( , ) ( , ) vv y y u x , y    E E w x y I x y I x y (0, 0 ) 0 (0, 0 ) 2 ( , ) ( , ) ( , ) u v x y v x , y

Second-order Taylor expansion of E ( u , v ) about (0,0):      2 w x y I x y w x y I x y I x y ( , ) ( , ) ( , ) ( , ) ( , )  u x x y  E u v u v  x y x y   ( , ) [ ] , ,    2 w x y I x y I x y w x y I x y v ( , ) ( , ) ( , ) ( , ) ( , )       x y y x y x y , ,    E E w x y I x y I x y (0, 0 ) 0 (0, 0 ) 2 ( , ) ( , ) ( , ) u u x x x , y    E E w x y I x y I x y (0, 0 ) 0 (0, 0 ) 2 ( , ) ( , ) ( , ) vv y y u x , y    E E w x y I x y I x y (0, 0 ) 0 (0, 0 ) 2 ( , ) ( , ) ( , ) u v x y v x , y

Corner Detection: Mathematics The quadratic approximation simplifies to   u  E u v u v M ( , ) [ ]   v   where M is a second moment matrix computed from image derivatives:   2 I I I  x x y    M w x y ( , ) 2 I I I     x y , x y y

The second moment matrix M:   2 I I I  x x y    M w x y ( , ) 2 I I I     x y , x y y Each product is a rank 1 2x2 Can be written (without the weight):         I I I I I   x   T       x x x y    M I I I I   ( )       x y I  I I I I        y x y y y

Interpreting the second moment matrix The surface E ( u , v ) is locally approximated by a quadratic form.   u  E u v u v M ( , ) [ ]   v     2 I I I   M w x y x x y ( , )   2 I I I   x y , x y y