Combining Images Combining Images Blending Seam Carving Corner - PowerPoint PPT Presentation

Combining Images • Combining Images – Blending – Seam Carving Corner Detection • Today: Use “similar” features Based on Richard Szeliski Notes (textbook author) Courtesy: Irfan Essa Challenge: Detect feature points What are Good Features? in both Images • Distinctive • Invariant to different acquisition conditions – Different view-points, – different illuminations, – different cameras (wide angle, different looking perspective) Y • Goal - Find points in an image that can be: perspective scale • Found in other images n o t i a l n s a rotation r t • Found precisely - well localized affine X • Found reliably - well matched Shi and C. Tomasi (June 1994). "Good Features to Track, ” (citeseer)

Find Corners Corner Detection: Basic Idea • Recognize • Key property: In the region around a the point by corner, image gradient has looking through a – two or more dominant directions small window • Corners are repeatable and distinctive � edge � : � corner � : � flat � region: • Shifting a no change significant no change in along the change in all directions window in edge all any direction direction directions should give a large change in intensity 2 Basic Idea E ( u , v ) = w ( x , y ) I ( x + u , y + v ) − I ( x , y ) ⎡ ⎤ ∑ ⎣ ⎦ x , y I(x, y) E(u, v) Change of intensity for the shift [ u,v ]: appearance 2 E ( u , v ) = w ( x , y ) I ( x + u , y + v ) − I ( x , y ) ⎡ ⎤ ∑ ⎣ ⎦ by shifting x , y the window Window Shifted Intensity function intensity E(0,0) E(3,2) Slide motivated by Alyosha Efros • Computation of the change in appearance Window function w(x,y) = or by shifting the window by u,v: 1 in window, 0 outside Gaussian H. Moravec (1980). "Obstacle Avoidance and Navigation in the Real World by a Seeing Robot Rover"

Intuition • Convert the above formula (uses Taylor Series expansion) – • Treat gradient vectors as a set of (dx,dy)  � u points with a center of mass defined as being ⇥ ⇤ E ( u, v ) ≈ u v M v at (0,0).  I 2 � I x I y M X M = w ( x, y ) x • Fit an ellipse to that set of points via scatter I 2 I x I y y x,y matrix • Analyze ellipse parameters for varying case. Familiar? These are just products of Component of the gradients I x , I y , • M is matrix computed from image derivatives – M is also called structure tensor λ 1~ λ 2 = small λ 1~ λ 2 = large λ 1 large; λ 2 = small Courtesy: R. Collins, Penn State University Courtesy: R. Collins, Penn State University

Harris Detector: Mathematics Harris Detector: Mathematics Intensity change in shifting window: eigenvalue analysis Classification of λ 2 � Edge �� u image points using λ 2 >> λ 1 ⎡ ⎤ � Corner � λ 1 , λ 2 – eigenvalues of M E u v ( , ) u v , M [ ] ≅ eigenvalues of M : ⎢ ⎥ λ 1 and λ 2 are large, v ⎣ ⎦ λ 1 ~ λ 2 ; direction of the E increases in all fastest change directions direction of the slowest change Ellipse E(u,v) = const ( λ max ) -1/2 λ 1 and λ 2 are small; � Edge �� E is almost constant ( λ min ) -1/2 � Flat � λ 1 >> λ 2 in all directions region λ 1 Harris Detector: Mathematics Harris Detector: Mathematics Measure of corner response: λ 2 � Edge �� � Corner � • R depends only on R < 0 2 R det M k ( trace M ) = − eigenvalues of M • R is large for a corner R > 0 • R is negative with large det M = λ λ 1 2 magnitude for an edge trace M = λ + λ • | R | is small for a flat 1 2 region � Flat � � Edge �� ( k – empirical constant, k = 0.04-0.06) |R| small R < 0 λ 1

Harris Detector: Workflow Harris Detector (Preview) `Detect Corners’ • The Algorithm: – Compute Gaussian derivatives at each pixel – Compute matrix M in a ga – Find points with large corner response function R ( R > threshold) – Take the points of local maxima of R (we are not matching corner to ‘another’ image’s corner Yet) C. Harris and M. Stephens (1988). "A combined corner and edge detector" Proceedings of the 4th Alvey Vision Conference. pp 147--151 Harris Detector: Workflow Harris Detector: Workflow Compute corner response R Find points with large corner response: R> threshold

Harris Detector: Workflow Harris Detector: Workflow Take only the points of local maxima of R Harris Detector: Some Harris Detector: Summary Properties • Rotation invariance • Average intensity change in direction [ u,v ] can be expressed as a bilinear form: u ⎡ ⎤ E u v ( , ) [ u v , ] M ≅ ⎢ ⎥ v ⎣ ⎦ • Describe a point in terms of eigenvalues of M : measure of corner response 2 R k ( ) = λλ − λ + λ Ellipse rotates but its shape (i.e. eigenvalues) 1 2 1 2 remains the same • A good (corner) point should have a large intensity change in all directions , i.e. R should be large Corner response R is invariant to image rotation positive

Harris Detector: Some Harris Detector: Some Properties Properties • Partial invariance to affine intensity • But: non-invariant to image scale ! change ü Only derivatives are used => invariance to intensity shift I → I + b ü Intensity scale: I → a I R R threshold All points will be Corner ! classified as edges x (image coordinate) x (image coordinate) Harris Detector: Some Models of Image Change Properties • Quality of Harris detector for different • Geometry scale changes – Rotation Repeatability rate: – Similarity (rotation + uniform scale) # correspondences # possible correspondences – Affine (scale dependent on direction) valid for: orthographic camera, locally planar object • Photometry – Affine intensity change ( I → a I + b ) C.Schmid et.al. � Evaluation of Interest Point Detectors � . IJCV 2000

Rotation Invariant Detection Reading • Harris Corner Detector • https://en.wikipedia.org/wiki/Corner_detection – See bibliography for additional resources • Shi and C. Tomasi (1994). Good Features to Track – http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.135.7147 • Moravec (1980) Corner Detector (Ch 5-6) ( here ). • Harris & Stephens (1998), A Corner & Edge Detector – http://www.bmva.org/bmvc/1988/avc-88-023.pdf – Very readable. • Mikolajczyk and Schmid (2001). “Indexing Based on Scale Invariant Interest Points” – http://www.ee.surrey.ac.uk/CVSSP/Publications/papers/Mikolajczyk-ICCV-2001.pdf • Lowe (2004) “Distinctive Image Features from Scale-Invariant Keypoints”. IJCV 2004 – http://people.eecs.berkeley.edu/~malik/cs294/lowe-ijcv04.pdf • Search for “Features” on OpenCV site C.Schmid et.al. � Evaluation of Interest Point Detectors � . IJCV 2000