lecture 10 scales and descriptors
play

Lecture 10: Scales and Descriptors Justin Johnson EECS 442 WI - PowerPoint PPT Presentation

Lecture 10: Scales and Descriptors Justin Johnson EECS 442 WI 2020: Lecture 10 - 1 February 11, 2020 Administrative HW2 out, due 1 week from tomorrow: Wednesday 2/19/2019, 11:59pm HW3 out tomorrow, due 2 weeks from Friday: Friday 2/27,


  1. Lecture 10: Scales and Descriptors Justin Johnson EECS 442 WI 2020: Lecture 10 - 1 February 11, 2020

  2. Administrative HW2 out, due 1 week from tomorrow: Wednesday 2/19/2019, 11:59pm HW3 out tomorrow, due 2 weeks from Friday: Friday 2/27, 11:59pm Justin Johnson EECS 442 WI 2020: Lecture 10 - 2 February 11, 2020

  3. Last Time: Motivation Are these pictures of the same object? If so, how are they related? Justin Johnson EECS 442 WI 2020: Lecture 10 - 3 February 11, 2020

  4. Last Time: Finding + Matching Finding and Matching 1: find corners+features 2: match based on local image data Slide Credit: S. Lazebnik, original figure: M. Brown, D. Lowe Justin Johnson EECS 442 WI 2020: Lecture 10 - 4 February 11, 2020

  5. Last Time: Edges + Corners Part 1: Finding Edges Part 2: Finding Corners Corner of the glasses Edge next to panel Justin Johnson EECS 442 WI 2020: Lecture 10 - 5 February 11, 2020

  6. Last Time: Edges via Image Gradients Compute derivatives Ix and Iy with filters Ix Iy Justin Johnson EECS 442 WI 2020: Lecture 10 - 6 February 11, 2020

  7. Last Time: Edges via Image Gradients Gradient Direction: atan2(Ix, Iy) Gradient Magnitude: sqrt(Ix 2 + Iy 2 ) I’m making the lightness equal to gradient magnitude Justin Johnson EECS 442 WI 2020: Lecture 10 - 7 February 11, 2020

  8. Last Time: Edges via Image Gradients 𝑒𝑦 𝑔 βˆ— 𝑕 = 𝑔 βˆ— 𝑒 𝑒 𝑒𝑦 𝑕 f d g dx d f * g dx Slide Credit: S. Seitz Justin Johnson EECS 442 WI 2020: Lecture 10 - 8 February 11, 2020

  9. Last Time: Corners Can localize the location, or any shift β†’ big intensity change. β€œflat” region: β€œedge”: β€œcorner”: no change in no change significant all directions along the edge change in all direction directions Diagram credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 9 February 11, 2020

  10. Last Time: Detecting Corners By doing a taylor expansion of the image, the second moment matrix tells us how quickly the image changes and in which directions. Can compute at each Directions pixel / ( 𝐽 ) ( 𝐽 ) 𝐽 + = 𝑺 12 πœ‡ 2 0 ),+∈- ),+∈- 𝑡 = 𝑺 0 πœ‡ / / ( 𝐽 ) 𝐽 + ( 𝐽 + ),+∈- ),+∈- Amounts Justin Johnson EECS 442 WI 2020: Lecture 10 - 10 February 11, 2020

  11. Last Time: Detecting Corners β€œEdge” 𝑆 = det 𝑡 βˆ’ 𝛽 𝑒𝑠𝑏𝑑𝑓 𝑡 / R < 0 β€œCorner” / = πœ‡ 2 πœ‡ / βˆ’ 𝛽 πœ‡ 2 + πœ‡ / R > 0 Ξ± : constant (0.04 to 0.06) |R| small β€œEdge” β€œFlat” R < 0 region Slide credit: S. Lazebnik; Note: this refers to visualization ellipses, not original M ellipse. Other slides on the internet may vary Justin Johnson EECS 442 WI 2020: Lecture 10 - 11 February 11, 2020

  12. Harris Corner Detector 1. Compute partial derivatives Ix, Iy per pixel 2. Compute M at each pixel, using Gaussian weighting w / ( π‘₯(𝑦, 𝑧)𝐽 ) ( π‘₯(𝑦, 𝑧)𝐽 ) 𝐽 + ),+∈- ),+∈- 𝑡 = / ( π‘₯(𝑦, 𝑧)𝐽 ) 𝐽 + ( π‘₯(𝑦, 𝑧)𝐽 + ),+∈- ),+∈- C.Harris and M.Stephens. β€œA Combined Corner and Edge Detector.” Proceedings of the 4th Alvey Vision Conference : pages 147β€”151, 1988. Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 12 February 11, 2020

  13. Harris Corner Detector 1. Compute partial derivatives Ix, Iy per pixel 2. Compute M at each pixel, using Gaussian weighting w 3. Compute response function R 𝑆 = det 𝑡 βˆ’ 𝛽 𝑒𝑠𝑏𝑑𝑓 𝑡 / / = πœ‡ 2 πœ‡ / βˆ’ 𝛽 πœ‡ 2 + πœ‡ / C.Harris and M.Stephens. β€œA Combined Corner and Edge Detector.” Proceedings of the 4th Alvey Vision Conference : pages 147β€”151, 1988. Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 13 February 11, 2020

  14. Computing R Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 14 February 11, 2020

  15. Computing R Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 15 February 11, 2020

  16. Harris Corner Detector 1. Compute partial derivatives Ix, Iy per pixel 2. Compute M at each pixel, using Gaussian weighting w 3. Compute response function R 4. Threshold R C.Harris and M.Stephens. β€œA Combined Corner and Edge Detector.” Proceedings of the 4th Alvey Vision Conference : pages 147β€”151, 1988. Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 16 February 11, 2020

  17. Harris Corner Detector Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 17 February 11, 2020

  18. Harris Corner Detector 1. Compute partial derivatives Ix, Iy per pixel 2. Compute M at each pixel, using Gaussian weighting w 3. Compute response function R 4. Threshold R 5. Take only local maxima (Non-Maxima Suppression, NMS) C.Harris and M.Stephens. β€œA Combined Corner and Edge Detector.” Proceedings of the 4th Alvey Vision Conference : pages 147β€”151, 1988. Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 18 February 11, 2020

  19. Harris Corner Detector: NMS Local Maxima are pixels with a higher R value than their neighbors R threshold x (image coordinate) Justin Johnson EECS 442 WI 2020: Lecture 10 - 19 February 11, 2020

  20. Harris Corner Detector Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 20 February 11, 2020

  21. Harris Corner Detector: Result Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 21 February 11, 2020

  22. Desirable Properties If our detectors are repeatable, they should be: β€’ Invariant to some things: image is transformed and corners remain the same: D(T(I)) = D(I) β€’ Covariant/equivariant with some things: image is transformed and corners transform with it: D(T(I)) = T(D(I)) Where I is an image, T is a transformation, and D is our detector Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 22 February 11, 2020

  23. Affine Intensity Change 𝐽 EFG = 𝑏𝐽 HIJ + 𝑐 M only depends on derivatives, so b is irrelevant But a scales derivatives and there’s a threshold R R threshold x (image coordinate) x (image coordinate) Partially invariant to affine intensity changes Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 23 February 11, 2020

  24. Image Translation All done with convolution. Convolution is translation invariant. Equivariant with translation Slide credit: S. Lazebnik Justin Johnson EECS 442 WI 2020: Lecture 10 - 24 February 11, 2020

  25. Image Rotation Rotations just cause the corner rotation to change. Eigenvalues remain the same. Equivariant with rotation Justin Johnson EECS 442 WI 2020: Lecture 10 - 25 February 11, 2020

  26. Image Scaling Corner One pixel can become many pixels and vice-versa. Not equivariant with scaling Justin Johnson EECS 442 WI 2020: Lecture 10 - 26 February 11, 2020

  27. Today β€’ Fixing scaling by making detectors in both location and scale β€’ Enabling matching between features by describing regions Justin Johnson EECS 442 WI 2020: Lecture 10 - 27 February 11, 2020

  28. Key Idea: Scale Left to right: each image is half-sized Upsampled with big pixels below 1/2 1/2 1/2 Note: I’m also slightly blurring to prevent aliasing (https://en.wikipedia.org/wiki/Aliasing) Justin Johnson EECS 442 WI 2020: Lecture 10 - 28 February 11, 2020

  29. Key Idea: Scale Left to right: each image is half-sized If I apply a KxK filter, how much of the original image does it see in each image? 1/2 1/2 1/2 Note: I’m also slightly blurring to prevent aliasing (https://en.wikipedia.org/wiki/Aliasing) Justin Johnson EECS 442 WI 2020: Lecture 10 - 29 February 11, 2020

  30. Solution to Scales: Try them all! Harris Detection Harris Detection Harris Detection Harris Detection See: Multi-Image Matching using Multi-Scale Oriented Patches, Brown et al. CVPR 2005 Justin Johnson EECS 442 WI 2020: Lecture 10 - 30 February 11, 2020

  31. Aside: This is a common trick! Given a 50x16 person detector, how do I detect: (a) 250x80 (b) 150x48 (c) 100x32 (d) 25x8 people? Sample people from image Justin Johnson EECS 442 WI 2020: Lecture 10 - 31 February 11, 2020

  32. Aside: This is a common trick! Detecting all the people The red box is a fixed size Sample people from image Justin Johnson EECS 442 WI 2020: Lecture 10 - 32 February 11, 2020

  33. Aside: This is a common trick! Detecting all the people The red box is a fixed size Sample people from image Justin Johnson EECS 442 WI 2020: Lecture 10 - 33 February 11, 2020

  34. Aside: This is a common trick! Detecting all the people The red box is a fixed size Sample people from image Justin Johnson EECS 442 WI 2020: Lecture 10 - 34 February 11, 2020

  35. Blob Detection Another detector (has some nice properties) Minima βˆ— = Maxima Find maxima and minima of blob filter response in scale and space Slide credit: N. Snavely Justin Johnson EECS 442 WI 2020: Lecture 10 - 35 February 11, 2020

  36. Gaussian Derivatives (1D) exp βˆ’ 𝑦 / 1 𝐻 M 𝑦 = 2𝜏 / 𝜏 2𝜌 𝐻 M 𝑦 Justin Johnson EECS 442 WI 2020: Lecture 10 - 36 February 11, 2020

  37. Gaussian Derivatives (1D) exp βˆ’ 𝑦 / πœ– 𝑦 = βˆ’ 𝑦 πœ–π‘¦ 𝐻 M 𝑦 = 𝜏 / 𝐻 M 𝑦 𝜏 U 2𝜌 2𝜏 / πœ– 𝐻 M 𝑦 πœ–π‘¦ 𝐻 M 𝑦 Justin Johnson EECS 442 WI 2020: Lecture 10 - 37 February 11, 2020

Recommend


More recommend