Introduction to Computer Vision for Robotics AE640A Autonomous Navigation 11 th March, 2019 Harsh Sinha 1 Introduction to Computer Vision 1
… From the last class Actual image plane behind the pinhole Focal distance Focal distance Harsh Sinha Introduction to Computer Vision 2
… From the last class R T 0 K P p’ H M = K [I 0] Harsh Sinha Introduction to Computer Vision 3
Lecture Outline ● Stereo Vision ○ Introduction to Stereo Vision ○ Epipolar Geometry ○ The correspondence problem ● Stereo Matching ○ Various methods for Stereo Matching ○ Stereo Block Matching ○ A look at SGBM Harsh Sinha Introduction to Computer Vision 4
Stereo Vision Harsh Sinha Introduction to Computer Vision 5
Introduction to stereo vision Credits: Kenji Hata, Silvio Savarese Harsh Sinha Introduction to Computer Vision 6
Introduction to stereo vision Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 7
Introduction to stereo vision Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 8
Introduction to stereo vision How do humans figure out 3D in 2D images? Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 9
Introduction to stereo vision How do humans figure out 3D in 2D images? 1. Shading Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 10
Introduction to stereo vision How do humans figure out 3D in 2D images? 1. Shading 2. Texture Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 11
Introduction to stereo vision How do humans figure out 3D in 2D images? 1. Shading 2. Texture 3. Focus Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 12
Introduction to stereo vision The stereo problem: ● Nature Inspired approach to vision, i.e, 3D with two sensors. ● How to figure out the shape, more specifically the depth, of objects from a set of two or more images? Credits: Gaurav Pandey, Ford Harsh Sinha Introduction to Computer Vision 13
Introduction to stereo vision So, How do we go we go from Stereo Images to Depth Information ? Harsh Sinha Introduction to Computer Vision 14
Introduction to stereo vision Credits: Gaurav Pandey, Ford Harsh Sinha Introduction to Computer Vision 15
Introduction to stereo vision Harsh Sinha Introduction to Computer Vision 16
Introduction to stereo vision Note : We have the image planes parallel here. Creating such images from non parallel cameras is called rectification . Harsh Sinha Introduction to Computer Vision 17
Introduction to stereo vision Credits: Gaurav Pandey, Ford Harsh Sinha Introduction to Computer Vision 18
Introduction to stereo vision Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 19
Introduction to stereo vision Given this point how do you find the corresponding point on the other image? Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 20
Introduction to stereo vision Given this point how do you find the corresponding point on the other image? Search the whole image? Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 21
Introduction to stereo vision Given this point how do you find the corresponding point on the other image? Search the whole image? Difficult to solve accurately, very expensive without special methods Harsh Sinha Introduction to Computer Vision 22
Epipolar Geometry Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 23
Epipolar Geometry Epipolar Plane Camera 2 Camera 1 Center Center Baseline Epipoles Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 24
Epipolar Geometry Epipolar Pencil Credits: Richard Hartley, Andrew Zisserman Harsh Sinha Introduction to Computer Vision 25
Epipolar Geometry Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 26
Epipolar Geometry Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 27
Epipolar Geometry Search along this line for the closest point. Computationally way more efficient. Easier to solve. Can use simple SSD or similar methods. Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 28
Epipolar Geometry Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 29
Epipolar Geometry Assumed to be canonical camera Harsh Sinha Introduction to Computer Vision 30
Epipolar Geometry R T p’ - R T T is p’ in S O R T T also lies in plane => R T T x (R T p’ - R T T) is Assumed to perpendicular to be canonical epipolar plane camera Harsh Sinha Introduction to Computer Vision 31
Epipolar Geometry => R T T x (R T p’ - R T T) = R T (T x p’) is perpendicular to p => (R T (T x p’)) T p = 0 Assumed to => (T x p’ T )Rp = 0 be canonical camera Harsh Sinha Introduction to Computer Vision 32
Epipolar Geometry From Linear Algebra, the cross product of two vectors can be written as : Harsh Sinha Introduction to Computer Vision 33
Epipolar Geometry From Linear Algebra, the cross product of two vectors can be written as : [a x ] : skew symmetric Harsh Sinha Introduction to Computer Vision 34
Epipolar Geometry Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 35
Epipolar Geometry: Essential Matrix (E) Credits: Fei Fei Li Harsh Sinha Introduction to Computer Vision 36
Epipolar Geometry Why? Harsh Sinha Introduction to Computer Vision 37
Epipolar Geometry ax + by + c = 0 i.e L = [a b c] T represents a line in homogeneous coordinates. => z T L = 0 where, z = [x, y, 1] T Harsh Sinha Introduction to Computer Vision 38
Epipolar Geometry Harsh Sinha Introduction to Computer Vision 39
Epipolar Geometry: Fundamental Matrix (F) F: Fundamental Matrix Harsh Sinha Introduction to Computer Vision 40
Epipolar Geometry: Properties of F Credits: Richard Hartley, Andrew Zisserman Harsh Sinha Introduction to Computer Vision 41
Epipolar Geometry: Estimating F Credits: Robert Collins, Penn State Harsh Sinha Introduction to Computer Vision 42
Epipolar Geometry: Estimating F Credits: Robert Collins, Penn State Harsh Sinha Introduction to Computer Vision 43
Epipolar Geometry: Estimating F Credits: Robert Collins, Penn State Harsh Sinha Introduction to Computer Vision 44
Epipolar Geometry: Estimating F Credits: Robert Collins, Penn State Harsh Sinha Introduction to Computer Vision 45
Epipolar Geometry: Estimating F Credits: Robert Collins, Penn State Harsh Sinha Introduction to Computer Vision 46
Epipolar Geometry: Estimating F Credits: Robert Collins, Penn State Harsh Sinha Introduction to Computer Vision 47
Stereo Matching Harsh Sinha Introduction to Computer Vision 48
Various Methods for Stereo Matching Harsh Sinha Introduction to Computer Vision 49
Various Methods for Stereo Matching Harsh Sinha Introduction to Computer Vision 50
Various Methods for Stereo Matching Harsh Sinha Introduction to Computer Vision 51
Various Methods for Stereo Matching Harsh Sinha Introduction to Computer Vision 52
Various Methods for Stereo Matching Harsh Sinha Introduction to Computer Vision 53
Various Methods for Stereo Matching Harsh Sinha Introduction to Computer Vision 54
Various Methods for Stereo Matching Harsh Sinha Introduction to Computer Vision 55
Various Methods for Stereo Matching Harsh Sinha Introduction to Computer Vision 56
Stereo Block Matching ● Similar to what we just saw in window sizes example. ● Idea is to instead of matching pixel values, match regions of image, this is done in order to increase robustness in the depth prediction. ● Sparse Stereo Matching : Use of key points or features to serve as corresponding points on the two images. ● Dense Stereo Matching : Match all pixels in a region along a scan line in pair of stereo rectified images. Harsh Sinha Introduction to Computer Vision 57
Stereo Block Matching Credits: Trym Vegard Haavardsholm Harsh Sinha Introduction to Computer Vision 58
Stereo Block Matching: Global Optimization Credits: Trym Vegard Haavardsholm Harsh Sinha Introduction to Computer Vision 59
Stereo Block Matching: Global Optimization Minimize E Cost of pixel Penalty based on over D to get wise matching neighbours mismatches, D* I.e, penalty for neighbours having different disparity Credits: HEIKO HIRSCHMÜLLER Harsh Sinha Introduction to Computer Vision 60
Stereo Block Matching: Global Optimization Guess the Drawbacks!! Minimize E Cost of pixel Penalty based on over D to get wise matching neighbours mismatches, D* I.e, penalty for neighbours having different disparity Harsh Sinha Introduction to Computer Vision 61
Stereo Block Matching: Global Optimization Guess the Drawbacks!! ● Too Computationally Intensive ● NP Complete Problem Minimize E Cost of pixel Penalty based on over D to get wise matching neighbours mismatches, D* I.e, penalty for neighbours having different disparity Harsh Sinha Introduction to Computer Vision 62
Recommend
More recommend
