Triangulation 16-385 Computer Vision (Kris Kitani) Carnegie Mellon - PowerPoint PPT Presentation

Triangulation 16-385 Computer Vision (Kris Kitani) Carnegie Mellon University

Structure Motion Measurements (scene geometry) (camera geometry) 3D to 2D estimate Pose Estimation known correspondences 2D to 2D estimate Triangulation known coorespondences 2D to 2D estimate estimate Reconstruction coorespondences

Triangulation image 1 image 2 x x 0 Given

Triangulation Find 3D object point Will the lines intersect? image 1 image 2 x x 0 C C 0

Triangulation Find 3D object point (no single solution due to noise) image 1 image 2 x x 0 C C 0

Triangulation Given a set of (noisy) matched points { x i , x 0 i } and camera matrices P , P 0 Estimate the 3D point X

x = P X known known Can we compute X from a single correspondence x ?

y image plane X no unique 3D point infinitely many 3D points along the ray x x z camera center z = f

x = P X known known Can we compute X from two correspondences x and x’ ?

x = P X known known Can we compute X from two correspondences x and x’ ? yes if perfect measurements

x = P X known known Can we compute X from two correspondences x and x’ ? yes if perfect measurements There will not be a point that satisfies both constraints because the measurements are usually noisy x 0 = P 0 X x = P X Need to find the best fit

x = P X (homogeneous coordinate) Also, this is a similarity relation because it involves homogeneous coordinates x = α P X (inhomogeneous coordinate) Same ray direction but differs by a scale factor   X     x p 1 p 2 p 3 p 4 Y    = α y p 5 p 6 p 7 p 8      Z   z p 9 p 10 p 11 p 12 1 How do we solve for unknowns in a similarity relation?

x = P X (homogeneous coordinate) Also, this is a similarity relation because it involves homogeneous coordinates x = α P X (inhomogeneous coordinate) Same ray direction but differs by a scale factor   X     x p 1 p 2 p 3 p 4 Y    = α y p 5 p 6 p 7 p 8      Z   z p 9 p 10 p 11 p 12 1 How do we solve for unknowns in a similarity relation? Direct Linear Transform Remove scale factor, convert to linear system and solve with SVD.

x = P X (homogeneous coordinate) Also, this is a similarity relation because it involves homogeneous coordinates x = α P X (inhomogeneous coordinate) Same ray direction but differs by a scale factor   X     x p 1 p 2 p 3 p 4 Y    = α y p 5 p 6 p 7 p 8      Z   z p 9 p 10 p 11 p 12 1 How do we solve for unknowns in a similarity relation? Direct Linear Transform Remove scale factor, convert to linear system and solve with SVD.

x = α P X Same direction but differs by a scale factor x × P X = 0 Cross product of two vectors of same direction is zero (this equality removes the scale factor)

Recall: Cross Product Vector (cross) product 
 takes two vectors and returns a vector perpendicular to both   a 2 b 3 − a 3 b 2 c = a × b a 3 b 1 − a 1 b 3 a × b =   a 1 b 2 − a 2 b 1 b cross product of two vectors in the same direction is zero a a × a = 0 remember this!!! c · b = 0 c · a = 0

  X     x p 1 p 2 p 3 p 4 Y    = α y p 5 p 6 p 7 p 8      Z   z p 9 p 10 p 11 p 12 1 —– p >       x 1 —–  X —– p >  = α y 2 —–     —– p > z 3 —– p >     1 X x p > 2 X  = α y    p > 3 X z

  X     x p 1 p 2 p 3 p 4 Y    = α y p 5 p 6 p 7 p 8      Z   z p 9 p 10 p 11 p 12 1 —– p >       x 1 —–  X —– p >  = α y 2 —–     —– p > z 3 —– p >     1 X x p > 2 X  = α y    p > 3 X z p > y p > 3 X − p >         1 X 2 X 0 x  =  = p > p > 1 X − x p >  × 2 X 3 X 0 y      p > x p > 2 X − y p > 1 3 X 1 X 0

Using the fact that the cross product should be zero x × P X = 0 p > y p > 3 X − p >         1 X 2 X 0 x  =  = p > p > 1 X − x p >  × 2 X 3 X 0 y      p > x p > 2 X − y p > 1 3 X 1 X 0 Third line is a linear combination of the first and second lines. (x times the first line plus y times the second line) One 2D to 3D point correspondence give you 2 equations

Using the fact that the cross product should be zero x × P X = 0 p > y p > 3 X − p >         1 X 2 X 0 x  =  = p > p > 1 X − x p >  × 2 X 3 X 0 y      p > x p > 2 X − y p > 1 3 X 1 X 0 Third line is a linear combination of the first and second lines. (x times the first line plus y times the second line) One 2D to 3D point correspondence give you 2 equations

 0  y p > 3 X − p > � � 2 X = p > 1 X − x p > 3 X 0  0  y p > 3 − p > � � 2 X = p > 1 − x p > 0 3 A i X = 0 Now we can make a system of linear equations (two lines for each 2D point correspondence)

Concatenate the 2D points from both images y p > 3 − p >     0 2 p > 1 − x p > 0     3  X =     y 0 p 0> 3 − p 0> 0    2 p 0> 1 − x 0 p 0> 0 3 sanity check! dimensions? A X = 0 How do we solve homogeneous linear system?

Concatenate the 2D points from both images y p > 3 − p >     0 2 p > 1 − x p > 0     3  X =     y 0 p 0> 3 − p 0> 0    2 p 0> 1 − x 0 p 0> 0 3 A X = 0 How do we solve homogeneous linear system? S V D !

Concatenate the 2D points from both images y p > 3 − p >     0 2 p > 1 − x p > 0     3  X =     y 0 p 0> 3 − p 0> 0    2 p 0> 1 − x 0 p 0> 0 3 A X = 0 How do we solve homogeneous linear system? S V D !

Concatenate the 2D points from both images y p > 3 − p >     0 2 p > 1 − x p > 0     3  X =     y 0 p 0> 3 − p 0> 0    2 p 0> 1 − x 0 p 0> 0 3 A X = 0 How do we solve homogeneous linear system? S V D !

Concatenate the 2D points from both images y p > 3 − p >     0 2 p > 1 − x p > 0     3  X =     y 0 p 0> 3 − p 0> 0    2 p 0> 1 − x 0 p 0> 0 3 A X = 0 How do we solve homogeneous linear system? S V D !

Recall: Total least squares ( Warning: change of notation. x is a vector of parameters!) X ( a i x ) 2 E TLS = i = k A x k 2 (matrix form) k x k 2 = 1 constraint k A x k 2 minimize k A x k 2 minimize k x k 2 k x k 2 = 1 subject to (Rayleigh quotient) Solution is the eigenvector corresponding to smallest eigenvalue of A > A

Structure Motion Measurements (scene geometry) (camera geometry) 3D to 2D estimate Pose Estimation known correspondences 2D to 2D estimate Triangulation known coorespondences 2D to 2D estimate estimate Reconstruction coorespondences