The Epipolar Geometry COMPSCI 527 Computer Vision COMPSCI 527 - PowerPoint PPT Presentation

The Epipolar Geometry COMPSCI 527 — Computer Vision COMPSCI 527 — Computer Vision The Epipolar Geometry 1 / 16

The Epipolar Geometry of a Pair of Cameras The Epipolar Geometry of a Pair of Cameras P projection ray y a r n o i t c e j o r p epipolar plane I epipolar line of p b I p a a center of epipolar line of p projection a b p a baseline epipole e b b center of camera a camera b b projection epipole e a COMPSCI 527 — Computer Vision The Epipolar Geometry 2 / 16

The Epipolar Geometry of a Pair of Cameras The Epipolar Constraint P I projection ray projection ray epipolar plane I e b p i p o p l a I r a l a i n center of e o epipolar line of p f p projection a b p a baseline epipole e b b center of camera a camera b b projection epipole e a • The point p a in image a that corresponds to point p b in image b is on the epipolar line of p b ... and vice versa • This is the only general constraint between two images of the same scene; 3D reconstruction depends on it • Epipolar lines come in corresponding pairs • Two pencils of lines supported by the two epipoles COMPSCI 527 — Computer Vision The Epipolar Geometry 3 / 16

The Epipolar Geometry of a Pair of Cameras P Another Way to State the Epipolar Constraint P projection ray projection ray epipolar plane I e b p i p o p l a I C r a C l a i n center of e o epipolar line of p f a b projection p a b p a baseline epipole e b b center of camera a camera b b projection epipole e a The two projection rays and the baseline are coplanar for corresponding points COMPSCI 527 — Computer Vision The Epipolar Geometry 4 / 16

The Epipolar Geometry of a Pair of Cameras Two Uses of the Epipolar Constraint • Stereo vision: • Relative position and orientation of the two cameras are known • So the epipoles can be determined • Given a point p a in image a , find its epipolar plane, and therefore the epipolar line for the point p b in b • This reduces search for correspondence from the image plane to the epipolar line • 3D camera motion and reconstruction: • Relative position and orientation of the two cameras are unknown • Given corresponding points p a , p b (found, say, by tracking) we can write one algebraic constraint on a P , a R b , a t b • With enough pairs of corresponding points, we can write and solve a system in these quantities COMPSCI 527 — Computer Vision The Epipolar Geometry 5 / 16

The Essential Matrix The Essential Matrix • How to write the epipolar constraint algebraically? • The essential matrix E is a compact representation of the relative position and orientation of two cameras • E is 3 × 3 and combines rotation and translation • The essential matrix E can be found by solving a homogeneous linear system • Given E , rotation and translation can be found by additional manipulation COMPSCI 527 — Computer Vision The Epipolar Geometry 6 / 16

The Essential Matrix P Coordinates O p o p b a Z Z b a X b X a Y Y b a • Image points as world points:  a x a   b x b  08 Oo a p a = a y a b p b = b y b and     f f • Each camera measures a point in its own reference system • Transformation: b p = a R b ( a p − a t b ) or • Inverse: a p = b R a ( b p − b t a ) b R a = a R T b t a = − a R ba t b where and b COMPSCI 527 — Computer Vision The Epipolar Geometry 7 / 16

The Essential Matrix Simplifying Notation • Too many super/subscripts to keep track of 0000 • Define: a = a p a b = b p b R = a R b t = a t b e = a e b , , , , in Hub b a E_r Z Z b a X e b X a Y Y b a R, t to • e and t are the same up to norm o a b as direction vector is R T b (all coordinates are in a now) • COMPSCI 527 — Computer Vision The Epipolar Geometry 8 / 16

The Essential Matrix The Epipolar Constraint, Algebraically a it IT VIE 0 b a F Z 9Et_ Z b a X e b X a Y Y b a R, t • The two projection rays and the baseline are coplanar OO O • The triple product of a b , t , and a is zero Q • We saw that a b = R T b O • The triple product of R T b , ( R T b ) T ( t × a ) = 0 t , and a is zero: COMPSCI 527 — Computer Vision The Epipolar Geometry 9 / 16

The Essential Matrix The Essential Matrix ( R T b ) T ( t × a ) = 0 co CEO b T R ( t × a ) = 0 b T R [ t ] × a = 0   0 − t z t y where t = ( t x , t y , t z ) T and [ t ] × = t z 0 − t x   − t y t x 0 b T E a = 0 where E = R [ t ] × e • This equation is the epipolar constraint , written in algebra • Holds for any corresponding a , b in the two images (as world vectors in their reference systems) • E is the essential matrix COMPSCI 527 — Computer Vision The Epipolar Geometry 10 / 16

The Essential Matrix The Epipolar Line in Image a • Think of b as fixed • What vectors x in image a b a Z Z b satisfy the epipolar constraint? a X e b 0 b T E x = 0 X a • Let λ T = b T E , a row vector Y Y b a II a b TEE E R, t • λ T x = 0, a line! • a satisfies this homogeneous equation (epipolar constraint) λ T t = b T E t = b T R [ t ] × t = 0 • So does t : . . . and therefore e • So the line is the epipolar line of b COMPSCI 527 — Computer Vision The Epipolar Geometry 11 / 16