Computer Vision Levente Hajder, Dmitry Chetverikov Etvs Lornd - PowerPoint PPT Presentation

Computer Vision Levente Hajder, Dmitry Chetverikov Eötvös Loránd University, Faculty of Informatics Hajder, Csetverikov (Faculty of Informatics) Computer Vision 1 / 85

Camera Models and Calibration Camera Models 1 Perspective (pin-hole) camera Weak-perspective camera Comparison of camera models Back-projection to 3D space Homography 2 Homography estimation Non-linear estimation by minimizing geometric error Camera Calibration 3 Calibration by a spatial object Calibration using a chessboard Radial distortion Summary 4 Hajder, Csetverikov (Faculty of Informatics) Computer Vision 2 / 85

Camera Models Outline Camera Models 1 Perspective (pin-hole) camera Weak-perspective camera Comparison of camera models Back-projection to 3D space Homography 2 Homography estimation Non-linear estimation by minimizing geometric error Camera Calibration 3 Calibration by a spatial object Calibration using a chessboard Radial distortion Summary 4 Hajder, Csetverikov (Faculty of Informatics) Computer Vision 3 / 85

Camera Models Perspective (pin-hole) camera Outline Camera Models 1 Perspective (pin-hole) camera Weak-perspective camera Comparison of camera models Back-projection to 3D space Homography 2 Homography estimation Non-linear estimation by minimizing geometric error Camera Calibration 3 Calibration by a spatial object Calibration using a chessboard Radial distortion Summary 4 Hajder, Csetverikov (Faculty of Informatics) Computer Vision 4 / 85

Camera Models Perspective (pin-hole) camera Gemoetric Imaging Models We introduce different geometric models General perspective camera Simplified camera models Perspective camera model equivalent to pin-hole camera . camera obscura Pin-hole camera is close to real optics → simple model of a thin optics → Physical models are significantly complicated. However, a perspective camera is a very good geometric approximation . We address separately the following issues: radiometric properties (brightness, colors) geometric distortions Hajder, Csetverikov (Faculty of Informatics) Computer Vision 5 / 85

Camera Models Perspective (pin-hole) camera Perspective camera model image plane φ image world coordinates coordinates v Z u R camera w coordinates Y X Y C t X C X u 0 C optical axis Z f principal C focal point point u projected X C point optical ray scene point X Hajder, Csetverikov (Faculty of Informatics) Computer Vision 6 / 85

Camera Models Perspective (pin-hole) camera Notations: coordinates and transformations Coordinates X = [ X , Y , Z ] T world X c = [ X c , Y c , Z c ] T camera u = [ u , v ] T image plane Homogeneous coordinates X = [ X , Y , Z , 1 ] T world X c = [ X c , Y c , Z c , 1 ] T camera u = [ u , v , 1 ] T image plane Transformations R : rotation (matrix) t : translation (vector) Hajder, Csetverikov (Faculty of Informatics) Computer Vision 7 / 85

Camera Models Perspective (pin-hole) camera Notations: camera u 0 = [ u 0 , v 0 ] T φ C f focal point image plane focal length principal point C focal point: central projection Optical ray: it connects a 3D point and focal point C Optical axis: Contains the focal point C and perpendicular to image plane φ Focal length: distance between C and φ . Principal point: the point in image plane where optical axis intersects φ Hajder, Csetverikov (Faculty of Informatics) Computer Vision 8 / 85

Camera Models Perspective (pin-hole) camera Perspective camera model image plane φ image world coordinates coordinates v Z u R camera w coordinates Y X Y C t X C X u 0 C optical axis Z f principal C focal point point u projected X C point optical ray scene point X Hajder, Csetverikov (Faculty of Informatics) Computer Vision 9 / 85

Camera Models Perspective (pin-hole) camera Translation and rotation World − → Camera Euclidean coordinates X c = R ( X − t ) (1) Homogeneous coordinates � X � X c = R [ I | − t ] (2) 1 I is a 3 × 3- identity matrix [ I | − t ] is a 3 × 4 -matrix → I completed by colums − t Hajder, Csetverikov (Faculty of Informatics) Computer Vision 10 / 85

Camera Models Perspective (pin-hole) camera Projection to an image plane image plane X c u = fk u X c + u 0 (3) Z c v = fk v Y c + v 0 (4) Z c Z c C focal point f k u , k v is the horizontal/vertical pixel size. → their unit is pixel/length . Usually, k u = k v = k . Hajder, Csetverikov (Faculty of Informatics) Computer Vision 11 / 85

Camera Models Perspective (pin-hole) camera Projection using homogeneous coordinates   u   ∼ KX c v (5) 1 ∼ homogeneous division yields scale ambiguity K is the (intrinsic) calibration matrix   fk u 0 u 0   K = 0 fk v v 0 (6) 0 0 1 upper triangular matrix consists of 5 parameters, but only four are realistic → fk u , fk v , u 0 , v 0 Hajder, Csetverikov (Faculty of Informatics) Computer Vision 12 / 85

Camera Models Perspective (pin-hole) camera Multi-view projection of a spatial point Locations of the same spatial point differ in images. Locations should be detected and/or tracked in the images. → They are called correspondences . Hajder, Csetverikov (Faculty of Informatics) Computer Vision 13 / 85

Camera Models Perspective (pin-hole) camera Perspective camera model Goal: to determine the location of the projected 3D points in camera images.   � X � � X � u   ∼ KR [ I | − t ] v = P (7) 1 1 1 P . = KR [ I | − t ] is the projection matrix consists of 11 parameters → 5 in K , 3 in R , another 3 in t . Hajder, Csetverikov (Faculty of Informatics) Computer Vision 14 / 85

Camera Models Weak-perspective camera Outline Camera Models 1 Perspective (pin-hole) camera Weak-perspective camera Comparison of camera models Back-projection to 3D space Homography 2 Homography estimation Non-linear estimation by minimizing geometric error Camera Calibration 3 Calibration by a spatial object Calibration using a chessboard Radial distortion Summary 4 Hajder, Csetverikov (Faculty of Informatics) Computer Vision 15 / 85

Camera Models Weak-perspective camera Weak-perspective projection 1/2 It is assumed that the object is not ’too close’ from the camera change in depth is significantly smaller than the camera-object distance Object plane is parallel to the image plane it is ideal if object center contains the center of gravity of the object. Objects are orthogonally projected into the object plane Then perspective projection is applied as there is no difference in depth, location of principal point does not matter. → for the sake of simplicity, u 0 = v 0 = 0. Hajder, Csetverikov (Faculty of Informatics) Computer Vision 16 / 85

Camera Models Weak-perspective camera Weak-perspective projection 2/2 image plane object plane X u = fk c X c + u 0 (8) � Z c ∆ v = fk Y c + v 0 (9) � Z Z c c C focal point ~ f Z If pixel is a square, k u = k v = k It is also assumed that Z c ≫ ∆ → Z c ≈ � Z c , where � Z c is the common depth → scaled orthographic projection Hajder, Csetverikov (Faculty of Informatics) Computer Vision 17 / 85

Camera Models Weak-perspective camera Weak-perspective camera model 1/2 Translation and rotation in conjunction with weak-perspective projection: u = q r T 1 ( X − t ) + u 0 (10) v = q r T 2 ( X − t ) + v 0 , where (11) = fk q . � Z c r T 1 and r T 2 are the first and second row vectors of rotation matrix R . u 0 represents offset: − → u 0 = v 0 = 0 u = q r T 1 ( X − t ) (12) v = q r T 2 ( X − t ) (13) Hajder, Csetverikov (Faculty of Informatics) Computer Vision 18 / 85

Camera Models Weak-perspective camera Weak-perspective camera model 2/2 Projection can be written with the help of a weak-perspective camera matrix: � u � � X � = [ M | b ] , where (14) v 1      r T  q r T 1 t M . b . 1  ,  = q = − r T q r T 2 t 2 Model has 6 degree of freedom (DoF) if k u � = k v , DOF=7 There is no scale ambiguity. Hajder, Csetverikov (Faculty of Informatics) Computer Vision 19 / 85

Camera Models Weak-perspective camera orthographic projection image plane Orthogonal projection can be applied if object is far from the camera depth is relatively static Model has 5 degree of freedom (DoF) R , t 1 , t 2 Hajder, Csetverikov (Faculty of Informatics) Computer Vision 20 / 85

Camera Models Comparison of camera models Outline Camera Models 1 Perspective (pin-hole) camera Weak-perspective camera Comparison of camera models Back-projection to 3D space Homography 2 Homography estimation Non-linear estimation by minimizing geometric error Camera Calibration 3 Calibration by a spatial object Calibration using a chessboard Radial distortion Summary 4 Hajder, Csetverikov (Faculty of Informatics) Computer Vision 21 / 85

Camera Models Comparison of camera models Affine camera General affine camera u = M 2 × 3 X + t 8 degrees of freedom M 2 × 3 is a 2 × 3matrix with rank two Hierarchy of affine cameras general affine camera ⇓ more constraints, less DoFs Hajder, Csetverikov (Faculty of Informatics) Computer Vision 22 / 85