Camera Models Shao-Yi Chien Department of Electrical Engineering - PowerPoint PPT Presentation

Camera Models 簡韶逸 Shao-Yi Chien Department of Electrical Engineering National Taiwan University Fall 2019 1

Outline • Camera models [Slides credit: Marc Pollefeys] 2

Camera Obscura: the Pre-Camera • First idea: Mo-Ti, China (470BC to 390BC) • First built: Alhazen, Iraq/Egypt (965 to 1039AD) Illustration of Camera Obscura Freestanding camera obscura at UNC Chapel Hill Photo by Seth Ilys

Camera Obscura Camera Obscura, Reinerus Gemma-Frisius, 1544

Pinhole Camera Model T T  ( , , ) ( / , / ) X Y Z fX Z fY Z     X X         0 fX f         Y Y     0 fY f       Z Z            1 0  Z      1   1 

Pinhole Camera Model   X         1 0 fX f         Y    1 0 fY f       Z         1 1 0       Z     1 x  PX    P diag ( , , 1 ) I | 0 f f

Principal Point Offset   T T  ( , , ) ( / , / ) X Y Z fX Z p fY Z p x y T ( , ) p x p principal point y     X X          0 fX Zp f p   x x       Y Y      0 fY Zp f p       x y Z Z            1 0  Z      1   1 

Principal Point Offset   x  K I | 0 X cam   f p x    K f p   y Calibration Matrix   1  

Camera Rotation and Translation   ~ ~ ~ cam  X R X - C   X   ~         R RC Y R R C     X cam X      0 1  Z  0 1        1   x  K I | 0 X cam

Camera Rotation and Translation   X ~   x KR I | C x  PX   ~ P    K R | t t R C Internal Camera Parameter External Camera Parameter Internal Orientation Exterior Orientation Intrinsic Matrix Extrinsic Matrix

CCD Camera Non-square pixels      m f p x x      K m f p 𝑛 𝑧     x y      1   1     p x x     K 𝛽 𝑧 p   x y   1  

Finite Projective Camera       s p p x x x x         K K p p     s: skew parameter, x x y y =0 for most normal cameras       1 1     ~   P KR I | C 11 dof (5+3+3) non-singular decompose P in K,R,C?     ~   P      1 p M | p K, R M C M RQ 4 4 P 4 : Last column of P {finite cameras}={P 4x3 | det M ≠ 0} If rank P=3, but rank M<3, then cam at infinity

Camera Anatomy • Camera center • Column points • Principal plane • Axis plane • Principal point • Principal ray

Camera Center Camera Center (C): Null-space of camera projection matrix PC  0    λA λ)C Proof: X (1     λPA λ)PC x PX (1 For all A all points on AC project on image of A, therefore C is camera center Image of camera center is (0,0,0) T , i.e. undefined    1 M p    Finite cameras: 4 C    1    d     Infinite cameras: , Md 0 C    0  The camera center is a point at infinity

Column Vectors   0   1        p p p p p   2 1 2 3 4 0     0 p 1 , p 2 , p 3 : vanishing points of the world coordinate X, Y, and Z axes Image points corresponding to X,Y,Z directions and origin

Row Vectors

Row Vectors   X   T   1 p   x     Y   T  2   p y     Z     T 3   0 p        1    X   T   1 0 p       Y   T  2   p y     Z   T   3     p w       1 note: p 1 ,p 2 dependent on image reparametrization

The Principal Point     ˆ 3 p , , , 0 p p p 31 32 33 principal point 𝑛 3   ˆ 3 3 x P p Mm 0

The Principal Axis Vector vector defining front side of camera 3 m 3           T v det M m 0 , 0 , 1 x P X K I | 0 X cam cam cam  4 P P k  v v k cam cam (direction unaffected)    ~   k   Direction unaffected because P KR I | C M | p 4  det( R ) 0 The principal axis vector v =det(M)m 3 is directed towards the front of the camera

Action of Projective Camera on Point Forward projection x  PX      x PD M | p D Md 4 Back-projection PC  0        1  PP  T T X P x P P PP I (pseudo-inverse)      λ λC X P x For finite camera  -1 d M x         μx -1 -1 -1 M x - M p M - p            λ μ 4 4 X             0 1 1 C D

Depth of Points   ~ ~   T T T      3 3 3 P X P X C m X C w (PC=0) (dot product) 3   det 0 ; m 1 M If , then m 3 unit vector in positive direction    T X X,Y,Z,T sign(detM )   w  depth X; P 3 m T

Camera Matrix Decomposition Finding the camera center PC  0 (use SVD to find null-space)            det p , p , p det p , p , p X Y 2 3 4 1 3 4            det p , p , p Z det p , p , p T 1 2 4 1 2 3 Finding the camera orientation and internal parameters M  KR (use RQ decomposition ~QR) (if only QR, invert) =( ) -1 = -1 -1 R R Q Q

When is Skew Non-zero?    s p x x arctan(1/s)     K p   x y g   1  1  for CCD/CMOS, always s=0 Image from image, s ≠0 possible (non coinciding principal axis) resulting camera: HP

Euclidean vs. Projective general projective interpretation   1 0 0 0          P 3 3 homography 0 1 0 0 4 4 homography      0 0 1 0  Meaningfull decomposition in K,R,t requires Euclidean image and space Camera center is still valid in projective space Principal plane requires affine image and space Principal ray requires affine image and Euclidean space

Cameras at Infinity Camera center at infinity   d    P 0   det M 0  0  Affine and non-affine cameras Definition: affine camera has P 3T =(0,0,0,1)

Affine Cameras

Affine Cameras ~    1T 1T r r C     ~ ~     2T 2T P KR I | C K r r C   0 ~   ~  3T 3T   r r C r 3T C   d 0   ~ ~       1T 1T 3 1T 1T r r C - r r r C t       ~ ~     2T 2T 3 2T 2T P K r r C - r K r r C     t   t ~      3T 3T 3 3T r r C - r r t d     t modifying p 34 corresponds to moving along principal ray

Affine Cameras now adjust zoom to compensate ~      1T 1T / r r C d d 0 t     ~   2T 2T P K / r r C   d d   0 t t     3T  1  r d   t ~    1T 1T r r C   ~ d   2T 2T t K r r C   d   0 3T r / d d d   0 0 t ~    1T 1T r r C   ~    2T 2T P lim P K r r C    t   t   0 d   0

Error in Employing Affine Cameras    αr 1 βr 2 point on plane parallel with principal    X   plane and through origin, then  1    P X P X P X  0 t      αr 1 βr 2 3 r    X   general points  1  ~ ~     x x     ~ ~         x P X K x P X K y y  affine proj 0      Δ     d d 0 0 x proj x affine x 0

Error in Employing Affine Cameras     x - x x - x affine proj proj 0 d 0 Approximation should only cause small error 1.  much smaller than d 0 2. Points close to principal point (i.e. small field of view)