CAMERA CALIBRATION INEL 6088 Computer Vision Ch. 18 Davies 4 th - PowerPoint PPT Presentation

CAMERA CALIBRATION INEL 6088 Computer Vision Ch. 18 Davies 4 th edition Ch. 12 Jain et. el.

REFERENCES • Discussion of Tsai’s calibration method by Berthold Horn • Original paper by Roger Tsai • Chapter 18 (21) on textbook by Davies, 4 th (3 rd ) edition. • Chapter 12 on textbook by Jain et. al.

WHAT IS CAMERA CALIBRATION? • Finding the camera parameters that affects the image acquisition process. • Extrinsic Parameters • Define the location of the Camera reference frame (CRF) with respect to a known world reference frame (WRF) . • Intrinsic Parameters • Link the pixel coordinates to the corresponding coordinates in the CRF 3

WHAT IS CAMERA CALIBRATION? P w z c y c • Extrinsic Parameters x c Z w • Rotation or Orientation r P c • Translation w 0 • Perspective Y 0 Y w Z 0 X 0 X w 4

Yc R, T Yw Zc Xw Xc Pc Pw Zw Relation between camera and world coordinate frames.

EXTRINSIC PARAMETERS • Identify the transformation between the unknown camera and known world reference frames. - 3-D translation vector, T , that relate the origins of the two frames - 3 × 3 rotation matrix, R orthogonal matrix (R T R=RR T =I) brings axes of the two frames into each other P c = R ( P w − T ) P c = R P w − T

Shorthand notation: c a = cos(a), s b = sin(b)       1 0 0 c ϕ 0 s ϕ c θ − s θ 0 r x = 0 c ψ − s ψ r y = 0 1 0 r z = s θ c θ 0       s ψ c ψ − s ϕ c ϕ 0 0 0 0 1     r 11 r 12 r 13 c θ c ϕ − s θ c ϕ s ϕ  = r x r y r z = r 21 r 22 r 23 c θ s ψ s ϕ + c ψ s θ − s ψ s ϕ s θ + c ψ c θ − s ψ c ϕ R =    r 31 r 32 r 33 − c ψ s ϕ c θ + s ψ s θ c ψ s ϕ s θ + s ψ c θ c ψ c ϕ

W to C: Rotation followed by translation     X c X w  = R  + T Y c Y w   Z c Z w X c = r 11 X w + r 12 Y w + r 13 Z w + T x Y c = r 21 X w + r 22 Y w + r 23 Z w + T y X c = r 31 X w + r 32 Y w + r 33 Z w + T z

W to C: Translation followed by rotation     X c X w − T x Y c Y w − T y  = R    Z c Z w − T z X c = r 11 ( X w − T x ) + r 12 Y w + r 13 Z w Y c = r 21 X w + r 22 ( Y w − T y ) + r 23 Z w Z c = r 31 X w + r 32 Y w + r 33 ( Z w − T z )

INTRINSIC PARAMETERS • o x , o y = Position of image centre (Principal Point) • Focal length f • d x , d y = pixel dimensions • k 1 = radial distortion coefficient

Image sensor sizes Industrial cameras Assume: ½ inch sensor (0.64cm × 0.48cm) sampled as a 640x480 pixel image d x = 0.01mm/pixel = d y τ 1 = 1 = no radial distortion x = τ 1 d x (c - u 0 ) = 0.01(c - 320) y = -d y (r - v 0 ) = -0.01(r - 240) u and v in mm

Perspective Camera Model x = f X x = f X ¯ Z Z y = f Y y = f Y ¯ Z Z Weak-perspective camera model Perspective camera model (difference in distance to scene points is small compared to average distance)

Transformation between Camera and Image frame coordinates Ignoring the lenses’ geometric distortions and assuming that the CCD array is made of a rectangular grid of photosensitive elements, x = d x ( c − o x ) y = − d y ( r − o y ) where (o x , o y ) represent the coordinates of the image center (the principal point), and (d x , d y ) the effective pixel size (in millimeters) in the horizontal and vertical directions, respectively. Transformation between World and sensor coordinates P c = R P w + T x = s x ( x im − o x ) = − f R T 1 P w + T R T 3 P w + T y = − s y ( y im − o y ) = − f R T 2 P w + T R T 3 P w + T R 1 , R 2 and R 3 are 3-D vectors formed by the 1 st , 2 nd and 3 rd rows of matrix R , respectively.

CALIBRATION METHOD BY TSAI (1) • P = [ X W , Y W , Z W ] : point in 3-D world coordinates • p = [ x i , y i ] : point in the real image plane - x axis : horizontal, points to the right - y axis : vertical, points upward • a = [ r, c] : row, column pixel coordinates on the pixel array - r axis : vertical, points downward - c axis : horizontal, points to the right (1)Tsai, R. 1987. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf cameras and lenses. IEEE Trans. Robotics and Automation, v3(4).

Move calibration to different positions

Calibration pattern: 7 corners + 3 side-centered points Y W Example setup 10 (see tsai.sci) 3 2 world reference 9 12 frame 7 4 1 10 X W 8 8 5 6 X W Y W Z W Z W virtual image 10 0 0 plane 10 8 0 y C 0 8 0 0 8 12 0 0 12 z C y i 10 0 12 10 8 12 x C x i 5 4 12 5 8 6 10 4 6 camera reference World coordinates (cm) frame

Image points are identified by number. Black lines indicate the transparent box “edges”. Face middle-points are indicated by the slotted lines to help in visualising the results. 7 4 8 9 6 10 2 5 3 1 Result of the rotation 20 ◦ ,180 ◦ and 10 ◦ around x, y and z axes, plus translation and projection.

The sensor coordinates ( x i , y i ) for the k th world point at ( X W k , Y W k , Z W k ) is d x ( c k − o x ) = − f R T = − f r 11 X W k + r 12 Y W + r 13 Z W 1 P + t x k + t x x i k = k R T r 31 X W k + r 32 Y W + r 33 Z W 3 P + t z k + t z k − d y ( r k − o y ) = − f R T = − f r 21 X W k + r 22 Y W + r 23 Z W 2 P + t y k + t y y i k = k R T r 31 X W k + r 32 Y W + r 33 Z W 3 P + t z k + t z k x i r 11 X W k + r 12 Y W + r 13 Z W k + t x k k = y i r 21 X W k + r 22 Y W + r 23 Z W k + t y k k Cross-multiplying, x i k ( r 21 X W k + r 22 Y W + r 23 Z W k + t y ) = y i k ( r 11 X W k + r 12 Y W + r 13 Z W k + t x ) k k r 21 k + r 22 + r 23 k = r 11 k + r 12 + r 13 k + t x x i k X W x i k Y W x i k Z W k + x i y i k X W y i k Y W y i k Z W y i k k k t y t y t y t y t y t y t y k = r 11 k + r 12 + r 13 k + t x k − r 21 k − r 22 − r 23 x i y i k X W y i k Y W y i k Z W y i x i k X W x i k Y W x i k Z W k k k t y t y t y t y t y t y t y � y i k X W � k y i k Y W � � k � y i k Z W � � � k � � r 11 r 12 r 13 t x r 21 r 22 r 23 x i � � y i k = � � t y t y t y t y t y t y t y k � � − x i k X W � � k � − x i k Y W � � � k − x i k Z W k

This equation, applied to N points, yields a matrix equation in terms of a row vector of image x coordinates x , a 7-element row vector containing the parameters, and a N-column, 7-row matrix M x = cM Since in general M is not square (more than 7 data points) c = xM t ( MM t ) − 1 where the − 1 exponent represents matrix inversion, and h i r 11 r 12 r 13 r 21 r 22 r 23 tx c = t y t y t y t y t y t y t y Since the R matrix is orthogonal, r 2 11 + r 2 12 + r 2 13 = 1. This from the first 3 elements of c , 3 = r 2 11 + r 2 12 + r 2 13 c 2 1 + c 2 2 + c 2 = 1 /t 2 y t 2 y 1 t y = ± p c 2 1 + c 2 2 + c 2 3

r 11 = c 1 t y ; r 12 = c 2 t y ; r 13 = c 3 t y t x = c 4 t y ; r 21 = c 5 t y ; r 22 = c 6 t y ; r 23 = c 7 t y To determine the sign of t y , observe that since the reference pattern must be in front of the camera, for a point p , x i p and r 11 X W p + r 12 Y W + r 13 Z W p + t x p (which determines the sign of x c , the point’s horizontal coordinate in the camera reference frame) must have opposite signs. Thus if x i p ( r 11 X W p + r 12 Y W + r 13 Z W p + t x ) > 0 p the signs of first two rows of R and the first two components of T must be changed. To find the remaining rotation parameters, observe that since the 3 axes are orthogonal and the rotation matrix rotates the original unit vectors, R 3 = R 1 × R 2 where R 3 , R 1 and R 2 are vectors formed from the rows of R and × represents the vector product.

To find f and t z , go back to the equation for x i for an arbitrary point k k = − f r 11 X W k + r 12 Y W + r 13 Z W k + t x x i k r 31 X W k + r 32 Y W + r 33 Z W k + t z k rearrange the equation x i k ( r 31 X W k + r 32 Y W + r 33 Z W k + t z ) = − f ( r 11 X W k + r 12 Y W + r 13 Z W k + t x ) k k f ( r 11 X W k + r 12 Y W + r 13 Z W k + t x ) + x i k t z = − x i k ( r 31 X W k + r 32 Y W + r 33 Z W k ) k k and express it in vectorial form ⇤  � f = − x i k ( r 31 X W k + r 32 Y W + r 33 Z W ⇥ r 11 X W k + r 12 Y W + r 13 Z W x i k + t x , k ) k k k t z Using the N points yields a matrix equation that can be solved for f and t z .  � f = b A t z  f � = ( A t A ) − 1 A t b t z