SIMULTANEOUS GEOMETRIC AND COLORIMETRIC CAMERA CALIBRATION Ilmenau, - PowerPoint PPT Presentation

SIMULTANEOUS GEOMETRIC AND COLORIMETRIC CAMERA CALIBRATION Ilmenau, 7th October 2010 Daniel Kapusi, Philipp Prinke, Rainer Jahn, Darko Vehar, Rico Nestler, Karl-Heinz Franke Zentrum für Bild- und Signalverarbeitung e. V. www.zbs-ilmenau.de

Outline SIMULTANEOUS GEOMETRIC AND COLORIMETRIC CAMERA CALIBRATION  Motivation  Calibration Target  Target Feature Detection  Geometric Calibration  Stereo Rectification  Color Calibration  Conclusion 07. 10. 2010 16. Farbworkshop 2

Application Scenario: Robot Work Area 3D scene modeling D D Motivation D Motivation D D stereo vision  Redundant scene observation camera from multiple points of view Calibration Target  Stereo vision for more robust Target Feature object recognition D Detection D Requirements monocular vision camera Geometric  Unified coordinate system for Calibration human all sensors Stereo Rectification robot  Unified colorimetric perception D D D D D between all sensors Color Calibration Vision based safety system: Avoiding collisions between robots and humans! Geometric and Conclusion Colorimetric Camera Calibration 07. 10. 2010 16. Farbworkshop 3

Chessboard Calibration Target With Colorimetric Features Motivation Calibration Target Calibration Target Target Feature Detection Geometric Calibration Colorimetric features Geometric features  24 circles filled with well distributed Stereo Rectification  Different parity in number of reference colors (natural and edges in horizontal and vertical primary colors as well as grey- direction [1] Color Calibration balanced fields) in the style of the Orientation is clearly detectable Macbeth [2] color checker Conclusion [1] Jean-Yves Bouguet. (2010, July) Camera Calibration Toolbox for Matlab. [2] C. S. McCamy, H. Marcus, and J. G. Davidson, "A Color- Rendition Chart,“ [Online]. http://www.vision.caltech.edu/bouguetj/calib_doc/index.html Journal of Applied Photographic Engineering, vol. 2, no. 3, pp. 95-99, Summer 1976. 07. 10. 2010 16. Farbworkshop 4

Values Of The ColorChecker Motivation Calibration Target Calibration Target Target Feature Detection Geometric Calibration Stereo Rectification Color Calibration Conclusion [3] Danny Pascale, "RBG coordinates of the Macbeth ColorChecker," The BabelColor Company, Montreal, Quebec, Canada, Comparison 2006. 07. 10. 2010 16. Farbworkshop 5

Detecting Edges On The Chessboard Using OpenCV 1. Step: rough edge detection [4]  Adaptive threshold binarization Motivation  Separate black squares by dilatation of white squares  Find and simplify contours to 4 edge-points at each black square Calibration Target 2. Step: sub pixel accurate edge detection [4] Target Feature Target Feature Detection Detection  Using the fact, that the dot product of two orthogonal vectors is zero Geometric  Consider all points within a neighborhood around the real corner position Q Calibration  Iterative solution of a linear system of equations Stereo Rectification p i – position vector to point P i q – position vector to point Q Color Calibration Conclusion     T I P p q ( ) ( ) 0 [4] Gary Bradski and Adrian Kaehler, Learning OpenCV. Sebastopol, USA/CA: O´Reilly i i Media, 2008 07. 10. 2010 16. Farbworkshop 6

Detecting Edges On The Chessboard Using OpenCV Motivation Calibration Target Multipose-Calibration Target Feature Target Feature Detection Detection Geometric Calibration Stereo Rectification Color Calibration Conclusion 07. 10. 2010 16. Farbworkshop 7

Detecting Color Features Calculating the start color value from each square  Middle position of the chessboard square Motivation  Median value from direct neighborhood Recursive region growing Calibration Target  Using a uniform criterion as a tolerance threshold Target Feature Target Feature Detection Detection Geometric Calibration Stereo Rectification Color Calibration Conclusion Average the color values from the segmented region 07. 10. 2010 16. Farbworkshop 8

Camera Parameters Coordinate transformation World Motivation   P  R t Calibration Target Camera Image Camera   f 0 H   x x Target Feature  World   M 0 f H y y Detection    0 0 1  Image Geometric Geometric   P Calibration Calibration    p M R t Stereo Rectification Extrinsic camera parameters Intrinsic camera parameters Color Calibration R – Rotation matrix Hx, Hy – Principal point coordinates t – Translation vector F – Focal length Conclusion [5] Roger Y. Tsai, "An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision," IEEE Journal of Robotics and Automation , vol. 3, no. 4, pp. 323-344, August 1987. [6] Zhengyou Zhang, "A Flexible New Technique for Camera Calibration," IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 22, no. 11, pp. 1330-1334, November 2000. 07. 10. 2010 16. Farbworkshop 9

Brown-Conrady-Distortion-Model radial distortion + tangential distortion radial Motivation           2 2 4 6 2 x x 1 k r k r k r 2 p x y p ( r 2 x ) v u 1 2 3 1 u u 2 u Calibration Target           2 2 4 6 2 y y 1 k r k r k r p ( r 2 y ) 2 p x y v u 1 2 3 1 u 2 u u Target Feature Detection   2 2 2 r x y tangential u u Geometric Geometric Calibration Calibration radial tangential distortion distortion Distortion parameters Stereo Rectification k1, k2, k3 - radial distortion coefficients p1, p2 - tangential distortion coefficients Color Calibration optical centre Conclusion [7] Alexander Eugen Conrady, "Decentering lens systems," Monthly notices of the Royal Astronomical Society , vol. 79, pp. 384-390, April 1919. [8] Duane C. Brown, "Decentering Distortion of Lenses," Photometric Engineering , vol. 32, no. 3, pp. 444-462, 1966. 07. 10. 2010 16. Farbworkshop 10

Overall Calibration Procedure BV-System BV-System MonoVision MonoVision BV-System BV-System StereoVision StereoVision BV-System BV-System StereoVision StereoVision Motivation R+T R+T R+T D D D D D D D D Calibration Target D D R+T R+T R+T R+T Target Feature Detection MonoVision MonoVision BV-System BV-System MonoVision MonoVision BV-System BV-System D D Geometric Geometric D D Calibration Calibration R+T Stereo Rectification R+T R+T R+T D D D D D D D StereoVision D n o m Color Calibration BV-System i D s D e i StereoVision V t StereoVision s o y e S BV-System BV-System r - e V t S B MonoVision MonoVision Conclusion BV-System BV-System R+T R+T R+T R+T 11 07. 10. 2010 16. Farbworkshop

Stereo Rectification[4] Extraction of depth information  Assuming a standard stereo geometry Motivation  Determination of the horizontal mismatch (disparity) Calibration Target from a stereoscopic view (Correspondence analysis)  Calculating the distance by simple triangulation Target Feature Detection Geometric Calibration Stereo Rectification Stereo Rectification Color Calibration Conclusion [4] Gary Bradski and Adrian Kaehler, Learning OpenCV. Sebastopol, USA/CA: O´Reilly Media, 2008 07. 10. 2010 16. Farbworkshop 12

Stereo Rectification[4] left right Motivation Calibration Target Target Feature Detection unrectified image Geometric Calibration Stereo Rectification Stereo Rectification Color Calibration rectified images Conclusion [4] Gary Bradski and Adrian Kaehler, Learning OpenCV. Sebastopol, USA/CA: O´Reilly Media, 2008 07. 10. 2010 16. Farbworkshop 13

Color Calibration Target based global color calibration[9]  Corresponding nominal and actual color values Motivation  Calculation of the transformation rule which causes the smallest middle aberration Linear transformation models Non-linear transformation model Calibration Target  Scaling (+Offset)  Polynomial regression  Linear Transformation Target Feature Detection  Linear Regression Geometric Calibration RGB nominal color C valence Stereo Rectification nom Calculation of actual color Transformation RGB Color Calibration valence Color Calibration Rule C act Conclusion [9] ZBS e. V. (2007, April) zbs CCal-Bibliothek Benutzerdokumentation. [Online]. http://www.zbs-ilmenau.de/intern/ccalc/ZBSColCalib.pdf 07. 10. 2010 16. Farbworkshop 14

Conclusion D D D D D stereo vision camera Requirements to 3D scene modeling Motivation  Unified coordinate system for all sensors D D monocular vision camera  Unified colorimetric perception between all sensors Calibration Target human robot D Geometric and Colorimetric Camera Calibration D D D D Target Feature Detection  Using a chessboard target with integrated color markers Geometric  Sub pixel accurate edge detection of the chessboard squares Calibration  Calculating intrinsic camera parameters, distortion coefficients and geometric relations to adjacent sensors from multiple target Stereo Rectification poses Color Calibration  Calculate rectification parameters for stereo vision systems  Target based color calibration Conclusion Conclusion Available as an ANSI C conform software library 07. 10. 2010 16. Farbworkshop 15