Cali libration for r Lid idar-Stereo Vehicle Sensor Setups Carlos - PowerPoint PPT Presentation

Automatic Extri rinsic Cali libration for r Lid idar-Stereo Vehicle Sensor Setups Carlos Guindel, Jorge Beltrán, David Martín and Fernando García Intelligent Systems Laboratory · Universidad Carlos III de Madrid IEEE 20th International Conference on Intelligent Transportation Systems Yokohama · 17 October 2017

Agenda 2 Motivation Calibration algorithm Synthetic test suite Results Conclusion Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Agenda 3 Motivation Calibration algorithm Synthetic test suite Results Conclusion Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Perception systems in vehicles 4 • Topologies with complementary sensory modalities IVVI 2.0 Research Platform Range Cameras scanners Stereo- Multi-layer vision 3D lidar systems scanner • Appearance • High accuracy information • 360º Field of • Cost-effective View • Dense 3D info. Correspondence Extrinsic Ovelapping Data fusion between data calibration FOVs representations required Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Previous works 5 • Camera-to-range calibration in robotic/automotive platforms • Complex setups / lack of generalization ability • Strong assumptions are usually made: sensor resolution, limited pose range, environment structure,… Velas et al., WSCG 2014 Geiger et al., ICRA 2012 • Assessment of calibration methods • Ground-truth of extrinsic parameters cannot be obtained in practice Levinson & Thrun, RSS 2013 Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Proposal overview 6 • Stereo-vision system – multi-layer lidar calibration • Suitable for use with different models of lidar scanners (e.g. 16-layer) • Very different relative poses are allowed • Performed within a reasonable time using a simple setup Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Agenda 7 Motivation Calibration algorithm Synthetic test suite Results Conclusion Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Calibration algorithm 8 • Calibration target • Single point of view • Holes visible from the camera and intersected by at least 2 lidar beams • No alignment required • Process overview 𝝄 CL = C AMERA C AMERA C AMERA 𝑢 𝑦 Target Circles 𝑢 𝑧 Data segmentation segmentation 𝑢 𝑨 Registration 𝜚 L IDAR L IDAR L IDAR 𝜄 Target Circles Data 𝜔 segmentation segmentation Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Data representation 9 C AMERA C AMERA C AMERA Target Circles Data segmentation segmentation Registration L IDAR L IDAR L IDAR Target Circles Data segmentation segmentation Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Data representation 10 3D point clouds, 𝒬 0 = { 𝑦, 𝑧, 𝑨 } • C AMERA Left image Stereo matching 𝑑 Point cloud: 𝒬 0 Data Right image Stereo matching • Accuracy in the depth estimation is required ( SGM ) Border localization problem will be tackled using intensity • L IDAR : 𝒬 0 𝑚 L IDAR 𝑚 Point cloud: 𝒬 0 Data Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Target segmentation · Step 1 11 C AMERA C AMERA 𝝄 CL = 𝑢 𝑦 Target 𝑢 𝑧 segmentation 𝑢 𝑨 L IDAR L IDAR 𝜚 𝜄 Target segmentation 𝜔 Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Target segmentation · Step 1 12 Extracting the points belonging to discontinuities in the target • • Successive segmentations: 𝒬 𝑗 0 = { 𝑦, 𝑧, 𝑨 } ⊆ 𝒬 𝑗 0 −1 C AMERA /L IDAR Plane model Remove pts. far Point clouds: 𝒬 0 extraction from the planes Target segmentation Plane model extraction Step 1 • Random sample consensus (RANSAC) • Tight threshold ( 1 cm ) and requirement for the plane to be roughly parallel to the vertical axis ( tol: 0.55 rad) 𝑚 𝑑 L IDAR : 𝒬 C AMERA : 𝒬 1 1 Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Target segmentation · Step 2 13 C AMERA Point cloud: 𝒬 𝑑 Target 1 Keep segmentation discontinuities Left image Sobel filtering Step 2 𝑚 C AMERA : Sobel edges C AMERA : 𝒬 2 𝑑 L IDAR : 𝒬 2 L IDAR for every point in 𝒬 𝑚 1 Target Filter out (50 cm) segmentation Step 2 Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Circles segmentation · Step 1 14 C AMERA 𝝄 CL = 𝑢 𝑦 Circles 𝑢 𝑧 segmentation 𝑢 𝑨 L IDAR 𝜚 𝜄 Circles segmentation 𝜔 Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Circles segmentation · Step 1 15 • Getting rid of the points not belonging to the circles: target boundaries C AMERA Point cloud: 𝒬 2 𝑑 3D-line RANSAC Point cloud: 𝒬 3 𝑑 Circles segmentation Geometrical Step 1 constraints 𝑚 C AMERA : 𝒬 3 𝑑 L IDAR : 𝒬 3 L IDAR Keep only the rings where a circle is possible • Circles segmentation • Remove the outer points Step 1 Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Circles segmentation · Step 2 16 • Detecting the center of the holes Geometrical constraints C AMERA /L IDAR Alignment with 2D Circle Circles Point cloud: 𝒬 3 XY plane RANSAC segmentation Step 2 Undo the 4 x centers 4 x centers + alignment coordinates radius Circle model extraction • 2D search: only three points are required Lidar Camera Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Circles segmentation · Step 3 17 • Robustness against noise 4 x center C AMERA /L IDAR 𝑢 0 coordinates Circles segmentation 4 x center 𝑢 1 tol: 2 cm coordinates Step 3 Euclidean clustering … 4 x cluster 4 x center centroids 𝑢 𝑂 coordinates L IDAR C AMERA Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Registration 18 𝝄 CL = 𝑢 𝑦 𝑢 𝑧 𝑢 𝑨 Registration 𝜚 𝜄 𝜔 Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Registration 19 C AMERA 4 x reference Circles points, 𝒒 𝑑 𝑗 segmentation Registration L IDAR 4 x reference Circles 𝑗 points, 𝒒 𝑚 segmentation Step 1 • Pure translation • Overdetermined system of 12 equations 𝝄 CL = Translation 𝑢 𝑦 𝑗 − ഥ 𝑗 𝒖 𝐷𝑀 = ഥ 𝒒 𝑚 𝒒 𝑑 𝑢 𝑧 • Column-pivoting QR decomposition 𝑢 𝑨 Composition 𝜚 Step 2 𝜄 Translation 𝜔 • Iterative Closest Points (ICP) + Rotation Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Agenda 20 Motivation Calibration algorithm Synthetic test suite Results Conclusion Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Synthetic Test Suite 21 • Our proposal for quantitative assessment of calibration algorithms • Exact ground-truth, but also noise and real constraints • Simulation of sensors and their environment based on Gazebo • Different calibration scenarios Calibration target Stereo-vision system model Velodyne models Gazebo models, plugins and worlds available at http://wiki.ros.org/ velo2cam_gazebo Open source · GPLv2 License Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Agenda 22 Motivation Calibration algorithm Synthetic test suite Results Conclusion Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Experimental setup 23 • Using the synthetic test suite • Nine different calibration setups • 7 simple setups to evaluate the parameters of the transform • 2 challenging situations • Gaussian noise added to the sensor measurements • Models simulated with real parameters • 12 cm stereo baseline and 16-layer lidar Translation error (linear) 𝑺 𝒖 𝑓 𝑢 = ‖𝒖 − 𝒖 𝒉 ‖ Rotation error (angular) 𝑓 𝑠 = ∠(𝑺 −𝟐 𝑺 𝒉 ) Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017

Experiments 24 • Accumulation of cluster centroids over 𝑂 frames C AMERA /L IDAR Circles • 𝑂 images and 𝑂 point clouds processed segmentation • Not every window provides clusters to be accumulated Step 3 Selection of the length of the window, 𝑂 Translation error (linear) Rotation error (angular) Automatic Extrinsic Calibration for Lidar-Stereo Vehicle Sensor Setups C. Guindel, J. Beltrán et al. · ITSC 2017