Low-Drift, Efficient Visual Odometry and SLAM Utilizing - PowerPoint PPT Presentation

Low-Drift, Efficient Visual Odometry and SLAM Utilizing Environmental Structures Seung Jae Lee 1 Pyojin Kim 2 sjlazza@gmail.com pjinkim1215@gmail.com 1 Seoul National University, South Korea 2 Simon Fraser University, Canada 2 nd International Workshop on Lines, Planes and Manhattan Models for 3-D Mapping (LPM 2019) May 23, 2019

Motivation Importance of Camera Rotational Motion □  The Main Source of Positional Inaccuracy in VO & SLAM  Accurately Estimated Translational Motion [1]  Rotations Causing Nonlinearity in VO & SLAM Estimated (left) and True (right) Camera Orientation 2

Contents Part 1: Absolute Camera Orientation from Multiple Lines and Planes [2-3] □ Part 2: Linear SLAM Formulation with Absolute Camera Rotation [4] □ 3

Related Work Separate Rotation & Translation Estimation □ Cvisic et al., '15 [6] Bazin et al., '10 [7] Kaess et al., '09 [5]  They cannot estimate drift-free rotational motion of the camera. Drift-Free Rotation Estimation in Structured Environments □ Bazin et al., '12 [8] Zhou et al., '16 [9] Straub et al., '18 [10]  Sensitive and fragile to  Require at least two  Require GPU & two outlier lines orthogonal planes orthogonal planes 4

Main Contributions A New Approach for Drift-Free Orientation from Both Lines and Planes □ A New Way for Accurate Translation on the De-Rotated Reprojection Error □ Evaluation on the Public RGB-D and Author-collected Datasets □ Lines Planes Structured Environment Surface Projection Normal Structured Environment Exhibiting Orthogonal Regularities For full details, refer to [2-3]

Overview of the Proposed VO LPVO [2] (Line and Plane based Visual Odometry) □ Manhattan Frame Depth Image Point Cloud Normal Extraction Tracking Line Detection VD Extraction De-rotated Reproj. Error Minimization RGB Image Point Tracking 6

Drift-Free Rotation Estimation Multiple Lines & Planes with Mean Shift □ Vanishing Direction Two Parallel Line Segments Normal Vectors of the Great Circles Surface Normal Vectors Gaussian Sphere 7

Translation Estimation De-rotated Reprojection Error Minimization □ 𝐮 ∗ Optimal 3-DoF Translation i -th Tracked Point Feature 𝑁 𝑂 ′2 𝐮 𝐮 ∗ = arg min 2 𝐮 + 𝑠 𝑗2 2 𝐮 + ෍ ෍ 𝑠 𝑗1 𝑠 𝑗 𝐮 Known Unknown 𝑗=1 𝑗=1 : De-rotated Reproj. Error w/ Depth 𝑠 𝑗1 𝐮 , 𝑠 𝑗2 𝐮 : Rotation ′ 𝐮 : De-rotated Reproj. Error w/o Depth 𝑠 𝑗 : Translation : # of Points w/ Depth : # of Points w/o Depth 8

Experiment Setup ICL-NUIM Dataset (~9.01 m) : only a single plane TUM RGB-D Dataset (~22.14 m) Building-scale Corridor Dataset (~120 m) We compare LPVO [2] with ORB [11] , DEMO [1] , DVO [12] , MWO [9] , OPVO [13] .  9

Qualitative Analysis with Floorplan Only LPVO can estimate 6-DoF Nearly 8x more accurate 10

Qualitative Analysis with Floorplan Video available at https://youtu.be/mt3kbv2TJZw 11

Quantitative Analysis with True Data Rotation Error [deg] Rotation error causes failure Average rotation error is ~0.2 deg Translation Error [m] On average, 5x more accurate Frame Index 15 Hz @ 10 FPS 12

Linear RGB-D SLAM with Planar Features

Motivation Development of Simple & Linear SLAM Approach □  SLAM is a High Dimensional Nonlinear Problem  SLAM as A Linear Least Squares Given the Rotation [14]  Planar Features in Low-Texture Indoor Environments Odometry Initialization Optimum Torus Effectiveness of the Prior Rotation Information [14] 14

Related Work Recent Plane-based SLAM Approaches □ 15

Main Contributions An Orthogonal Plane Detection Method in Structured Environments □ A New, Linear Kalman Filter SLAM Formulation □ Evaluation and Application to Augmented Reality (AR) □ Linear RGB-D SLAM (L-SLAM) with a Global Planar Map For full details, refer to [4]

Pipeline of the Proposed SLAM L-SLAM [4] (Linear SLAM in Planar Environments) □ L-SLAM Orthogonal Plane Detection Depth Linear Point Surface SLAM Cloud Normals Drift-Free within Rotation Tracking Kalman Line Vanishing Detection Directions Filter RGB Translation Point Detection & Tracking Estimation LPVO 17

Orthogonal Plane Detection The Plane Model in RANSAC [18] □ 𝑣, 𝑤 : The Normalized Image Coordinates : The Measured Disparity Detected Planes Overlaid on the RGB Image 18

Linear SLAM Formulation in KF KF State Vector Definition □  State Vector in Linear KF  3-DoF Camera Translation  1-D Distance (Offset) of the Plane 3-DoF rotational motion is PERFECTLY compensated by LPVO [2] .   Camera, map position are expressed in global Manhattan map frame. 19

Linear SLAM Formulation in KF Propagation Step (Predict) with LPVO □  Process Model with LPVO where ,  Only 3-DoF camera translation is propagated with LPVO method.  A constant position model is used in 1-D map position (& alignment). 20

Linear SLAM Formulation in KF Correction Step (Update) with Orthogonal Planes □  Measurement Model where  Observation model is nothing but a distance from the orthogonal plane.  1-D map positions are also updated in linear KF framework. 21

Evaluation & AR Application Results Author-collected RGB-D Dataset (in SNU Building 301) □ Video available at https://youtu.be/GO0Q0ZiBiSE 22

Evaluation & AR Application Results AR Objects Rendering on ICL-NUIM Dataset □ Video available at https://youtu.be/GO0Q0ZiBiSE 23

Quantitative Analysis on ICL-NUIM Dataset Comparison of the Absolute Translation Error (meter) □ lr-kt0n of-kt1n of-kt2n of-kt3n  L-SLAM presents comparable results compared to other SLAM approaches.  Estimated (magenta) and true (black) trajectories overlap significantly. 24

Thank You for Your Time!  If there are some more questions… Pyojin Kim , Postdoctoral Fellow @ SFU Email: pjinkim1215@gmail.com Website: http://pyojinkim.me/ (Paper, Video, Code, etc.) Affiliation: GrUVi Lab. School of Computing Science Simon Fraser University (SFU), Burnaby, BC, Canada

Reference 1. Zhang, Ji, Michael Kaess, and Sanjiv Singh. "A real-time method for depth enhanced visual odometry." AURO 2017. 2. Kim, Pyojin, Brian Coltin, and H. Jin Kim. "Low-drift visual odometry in structured environments by decoupling rotational and translational motion." IEEE ICRA 2018. 3. Kim, Pyojin, Brian Coltin, and H. Jin Kim. "Indoor RGB-D Compass from a Single Line and Plane." IEEE CVPR 2018. 4. Kim, Pyojin, Brian Coltin, and H. Jin Kim. "Linear RGB-D SLAM for planar environments .“ ECCV 2018. 5. Kaess, Michael, Kai Ni, and Frank Dellaert. "Flow separation for fast and robust stereo odometry." IEEE ICRA 2009. 6. Cvišić , Igor, and Ivan Petrović . "Stereo odometry based on careful feature selection and tracking .“ IEEE ECMR 2015. 7. Bazin, Jean Charles, et al. "Motion estimation by decoupling rotation and translation in catadioptric vision .“ CVIU 2010. 8. Bazin, Jean-Charles, and Marc Pollefeys. "3-line ransac for orthogonal vanishing point detection." IEEE IROS 2012. 9. Zhou, Yi, et al. "Divide and conquer: Efficient density-based tracking of 3D sensors in Manhattan worlds." ACCV 2016. 10. Straub, Julian, et al. "The manhattan frame model — manhattan world inference in the space of surface normals." IEEE TPAMI 2018. 11. Mur-Artal, Raul, and Juan D. Tardós. "ORB-SLAM2: an open-source slam system for monocular, stereo, and RGB-D cameras." IEEE TRO 2017. 12. Kerl, Christian, Jürgen Sturm, and Daniel Cremers. "Robust odometry estimation for RGB-D cameras." IEEE ICRA 2013. 13. Kim, Pyojin, Brian Coltin, and H. Jin Kim. "Visual odometry with drift-free rotation estimation using indoor scene regularities." BMVC 2017. 14. Carlone, Luca, et al. "Initialization techniques for 3D SLAM: a survey on rotation estimation and its use in pose graph optimization .“ IEEE ICRA 2015. 15. Hsiao, Ming, et al. "Keyframe-based dense planar SLAM." IEEE ICRA 2017. 16. Le, Phi-Hung, and Jana Košecka . "Dense piecewise planar RGB-D SLAM for indoor environments." IEEE IROS 2017. 17. Ma, Lingni, et al. "CPA-SLAM: Consistent plane-model alignment for direct RGB-D SLAM .“ IEEE ICRA 2016. 18. Taylor, Camillo J., and Anthony Cowley. "Parsing indoor scenes using rgb-d imagery." RSS 2013.