Visual Odometry and SLAM using Line Segment Features Ruben - PowerPoint PPT Presentation

Visual Odometry and SLAM using Line Segment Features Ruben Gomez-Ojeda Machine Perception and Intelligent Robotics (MAPIR), University of Malaga http://mapir.isa.uma.es

Ou Outline I. Motivation II. Contributions to Visual Odometry i. Stereo VO ii. Monocular VO III. Contributions to Visual SLAM IV. Future Work

I. I. Mo Motivation

Mo Motivation Despite its high accuracy, point-based VO approaches such as ORB-SLAM2 can lose the tracking in low-textured or bad illuminated scenarios , as the number of features decreases R. Mur-Artal, J.M.M. Montiel, & J.D. Tardos (2015). OR ORB-SL SLAM: a versatile and accurate monocular SL SLAM sy syst stem. IEEE Transactions on Robotics. R. Mur-Artal & J.D. Tardos (2017). OR ORB-SL SLAM2: An An open-so source sl slam system for monocular, stereo, and rg rgb-d d ca camera ras. IEEE Transactions on Robotics.

Mo Motivation R. Mur-Artal, J.M.M. Montiel, & J.D. Tardos (2015). OR ORB-SL SLAM: a versatile and accurate monocular SL SLAM sy syst stem. IEEE Transactions on Robotics. R. Mur-Artal & J.D. Tardos (2017). OR ORB-SL SLAM2: An An open-so source sl slam system for monocular, stereo, and rg rgb-d d ca camera ras. IEEE Transactions on Robotics.

II. II. Con Contribution ons to to Visual Odo Odometry i. . St Stereo VO VO R. Gomez-Ojeda & J. Gonzalez-Jimenez (2016). Ro Robust Stereo Visual Od Odometry th through a Probabilisti tic Combinati tion of Po Points and Line Segments. IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 2016.

St Stereo VO VO Over Ov erview: o Frame to frame approach o Robust performance in most scenarios o Probabilistic SE(3) minimization of reprojection errors OR ORB po poin ints: o Very efficient o Good performance o Only best mutual matches LSD+ LS D+LB LBD D line se segme ments: o High precision and repeatability o Time consuming (modified version) o Only best mutual matches o Stereo projection of the end-points E. Rublee, V. Rabaud, K. Konolige & G. Bradski(2011). OR ORB: An efficient alternative ve to SIFT or SUR URF. IEEE Conference on o Deal with partial occlusions Computer Vision (ICCV), Barcelona, Spain, 2011. R. G. Von Gioi, J. Jakubowicz, J.M. Morel & G. Randall (2010). LS LSD: A fast st line se segment detector with a false se detection control. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010. co L. Zhang & R. Koch (2013). An An efficient and robust line segment matching approach based on LBD descriptor and pairwise ge geometric cons nsistenc ncy. Journal of Visual Communication and Image Representation, 2013.

St Stereo VO VO

St Stereo VO VO ORB points LSD line segments

St Stereo VO VO: : KI KITTI seq equen ences es

St Stereo VO VO KITTI-00 EuRoC/V1_01_easy 5x 2x Source code available: ht https://github.com/rubengooj/StVO-PL PL

II. II. Con Contribution ons to to Visual Odo Odometry ii. ii. Monocula lar VO R. Gomez-Ojeda, J. Briales, & J. Gonzalez-Jimenez. PL PL-SV SVO: : Semi-Di Direct Monocular Visual Od Odometry by Combining Points and Li Line ne Se Segments. Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, Daejeon, Korea, 2016

Mo MonoVO Pros Cons Fast to detect and match Presence of outliers Point-based Usually abundant Reduction in structured scenarios Good behavior in most scenarios Slow detection and matching Line-based Less outliers (more distinctive Tracking of endpoints and occlusions feat.) Points along the line usually are not key- SVO approach allow for the fast points, so the feature alignment step is less tracking of line segments reliable Our approach Robust performance in both kind Depth estimation of endpoints might be of scenarios affected by occlusions R. Gomez-Ojeda, J. Briales, & J. Gonzalez-Jimenez. PL PL-SV SVO: : Semi-Di Direct t Monocular Vi Visual Odometr try by Combining P Co Points a and L Line Se Segments. Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, Daejeon, Korea, 2016

Mo MonoVO: : Sp Sparse Model-ba based d Image Alignm nment Seek: rigid body transformation between frames using current model • Direct approach, min photometric error • 4x4 patches around points are used • Coarse-to-fine scheme Segment regions are not small: Figure adapted from [SVO] • Warping • Point sampling SV SVO [1]: PL-SV PL SVO: C. Forster, M. Pizzoli & D. Scaramuzza (2014). SV SVO: Fast semi-di direct mono nocul ular ar od odome ometry. . IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 2014.

Mo MonoVO: : In Individual Feature Alignment Refine feature correspondences Seek: new feature positions • Min photometric error wrt corresponding path in closest KF • 8x8 patch with affine warping Segment features: Figure adapted from [ SVO ] • Use endpoints only (simplicity) • We deal with outliers in the final refinement step. Feature re Fe refinement (S (SVO O [1], PL PL-SV SVO): C. Forster, M. Pizzoli & D. Scaramuzza (2014). SV SVO: Fast semi-di direct mono nocul ular ar od odome ometry. . IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 2014.

MonoVO: Mo : Po Pose and Structure Refinement Fast bundle adjustment Seek: poses and map • Min reprojection error • Reduces drift Robustified framework: • Heuristics may introduce outliers • Cauchy loss function & outlier filter Figure adapted from [ SVO ] Mo Motion-on only BA BA: Re Reprojection er errors: C. Forster, M. Pizzoli & D. Scaramuzza (2014). SV SVO: Fast semi-di direct mono nocul ular ar od odome ometry. . IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, China, 2014.

Mo MonoVO EuRoC/MH_05_difficult Low-tectured scene 1x 1x PL PL-SV SVO: Source code available: • More robust performance ht https://gi github. thub.com/rube /rubeng ngooj/pl j/pl-sv svo • Still real-time: 60 Hz • Fast tracking and mapping of line segments

III. III. Con Contribution ons to to Visual SLAM R. Gomez-Ojeda, F.A. Moreno, D. Scaramuzza & J. Gonzalez-Jimenez (2017). PL PL-SL SLAM: : a Stereo SLAM System through the Co Combination of Points and Line Se Segments. ArXiv preprint arXiv:1705.09479, 2017.

St Stereo Vi Visua ual SLAM LB LBA:

St Stereo Vi Visua ual SLAM LB LBA:

St Stereo Vi Visua ual SLAM LC LC:

St Stereo SLA SLAM Mapping example in EuRoC/V1_01_easy Low-textured scene 5x 0.5x Source code available: https://gi ht github. thub.com/rube /rubeng ngooj/pl j/pl-sl slam am

IV IV. Fut Futur ure Wo Work R. Gomez-Ojeda & J. Gonzalez-Jimenez (2017). Fa Fast Li Line ne Segm gment nt Mat atchi hing ng for Stereo Visual sual Od Odometry. . Submitted to ICRA 2018.

Pe Performance Co Compariso ison be between PL PLVO and and F-PL PLVO Da Dataset: : EuRoC Se Sequence ce: V1_01_easy Mo Motion ty type: MAV Te Texture ty type: indoor, well-structured Il Illumination: : well-illuminated

PLVO F-PLVO 5x 5x

Pe Performance Co Compariso ison be between PL PLVO and and F-PL PLVO Da Dataset: : EuRoC Se Sequence ce: V1_02_medium Mo Motion ty type: MAV (fast motion) Te Texture ty type: indoor, low-textured (partially) Il Illumination: : well-illuminated

PLVO F-PLVO 5x 5x

Pe Performance Co Compariso ison be between PL PLVO and and F-PL PLVO Da Dataset: : Tsukuba Se Sequence ce: Lamps Mo Motion ty type: Synthetic Te Texture ty type: indoor, textured Il Illumination: : low-illuminated

PLVO F-PLVO 3x 3x

Pe Performance Co Compariso ison be between PL PLVO and and F-PL PLVO Da Dataset: : EuRoC Se Sequence ce: MH_05_difficult Mo Motion ty type: MAV Te Texture ty type: indoor, well-structured Il Illumination: : changes of illumination

PLVO F-PLVO 5x 5x

Visual Odometry and SLAM using Line Segment Features Ruben Gomez-Ojeda Machine Perception and Intelligent Robotics (MAPIR), University of Malaga http://mapir.isa.uma.es