StructSLAM: Visual SLAM with Building Structure Lines Danping Zou Assistant Professor Key Laboratory of Navigation and Location-based Services Shanghai Jiao Tong University dpzou@sjtu.edu.cn http://drone.sjtu.edu.cn/dpzou
Shanghai Jiao Tong University What’s SLAM? • SLAM : S imultaneous L ocalization A nd M apping. Constructing a map of an unknown environment while simultaneously keeping track of the robot’s location. • Autonomous navigation • motion planning 2016/3/30 - 2 -
Active areas for SLAM research Shanghai Jiao Tong University Robotics Augmented Reality (AR) • UAVs ， Service robots 2016/3/30 - 3 -
SLAM with different kinds of sensors Shanghai Jiao Tong University • Active sensing • Heavy and bulky • Active sensing • Energy consuming RGB-D camera 3D liDar 2D laser rangefinder Since 2005, there has been intense research into VSLAM (Visual SLAM) using primarily visual (camera) sensors. • Passive sensing • Light & compact • Energy saving • Ubiquity 2016/3/30 - 4 -
List of Visual SLAM methods Shanghai Jiao Tong University MonoSLAM Extended Kalman filter FastSLAM GraphSLAM Sparse feature points PTAM ORB-SLAM Structure-from-motion CoSLAM* DTAM Dense / Semi dense LSD-SLAM *Zou, Danping, and Ping Tan. "CoSLAM: Collaborative visual slam in dynamic environments." Pattern Analysis and Machine Intelligence, IEEE Transactions on 35.2 (2013): 354-366. 2016/3/30 - 5 -
Knowing issues for Visual SLAM Shanghai Jiao Tong University Scale ambiguity Lack of Illumination GNSS （ GPS 、 Beidou ） Accumulated error (Drift error) Inertial Units （ Gyro+Acc+Compass ） Lack of Texture Vision + Ｘ Wireless （ Wifi ， Bluetooth ） Map / Floor plan 2016/3/30 - 6 -
Shanghai Jiao Tong University What about if we consider one special case : indoor scenes Man-made scenes Natural scenes 2016/3/30 - 7 -
StructSLAM Shanghai Jiao Tong University A novel visual SLAM method aided with structure lines • Structure lines : The line feature aligned with dominant orientations Zhou, Huizhong, Danping, Zou, et al. "StructSLAM: Visual SLAM with building structure lines." Vehicular Technology, IEEE Transactions on 64.4 (2015): 1364-1375. - Special session for indoor localization 2016/3/30 - 8 -
Indoor scenes Shanghai Jiao Tong University Motivations: • Lines are better landmarks in texture-less scenes (like many indoor scenes with only white walls) than points. • Some lines (structure lines) encode the global orientation information. Less accumulated error 2016/3/30 - 9 -
Shanghai Jiao Tong University 2016/3/30 http://drone.sjtu.edu.cn/dpzou/project/structslam.php - 10 -
Pipeline of StructSLAM Shanghai Jiao Tong University Extended Kalman filter: Initialization • Simple to be implemented • Easily fused with other State prediction sensors (IMU, odometers) • More robust than structure- from-motion pipeline. Data Association EKF pipeline State update Initialize new features Feature management Remove old features 2016/3/30 - 11 -
Dominant directions Shanghai Jiao Tong University The man-made world are generally dominated by several directions. • Vertical direction (always points to the sky) • Horizontal directions (are usually perpendicular to each other, although not always) 2016/3/30 - 12 -
Parameter planes Shanghai Jiao Tong University Parameter plane is one of xy,yz,xz Structure line planes of the world frame. A structure line is represented by a point on the parameter plane. z The parameter plane is selected so y as to make sure it is the most x perpendicular to the dominant direction. Parameter plane 2016/3/30 - 13 -
Structure line Shanghai Jiao Tong University Each structure line is represented by a point on the parameter plane, denoted by a 4x1 vector. It is in fact a 2D inverse depth representation* Projection of camera center Direction Inverse depth * Montiel, J. M. M., Javier Civera, and Andrew J. Davison. "Unified inverse depth parametrization for monocular SLAM." analysis 9 (2006): 1. 2016/3/30 - 14 -
StructSLAM – State representation Shanghai Jiao Tong University State vector and covariance matrix (MonoSLAM) Camera pose + Points + Structure Lines 2016/3/30 - 15 -
Initialization Shanghai Jiao Tong University Step 1: Initialization • Use LSD line detector* to detect line segments on the image. State prediction Data Association State update Initialize new features Remove old features * Von Gioi, Rafael Grompone, et al. "LSD: A fast line segment detector with a false detection control." IEEE Transactions on Pattern Analysis & Machine Intelligence 4 (2008): 722-732. 2016/3/30 - 16 -
Initialization Shanghai Jiao Tong University Step 2: Initialization • Apply J-linkage* to classify parallel line segments into groups and detect vanishing points State prediction Data Association State update Initialize new features Remove old features *Toldo, Roberto, and Andrea Fusiello. "Robust multiple structures estimation with j-linkage." Computer Vision – ECCV 2008 . Springer Berlin Heidelberg, 2008. 537-547. 2016/3/30 - 17 -
Initialization Shanghai Jiao Tong University Step 3: Initialization • Estimate the dominant direction from the vanishing points. State prediction - Dominant direction Data Association - Vanishing point State update : Camera intrinsic Initialize new : Rotation from the world features frame to the camera frame Remove old features 2016/3/30 - 18 -
Initialization Shanghai Jiao Tong University Step 4: Initialization • Refine the dominant directions by non-linear least square optimization • Initialize new lines (See the feature management section) State prediction Data Association State update Initialize new features Remove old features 2016/3/30 - 19 -
State prediction Shanghai Jiao Tong University Camera – constant velocity or odometer data (if Initialization available) State prediction Data Association State update Odometer velocity Initialize new features Landmarks – static Remove old features 2016/3/30 - 20 -
State prediction Shanghai Jiao Tong University Covariance propagation Initialization State prediction Data Association State update Initialize new features Remove old features 2016/3/30 - 21 -
Data association Shanghai Jiao Tong University Find the line segments corresponding to the structure line Structure line 2016/3/30 - 22 -
Data association Shanghai Jiao Tong University Find the line segments corresponding to the structure line Structure line • One-to-multiple matching • There could be false matchings (outliers). 2016/3/30 - 23 -
Data association Shanghai Jiao Tong University Step1 : Get candidate matching by : residual vector :Jacobian of observation function 2016/3/30 - 24 -
Data association Shanghai Jiao Tong University Step 2: Comparing appearance by ZNCC (zero mean normalized cross-correlation) ZNCC > ０ .8 Current frame Previous frame 2016/3/30 - 25 -
Data association Shanghai Jiao Tong University Step3 : One-feature RANSAC to eliminate false matchings (outliers): • Randomly sample a candidate matching • Run a tentative EKF update using the sampled matching and check the number of inliers • Keep the inlier set with the maximum number • Repeat the above steps • Use the inlier set to run EKF update. 2016/3/30 - 26 -
State update Shanghai Jiao Tong University Observation model: Initialization Structure line State prediction Data Association State update Initialize new features Remove old features 2016/3/30 - 27 -
State update Shanghai Jiao Tong University Project a structure line onto the image Dominant direction Vanishing point World frame Camera frame Image Structure line Connect with vanishing point Projection 2016/3/30 - 28 -
State update Shanghai Jiao Tong University Observation model • Observation function (measurement function): • Since the desired distance is zero, the residual is computed as : Small residual Large residual 2016/3/30 - 29 -
State update Shanghai Jiao Tong University Standard Extended Kalman Filter: :Jacobian of observation function Predicted state and covariance: : Observation noise - Uncertainty of detected line segments 2016/3/30 - 30 -
State update Shanghai Jiao Tong University Multiple-pass EKF update Initialization (Point + Structure lines) State prediction Data Association State update using point observation State update State update using structure line observation Initialize new features Remove old features 2016/3/30 - 31 -
Feature management Shanghai Jiao Tong University Initialize new structure lines Initialization :Dominant direction State prediction Point in world frame: Data Association Line though the point: State update Intersection with parameter plane: Initialize new features Expressed in parameter plane (xy-plane): Remove old features 2016/3/30 - 32 -
Feature management Shanghai Jiao Tong University Initialize new structure lines Initialization State prediction Data Association State update Initialize new features Remove old features 2016/3/30 - 33 -
Recommend
More recommend
