Vehicle Localization based on Lane Marking Detection Yuncong Chen - PowerPoint PPT Presentation

Vehicle Localization based on Lane Marking Detection Yuncong Chen UCSD HRI intern 2014

Overview Input Goal ● Odometry ● lateral localization on highway (noisy GPS / IMU for now) ● give correct estimate on merge / split points ● Monocular camera Assumptions ● road surface is flat ● Lane level map

Coordinate System local plane origin = first gps position GPS Map Algorithm Longitude / Latitude California State Plane North / East / Down

Map no semantic information, interpolate

Particle Filter pose = (north, east, yaw) ● Motion model previous pose GPS / odometry current pose ● Observation model map = a set of points labeled by marking groups current image map

Particle Filter all with same weight here resample particles weight each particle by according to their weights its likelihood computed propagate using from observation model motion model

Motion Model final rotation noise translation noise ● rotations and translation computed from odometry initial rotation noise

Observation Model project map points to bird’s-eye view Given the vehicle pose, our bird’- eye view image is expected to look like this ...

Observation Model project map points to bird’s-eye view Given the vehicle pose, our bird’- eye view image is expected to look like this ...

Observation Model … while what we really observe is ... inverse perspective filter Hough line fitting transform

Observation Model match lines expected observed

Maximum Bipartite Matching expected observed

… not so simple matches cannot be order must be consistent some map lines may too far away not be detected in the image # candidate matchings

Likelihood Score prob. of not detected line detecting a matched to map line map line i map line i distance between the line pair total number of map lines

Speed Up Matching share among particles ● Sample to obtain a very small set of candidate matchings (map, detected) (0,0)(1,1)(2,2)(3,3)(4,4) (0,0)(1,1)(2,2)(3,3)(4,4) (0,0)(2,1)(3,2)(4,3) (0,0)(1,1)(2,2)(3,3)(4,4) (0,1)(2,3)(3,4) ● For the rest of the particles, only evaluate these candidate matchings (0,1)(2,3)(3,4) (0,0)(2,1)(3,2)(4,3) ● Exploit spatial correlation of matchings among nearby particles ● Preferable to sample particles spread out in space. (0,0)(1,1)(2,2)(3,3)(4,4)

Speed Up Matching search in previous map lines’ extent ● Keep track of extent of every map line ● For a new set of detected lines, search matchings for each map line only within its extent map group 16 map group 3 map group 7 map : ipm map group 8 g16: 0 g3: 1 map group 11 g7: 2, 3 g8: 2, 3 g11: 3, 4 ● Exploit temporal invariance of matchings for a single particle at different times

Process Images inverse perspective filter Hough line fitting transform

Inverse Perspective Mapping pitch height yaw measured by hand

Inverse Perspective Mapping

Inverse Perspective Mapping

Inverse Perspective Mapping

Top-hat Filter high response if one side of an edge is very dark

Top-hat Filter * threshold & threshold high response if one * side of an edge is very dark more robust for detecting dark-bright-dark patterns

Steerable Filter Second derivative of Gaussian separable rotated to arbitrary angle

Map-guided Filtering SC logic OR ...

Steerable vs. Top-hat noisy image map-guided steerable top-hat

Take Advantage of Map Observation model ● map-guided image filtering ● map-guided line fitting Motion model ● more likely to go along the current lane ● cannot move beyond road edges

Hough Transform Line Fitting 25 line segments detected by 6 lines remains after merging OpenCV’s probabilistic Hough transform

Experiments on straight lanes straight avg lateral error: 0.22, max: 1.35

Straight lanes

Deal with Curved Lanes curved avg lateral error: 0.25, max: 0.98

Deal with Curved Lanes ● detect whether the line is a curve (i.e. residual of a linear regression is large) ● if so, match only the bottom segment

Steerable vs. Top-hat straight steerable, avg lateral error 0.23, max 1.15 top-hat, avg lateral error 0.2, max .86

Steerable vs. Top-hat curve steerable, avg lateral error 0.3, max 0.79 top-hat, avg lateral error 0.47, max 1.8 avg lon. error 0.7, max 1.55 avg lon. error 0.67 max 2.83

Effect of the Number of Particles

Issues and Extensions ● shadows ● more general markings (urban environment) ○ stop-lines (longitudinal correction) ○ curved lanes ○ model-free ● investigate how number of particle affects performance