Detection and Segmentation of Road Images with Deep Learning Frank Geujen – Senior Product Manager William Raveane – Computer Vision Engineer Mapscape, a Navinfo company GTC Europe, October 2017, Talk #23304
Who is NavInfo 目录 SD & HD Map Making CONTENTS Traffic Sign Detection Road Feature Extraction Looking Ahead
Who is NavInfo
NavInfo Introduction NavInfo is the leading map provider in China, with focus on location big data platform, HD / SD map, Telematics and ADAS comprehensive solutions. • Established on 2002 in Beijing China • More than 4500 employees Globally Automated Driving Navigation Connected Car
Branches Ameri merica ca Nethe Ne therlan rlands ds China hina 31 31 loca locali liza zati tion ba base e for r da data ta colle collect ct and d te tech chnolo ology Mapscape Mapscape: : Com Compila pilatio tion Tech echnol olog ogy (ND (NDS) In Inte ternati tion onal l bu busines iness ser ervice. ice. EU EU Tech echnolo ology Cent Centre re: exp expansion ion 4 4 R&D &D Center Centers • Com Compute ter r Vis ision ion Adv Advanced ced te tech chnol ology ogy （ Sh Shangh ghai 、 Xia Xian 、 Sh Shenyang 、 Wuhan ） • Deep Deep Le Learni rning res esea earch rch Beij Beijing ing Hea eadqu quarte rters rs
SD & HD Map Making
Challenge of Map Making Ingestion/ Ingestion/ Map Cre Map Creat ation ion Del Deliv iver ery y Data Sour Data Source Extra xtraction tion 600+ 600+ 3.2+ 3. 2+ Mill Millio ion 6.16+ Mil 6.16+ Milli lion n > 500 500 dispersed field Signs processed Kilometer staff in China production staff per year 24.9 .95 + Million 31 31 Core POI field local offices 10 100+ 0+ 20+ Mil 20 + Milli lion 4,0 ,000 00+ Hong Kon Kong, , Lao Laos, s, Ma Maca cao, Cam , Cambodia ia page Collection POI updated specifications Map data vehicles per year 360 360+ + cities cities 60+ 60+ Big data mining 80%/7 80 %/70% 0% Cities of ground 4+ 4+ Mi Mill llion ion 99 99% Highway truth testing in Update Road distance 80 80% Main Road 2017 Q1 80% POI and updated 70% road link 22 22M+ M+ per year 260+ 260+ in China per community Attributes in year contributions the portfolio
SD Map St Standa andard Def rd Defin init ition ion Map, Map, i is s pr prim imar arily fo ly for A r A to B to B routing outing & guidan & guidance and and is is a sim a simpl plif ified represen ied representa tation tion of of th the road in e road in lin links s and and nodes nodes. POI OI
HD Map Hi High gh Defini Definition ion M Map, is ap, is us used for au ed for automated/ tomated/autono autonomous ous driv drivin ing and in and includes ludes high high accurat accurate lane e lane and and ro road ad fea features tures.
The Field drive use cases On-board SD map ( Semi ) Live feed Real Time automated mono sign core map camera extraction updating Off-board SD map 2 FPS ( Semi ) stored Off line mono automated Feature core map camera extraction imagery updating Off-board HD map Feature Road Projection Panoramic HD map extraction feature on LIDAR imagery creation/ from LIDAR extraction data updating
Technical Deep Dive: Traffic Sign Detection
Real Time Traffic Sign Detection Use case: In-car data collection on an NVIDIA Jetson TX2 • Over 180 traffic sign classes supported today • Up to 32 fps at 1920x1080 in 15W MAX-N mode • Detection based on Single Shot Detector (SSD) • Training on a Titan X GPU server • Inference through TensorRT
Supported Features Supported today • Speed Limits • Warning Signs • Information Signs • Prohibition Signs • Directional Signs In development • Gantry Sign Boards • Traffic Lights • Digital Traffic Signs
Sign Detection Demo Watc tch O Onli line: e: htt ttps ps://go ://goo.gl/ o.gl/KBgG KBgGo8 o8
Performance Highlights Multip ltiple le Si Simu mult ltaneou aneous s Detect Detections ons
Performance Highlights Dist istant ant Tr Traffic ic Si Sign gns
Performance Highlights Bad ad Li Light ghting ing Cond Conditio itions
Optimization of SSD on Jetson TX2 • With TensorRT: • 6x speedup in inference performance • 3x reduction in memory consumption • And with our in-house CUDA Kernels • Additional 3x speedup in inference performance • Allows full utilization of GPU resources
Implementation of SSD Two-stage system: • ResNet-based SSD for Detection • ResNet for Fine Classification Custom Layer API: • Bridges both TensorRT Stages Detector: Classifier: SSD ResNet
SSD Custom Layers • Implementation of SSD layers as custom CUDA kernels: • Executed by Custom Layer API • Priors replaced by on-demand calculations • Softmax calculated only when required • Non-maximum suppression replaced by a batched data feeder for the classifier
SSD on the Jetson TX2 Profile visualization of SSD inference 510ms 31ms SSD Caffe TensorRT + our CUDA kernels
Detection Accuracy Single Image Detection PR Curve • Single Image: • Precision: 92.5% • Recall: 98% • Tracking over Time: • Precision: 96.0% • Recall: 98.5% Single Image Per-Class Detection Accuracy 100 Accuracy 0 Class ID Single Image Per-Class Classification Accuracy 100 Accuracy 0 Class ID
Technical Deep Dive: Road Feature Extraction
Road Feature Extraction • Road feature and object extraction Road features • Semantic segmentation network architecture • Automatic lane grouping • Training & inference on NVIDIA Titan X GPU server Lane numbering Gantry sign boards
Road Features Supported today: • Surface level: • Lane markings • Text, numbers, speed limits • Arrows • Road objects: • Gantry sign boards • Guard Rails • Curbs In Development: • Poles • Traffic Lights • Tunnels
The On-road feature extraction process Camera Segmentation Calibration Transformation Network Crop from Panoramic Image Semantic Segmentation to Top View Deep Neural Network Transformation to Front View Lane Number Grouping
Lane Segmentation Demo Watc tch O Onli line: e: htt ttps ps://go ://goo.gl/ o.gl/4CXT 4CXTD5 D5
Semantic Segmentation Performance • Inference at 5 images per second using an NVIDIA Titan X GPU • Common lane marking classes • Recall: 92.8% • Precision: 82% • Common road arrow marking classes • Recall: 85.6% • Precision: 72.8% Performance of the system expected to further improve as we continue development Confusion Matrix
Looking Ahead
Looking Ahead • Deep Learning continues being more integrated into our: • Field collection • Map creation • Distribution processes • On-going developments: • Real-time semantic segmentation system on-board vehicles • Crowdsource data processing supporting self-healing maps • Applications for crowdsourcing
Detection and Segmentation of Road Images with Deep Learning Frank Geujen – Senior Product Manager William Raveane – Computer Vision Engineer Mapscape, a Navinfo company GTC Europe, October 2017 , Talk #23304
Recommend
More recommend
