Perception at the edge to heal the HERE HD Live Map Vlad Shestak & Josh Finken Edge Perception March 20, 2019
HERE in numbers HERE Maps on board of 100M 9,000 + vehicles and counting Employees in 56 countries focused on delivering the world’s best map and location technologies collected per day Years of Countries mapped experience TB map data transforming location technology 700,000 3D data points per second per car In-car + HERE cars navigation systems collecting 4 of 5 HD Live Map covering in Europe and North data for 555k+ km America use HERE maps maps for Autonomous Driving 1 GTC 2019
Key building blocks for autonomous driving HD maps are essential in autonomous solutions Sensors HD Maps Perception Stack Decision Making Intelligent sensors to see, Highly detailed, helping Real-time information Deep learning and feel and sense surroundings pinpoint a car’s location and for localization and sensor fusion for decision understand its surroundings path planning making 2 GTC 2019
HERE HD Live Map Data layers providing key benefits Published in NDS and Protobuf formats Layer Data Benefits Localization Road side objects (furniture) HD Localization Model Enhanced sensor functionality Ex: road signs, barriers Localization Lane level features HD Lane Model Route planning Ex: Lane lines, lane widths, lane markings, etc. Rules of the road Enhanced sensor functionality HERE’s Standard Definition (Infotainment) Route planning Road map with ADAS attributes Rules of the road Model Ex: Road topology, direction of travel, elevation, Enhanced sensor functionality slop, etc. 3 GTC 2019
Industry Capture Hardware Novatel GPS Antenna Velodyne HDL-32 LiDAR 80 MP Cameras Novatel Span CPT 7 IMU 4 GTC 2019
Typical Urban Scene 5 GTC 2019
6 GTC 2019
Active Learning Loop Machine Learning Model Learn a Model B illions of images Labeled Unlabeled Library Pool Human Annotator and/or Machine Model Selection Queries 7 GTC 2019
HERE’s Maintenance Strategy to HD Maps • The world isn’t static , it’s constantly shifting and evolving • Mapping systems that support autonomous driving functionalities require rapid map updates to reflect real-world changes Maintenance Localization Quality • Freshness can only be achieved through a constant and broad flow of Crowd- Agnostic Quality Index sensor data sourced • Map maintenance is transitioning from a systematic industrial capture (HERE True) mapping model to a self-healing crowd-source model Industrial OneMap Build Out Capture Alliance 8 GTC 2019
A high definition, continuously updated map requires the crowd HD Live Map Harness crowdsourced sensor data… …for near real-time updates 9 GTC 2019
HERE’s self-healing map process 3 rd Party Sources Many to one HERE True Pipeline Localization Change detection Map Internal HD Publish Points Features Polylines update map (signs, signals, Aggregation (lane markings, road pavement Drive paths edges…) markings…) Observations HD Live Map Mapltes | RWOs Creation of a new Update map & Aggregation in the Sensor collection & feature publish cloud ingestion HD map tiles sensor observations 10 GTC 2019
HERE’s self-healing map process 3 rd Party Sources Many to one HERE True Pipeline Localization Change detection Map Internal HD Publish Points Features Polylines update map (signs, signals, Aggregation (lane markings, road pavement Drive paths edges…) markings…) Observations HD Live Map Maplets | RWOs HD map tiles sensor observations 11 GTC 2019
HERE’s self-healing map process 3 rd Party Sources Many to one HERE True Pipeline Localization Change detection Map Internal HD Publish Points Features Polylines update map (signs, signals, Aggregation (lane markings, road pavement Drive paths edges…) markings…) Observations HD Live Map Maplets | RWO HD map tiles sensor observations 12 GTC 2019
Aggregation 13 GTC 2019
Aggregation 14 GTC 2019
HERE’ self-healing map process 3 rd Party Sources Many to one HERE True Pipeline Localization Change detection Map Internal HD Publish Points Features Polylines update map (signs, signals, Aggregation (lane markings, road pavement Drive paths edges…) markings…) Observations HD Live Map Maplets | RWOs sensor observations HD map tiles 15 GTC 2019
HERE’s self-healing map process 3 rd Party Sources Many to one HERE True Pipeline Localization Change detection Map Internal HD Publish Points Features Polylines update map (signs, signals, Aggregation (lane markings, road pavement Drive paths edges…) markings…) Protobuf NDS Observations HD Live Map Maplets | RWOs sensor observations HD map tiles 16 GTC 2019
Cloud and edge analytics: What to do where, best? 3 rd Party Sources Many to one HERE True Pipeline Localization Change detection Map Internal HD Publish Points Features Polylines update map (signs, signals, Aggregation (lane markings, road pavement Drive paths edges…) markings…) Observations HD Live Map Maplets | RWOs sensor observations HD map tiles 17 GTC 2019
In-Vehicle Processing HERE Maplets Change Detection (optional) HD Live Map Edge Perception Stack Cameras Lidar Radar Ultrasonic … In-Vehicle HERE Backend 18 GTC 2019
Cyclops Off the Shelf, At the Edge, In the Vehicle 19 GTC 2019
Cyclops Off the Shelf, At the Edge, In the Vehicle Standard Configuration • Android smartphone • Imagery at 30 Hz • Android Fused GPS positions at 1 Hz • MEMS readings at 50 Hz • Efficiently packed • Wireless stream • NVIDIA GPU compute 20 GTC 2019
Cyclops: RTK GNSS Optional configuration: positioning via RTK • Lightweight digital antenna • Connects directly to an Android phone via USB • Greatly improved GNSS accuracy • 10Hz position update rate rate 21 GTC 2019
Real-Time Perception Architecture Optional Laptop: visualization Compute: Jetson AGX Xavier Sensor stream: smartphone DNN Model Tracking and Maplet Observation Change or Cloud Runtime Sensor Fusion Reconstruction Generation • Peer-to-peer set of decentralized microservices • Nodes run stand-alone or deployed as a set of services • Wireless: UDP, TCP and HTTP 22 GTC 2019
Cyclops: Maplets Sensoris: Sensor Interface Specification • International, standardized interface for exchange • Within the vehicle • Vehicle to Cloud • Cloud to Cloud • Is at the core of HERE Maplets Maplets: • Structurally combines in-vehicle observations and related accuracy requirements • Sensor-agnostic, low data footprint for instant data transmission • Populate into SENSORIS • Python and C++ implementations available 23 GTC 2019
Cyclops: Demonstration Setup Sensor stream and visualization • Xiaomi Mi 8: sensor stream and wifi network • RTK GNSS positioning • Optional laptop: monitoring and visualization Xavier in-vehicle • Power via USB Type-C (cigarette-lighter adapter) • Headless via an Intel 8265 wifi chip, antennas • Edge Perception software stack 24 GTC 2019
Cyclops: Demonstration 25 GTC 2019
Cyclops: Assessment pole sign 26 GTC 2019
Stronger with our partners 27 GTC 2019
Recommend
More recommend
