M ULTI -V EHICLE M AP F USION USING GNU R ADIO O PTIMIZATION AND A CCELERATION O PPORTUNITIES Augusto Vega Akin Sisbot Alper Buyuktosunoglu Arun Paidimarri David Trilla John-David Wellman Pradip Bose IBM T. J. Watson Research Center IBM Research
Acknowledgment § Thanks to the many IBM colleagues who contribute to and support different aspects of this work + our esteemed university collaborators at Harvard, Columbia, and UIUC (Profs. David Brooks, Vijay Janapa Reddi, Gu-Yeon Wei, Luca Carloni, Ken Shepard, Sarita Adve, Vikram Adve, Sasa Misailovic) + many brilliant graduate students and postdocs! § Special thanks to Dr. Thomas Rondeau , Program Manager of the DARPA MTO DSSoC Program This research was developed, in part, with funding from the Defense Advanced Research Projects Agency (DARPA). The views, opinions and/or findings expressed are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government. This document is approved for public release: distribution unlimited. February 2020 2 IBM Research
Outline § Part 1: DARPA-funded EPOCHS project – Domain-specific (heterogeneous) SoC development § Part 2: EPOCHS Reference Application (“ERA”) – Application domain: multi-vehicle cooperative perception § Part 3: 802.11p Transceiver – Optimization and acceleration opportunities February 2020 3 IBM Research
Outline § Part 1: DARPA-funded EPOCHS project – Domain-specific (heterogeneous) SoC development § Part 2: EPOCHS Reference Application (“ERA”) – Application domain: multi-vehicle cooperative perception § Part 3: 802.11p Transceiver – Optimization and acceleration opportunities February 2020 4 IBM Research
DARPA’s Domain-Specific System on Chip (DSSoC) Program Program Manager: Dr. Tom Rondeau § Goal: to develop a heterogeneous system-on-chip (SoC) comprised of many cores that mix general purpose processors, special purpose processors, hardware accelerators, memory, and input/output (I/O) devices to significantly improve performance of applications within a domain * § A domain is larger than any one application – We target the “super” domain of embedded Source: IEEE Spectrum (July 2018) processors for autonomous/connected cars computer vision “c “cooperative perception” software radio * Source: https://www.darpa.mil/program/domain-specific-system-on-chip February 2020 5 IBM Research
Application Domain: Cooperative Perception § Automakers use arrays of sensors to build redundancy into their systems This Image is Why Self-Driving Cars Come Loaded with Many Types of Sensors When’s a pedestrian not a pedestrian? When it’s a decal. Source: MIT Technology Review February 2020 IBM Research 6
Application Domain: Cooperative Perception § Automakers use arrays of sensors to build redundancy into their systems This Image is Why Self-Driving Cars Come Loaded with Many § We propose a complementary approach: Types of Sensors multi-vehicle (cooperative) perception – Cars exchange locally-generated maps When’s a pedestrian not a pedestrian? When it’s a decal. – Each vehicle merges its local map and the received ones in real time predic4ons car-centric False swarm-based Sensing and computation capabilities Source: MIT Technology Review February 2020 IBM Research 7
Efficient Programmability Of Cognitive Heterogeneous Systems “EPOCHS” à our proposed solution for the design challenge presented by the DSSoC program February 2020 8 IBM Research
Efficient Programmability Of Cognitive Heterogeneous Systems “EPOCHS” à our proposed solution for the design challenge presented by the DSSoC program EPOCHS Compiler Ontology & Reference + Design Space Application Scheduler Exploration 10X – 100X reduction in person-years Agile Flow FPGA prototype, FPGA Prototype emulation, optimization, software bring-up Accelerators + Domain-Specific NoC + Memory Implementation SoC Hardware Architecture Agile methodology to quickly design and implement an easily programmed domain-specific SoC for real-time cognitive decision engines in connected vehicles “Super”-Domain: Software-Defined Radio + Computer Vision February 2020 9 IBM Research
Efficient Programmability Of Cognitive Heterogeneous Systems “EPOCHS” à our proposed solution for the design challenge presented by the DSSoC program EPOCHS Compiler Ontology & Reference + Design Space Application Scheduler Exploration 10X – 100X reduction in person-years Agile Flow FPGA prototype, FPGA Prototype emulation, optimization, software bring-up Accelerators + Domain-Specific NoC + Memory Implementation SoC Hardware Architecture Agile methodology to quickly design and implement an easily programmed domain-specific SoC for real-time cognitive decision engines in connected vehicles “Super”-Domain: Software-Defined Radio + Computer Vision February 2020 10 IBM Research
The Big Picture (Where Does This Talk Fit In?) DSSoC’s Full-Stack Integration Decoupled Software Development Environment and Programming development Languages Multi-vehicle map fusion Application using GNU Radio Libraries Operating System Hardware-Software Co-design Heterogeneous architecture Integrated performance analysis composed of Processor Elements: Intelligent scheduling/routing • CPUs Compiler, linker, assembler • Graphics processing units Medium Access Control • Tensor product units • Neuromorphic units • Accelerators (e.g., FFT) • DSPs • Programmable logic • Math accelerators February 2020 11 IBM Research
Outline § Part 1: DARPA-funded EPOCHS project – Domain-specific (heterogeneous) SoC development § Part 2: EPOCHS Reference Application (“ERA”) – Application domain: multi-vehicle cooperative perception § Part 3: 802.11p Transceiver – Optimization and acceleration opportunities February 2020 12 IBM Research
ERA: EPOCHS Reference Application § “Cooperative Perception” for V2V EPOCHS Real-Time Communications Reference connected/autonomous vehicles Map Fusion To other control Communication Application – Multimodal sensing modules Fabric – Local occupancy map Computer generation Map Vision Generation – DSRC-based V2V Sensing Fabric communication – Real-time map fusion Contribute! https://github.com/IBM/era February 2020 13 IBM Research
ERA Main Components (Single Robot’s Viewpoint) § Raw sensor data generated (simulated) using Gazebo in ERA v2 – ERA v3 will replace Gazebo with an automotive simulation platform Gazebo Pose Depth ERA Camera Msg Payload ERA Msg ROS-GR GNU Radio Builder Interface Scan Costmap 2D Occupancy ERA Grid Map Msg Occupancy Grid Map Occupancy Map ERA Msg Grid Map 2D Map Merger Interpreter February 2020 14 IBM Research
ERA Main Components (Single Robot’s Viewpoint) Depth image à laser scans 2D occupancy map generation § Raw sensor data is first conversion label label label converted into laser scans label label label which are used to generate a 2D occupancy grid map Gazebo Pose Depth ERA Camera Msg Payload ERA Msg ROS-GR GNU Radio Builder Interface Scan Costmap 2D Occupancy ERA Grid Map Msg Occupancy Grid Map Occupancy Map ERA Msg Grid Map 2D Map Merger Interpreter February 2020 15 IBM Research
ERA Main Components (Single Robot’s Viewpoint) § Occupancy grid maps are serialized, compressed and put into a GNU Radio PDU Transmitter Receiver § Outbound PDUs are injected into the 802.11p transceiver Gazebo Pose Depth Open-source implementation ERA Camera Msg Payload ERA Msg ROS-GR by Bastian Bloessl GNU Radio Builder Interface Scan https://github.com/bastibl/gr-ieee802-11 Costmap 2D Occupancy ERA Grid Map Msg Occupancy Grid Map Occupancy Map ERA Msg Grid Map 2D Map Merger Interpreter February 2020 16 IBM Research
ERA Main Components (Single Robot’s Viewpoint) § Locally- and remotely-generated occupancy maps are merged in real time to improve the accuracy of the surroundings’ view § In ERAv2, merging is merely adding maps – Executed several times per second (!) Gazebo Pose Depth ERA Camera Msg Payload ERA Msg ROS-GR GNU Radio Builder Interface Scan Costmap 2D Occupancy ERA Grid Map Msg Occupancy Grid Map Occupancy Map ERA Msg Grid Map 2D Map Merger Interpreter February 2020 17 IBM Research
Option 1: Two-Computer Setup § One Gazebo instance simulating one single robot/vehicle in each computer § Over-the-air 802.11p communication (10-MHz OFDM with up to 64-QAM modulation) § More info: https://github.com/IBM/era/wiki/ERA-in-two-computers Gazebo Robot 1 Robot 2 Gazebo Pose Pose 802.11p Depth Depth ERA ERA Camera Camera Msg Msg Payload Payload ERA Msg ROS-GR ROS-GR ERA Msg GNU Radio GNU Radio Builder Interface Interface Builder Scan Scan USRP USRP Costmap Costmap 2D 2D Occupancy Occupancy ERA ERA Grid Map Grid Map Msg Msg Occupancy Occupancy Grid Map Grid Map Occupancy Occupancy Grid Map Grid Map Map ERA Msg ERA Msg Map 2D Map 2D Map Merger Interpreter Interpreter Merger February 2020 18 IBM Research
Recommend
More recommend
