Cars at GM Efrat Rosenman, Ph.D. Head of Cognitive Driving Group - PowerPoint PPT Presentation

Leveraging AI for Self-Driving Cars at GM Efrat Rosenman, Ph.D. Head of Cognitive Driving Group General Motors – Advanced Technical Center, Israel

Agenda • The vision • From ADAS (Advance Driving Assistance Systems) to AV (Autonomous Vehicles) • AI for Self-Driving cars • ADAS, AV and in-between • Summary 10/19/2017 General Motors 2

The Vision • Mobility – one of the most significant revolutions of modern times • Self-driving cars will take mobility to a completely new phase… ”Zero Crashes, Zero Emissions, Zero Congestion” (Mary Barra, GM CEO) ? 10/19/2017 General Motors 3

The Vision Increase Safety Increase Productivity Increase Car Sharing & Reduce Road Capacity and Parking needs Increase Mobility: anywhere, anytime 10/19/2017 General Motors 4

From ADAS to AV L5: Full automation Anywhere, anytime Level 4: High Fully autonomous specific scenarios automation Level 3: Conditional Highway driving (driver takes automation control with notice) Level 2: Partial automation Traffic jam assist Level 1: Driver assistance Cruise control, lane position Level 0: Driver in full control Info, warnings 10/19/2017 General Motors 5

From ADAS to AV • Will incremental steps get us to the top of this pyramid? 10/19/2017 General Motors 6

Components of self driving cars Decision Sensing Mapping Perception Control Making 10/19/2017 General Motors 7

Components of self driving cars AI AGENT serves as the “brain” of the car Decision Perception Control Making 10/19/2017 General Motors 8

AI for Self-Driving Cars 9

AI in Perception • Unsupervised learning • Finding structure in point clouds • Feature learning • Supervised learning • Object detection • 2D object recognition (Classification) • 3D scene understanding and modeling (3D objects pose) • Semantic segmentation (boundaries of objects, free space) 10/19/2017 General Motors 11

AI in Perception - E2E trend • Classification: Labels Key Points SIFT features Model Pixels • Scene understanding: Scene Segmentation Object Contextual Pixels description detection relations • Perception: 3D World state Sensors 2D object Depth Pose estimation detection estimation 10/19/2017 General Motors 12

AI in Perception - E2E trend • Classification: DNN Labels Key Points SIFT features Model Pixels • Scene understanding: DNN Scene Segmentation Object Contextual Pixels description detection relations • Perception: 3D World state Sensors 2D object Depth Pose estimation DNN detection estimation 10/19/2017 General Motors 13

Towards E2E: Sensors Fusion Low Level: raw data High Level: tailored hierarchy combined in input stage between sensors • All sensors • Utilizes domain contribute knowledge • Enables learning • Model is of complex explainable dependencies “optimally” • Based on tailored • Sparse Vs. dense rules sensors • Suboptimal • Larger models, performance harder to learn 10/19/2017 General Motors 14

Towards E2E: Multi-Task Learning • Most our outputs are inter related • Objects, free space, lanes, etc. • Cross regularization allows reaching a better local minima • TPT • Major parts of the Deep Net are used for multiple tasks • Data Efficiency Mask R-CNN Facebook AI Research (FAIR); Apr 2017 10/19/2017 General Motors 15

What about data? 16

Automatic Data Annotation • Data is the key contributor to perception accuracy – With no visible saturation • How can we create annotated data • Manual annotation – Expensive and inaccurate • Automatically Revisiting Unreasonable Effectiveness of Data in Deep Learning Era, Google 2017 10/19/2017 General Motors 17

Automatic Data Annotation • Technology • High end sensors (Lidar, IMU, etc.) • High accuracy detectors (on behalf of computation time) 10/19/2017 General Motors 18

Example – AGT for StixelNet • StixelNet - Monocular obstacle detection • Based on stixel representation • Identify road free space [Badino, Franke, Pfeiffer 2009] • Ground truthing is based on Lidar Compact, local representation Lidar (Velodyne HDL32) is used to identify Dan Levi, Noa Garnett, Ethan Fetaya. StixelNet : A Deep Convolutional Network for Obstacle Detection and Road Segmentation. In BMVC 2015 . obstacle on each stixel in the image 10/19/2017 General Motors 19

Is Perception “solved”? • Challenge of Cost • Sensors • Mapping • Computation • Challenge of false positive & false negative • Data uncertainty (noise) • Model uncertainty (confidence) Label: Cyclist RGB: Pedestrian (0.56) 10/19/2017 General Motors 20

Decision Making Decision Perception Control Making 10/19/2017 General Motors 21

Learning Decision Making Decision Making cannot learn from static examples Need interactive domain - > Reinforcement Learning (RL) RL has seen some major successes in the recent years: Atari Autonomous Helicopter Poker Go [Google Deepmind] source: nbcnews Flight [Google deepmind] source: uk business [Bowling et al] source: wikipedia insider [Ng et al] source: ai.stanford.edu 10/19/2017 General Motors 22

RL challenges in Self-Driving agents • Learn to act in a very high dimensional space • Plan sequences of driving actions • Predict long term behaviors of other road users • Few sec • Complicated situations • Negotiate with other road user • Guarantee safety 10/19/2017 General Motors 23

Simulation • Advanced simulations are required • Multi-agent • Various conditions • Focus on “interesting miles” • Drive billions of “virtual miles” (fuzzing) Waymo simulation: https://www.engadget.com/2017/09/11/waymo-self- driving-car-simulator-intersection/ “Any system that works for self driving cars will be a combination of more than 99 percent simulation.. plus some on- road testing.” [ Huei Peng director of Mcity, the University of Michigan’s autonomous - and connected- vehicle lab] 10/19/2017 General Motors 24

Safety Guarantees - From ADAS to AV Will incremental steps get us to the top of this pyramid? The technological heart is different in kind 10/19/2017

What’s the difference? • For ADAS – Safety guarantee is based on the driver • For autonomous – Safety guarantee should come from the system itself 10/19/2017 General Motors 26

Example: Highway Driving in Super Cruise™ The 2018 Cadillac CT6 will feature Super Cruise™ - a hands-free driving technology for the highway It includes an Exclusive driver attention system to support safe operation 10/19/2017 General Motors 27

Safe Driving for level 4/5 • System should handle 100% of the cases • Redundancy requires at all levels • Sensing • Algorithm • Computing • Control • Fallback strategies • Guarantee of Safety is a must to the acceptance of AV • Statistical data-driven approach [miles-per-interrupts] requires driving billions of miles to validate an agent • Should be repeated with every SW version • Need safety constrains (rule-based/model-based) 10/19/2017 General Motors 28

Summary • Advances in AI are key to success of self- driving cars • AI-based features can bring ADAS to a new level in terms of accidence avoidance, productivity gain and saving in human lives • Level 4/5 AV should be a parallel effort focus on redundancy and safety constrains 29 10/19/2017 General Motors

GM Advanced Technical Center in Israel (ATCI)

Thank you 10/19/2017 General Motors 31