Ma Machine chine Lear arning ning for r Auton tonomous mous Dr Driving ving Nasser r Mohamm mmadi adiha ha Senior An Analysi sis s Engine neer er at Volvo Cars/ s/Ze Zenu nuity ty & & Adjun unct ct Docent nt at Chalmer almers Sto Stock ckholm holm (K (KTH TH), ), 2017-04 04-03 03
Zenuity! • Develop software for autonomous driving and driver assistance systems • Autoliv and Volvo Cars will own Zenuity 50/50 • Starting with 200 employees from Autoliv and Volvo Cars • Volvo Cars and Autoliv will license and transfer the intellectual property for their ADAS systems to Zenuity • Headquartered in Gothenburg with additional operations in Munich, Germany, and Detroit, USA CEO: Dennis Nobelius
AD and AI among hottest trends Top 10 trends from Nyteknik Förarlöst i Gartner's 2016 Hype Cycle for Emerging Technologies praktiken Ai i allt
Outline Autonomous Driving (AD) ML in the era of AD • Interest in ML in the IV/ITS community • Applications • Some examples
Drive Me: Self driving cars for sustainable mobility
Global Trends • Urbanisation • Growing mega cities • Air quality major health problem • Traffic accident global health issue • Time for commuting • Desire for time efficiency • Desire for constant connectivity
Today’s reality
SHAPING THE FUTURE MOBILITY AD will be important for a sustainable mobility • Improved traffic safety • Improved environmental outcomes • Regain time
Volvo vision 2020 No one should be killed or seriously injured in a new Volvo car by 2020
It was always about freedom
And it’s still about freedom
joint effort and proper reseach Global challenges – Demand a joint effort Drive Me – Nordic model of collaboration Research platform – How autonomous cars can contribute to a sustainable development Pilots with real customers in real traffic
The pilots – all about learning Gothenburg 2013 • Traffic environments London • Customer preferences China 2017 2017 • Exporting the Nordic model of collaboration Gothenburg – proof of concept China and London – verify our technology
well-defined commute highways • Customer focus • Simplification • Risk Management A B N e i g h b o r h o o d F r u s t r a t i n g c o m m u t e C l o s e t o w o r k Your neighborhood, children, Well defined use case on city highways Traffic lights, pedestrians, bicyclists, ... no lane markings, roundabouts , ...
Pilot Assist versus Autopilot Pilot assist / Supervised Autopilot / UnSupervised • • Driver is responsible, should monitor and supervise Manufacturer responsible • • Driver responsible to intervene whenever needed Tested on and expects extreme situations • • Limitations: Lane markings, road design, oncoming objects, Takes precautions, takes decisions pedestrians, animals, restrictions in steering/braking/acceleration • Driver free to do something else force that can be applied
SAFETY Impact Safety Benefits Precautionary Safety Crash Avoidance Injury Reduction Risky driving Conflict/ Low-risk Crash After crash behaivours Net-Crash driving & situations Unsupervised AD (Level 4) AD Car Supervised AD (Level 1&2) Manual (Level 0)
We assume liability “Volvo will assume liability for its autonomous technology, when used properly. ”
TECHNOLOGY • Camera • Radar • Laser • Ultrasonic • Map data • Cloud connection • Traffic Control Centre “Machine learning for sensor signal processing is in the core!”
redundancy Perception Decision Action Brake Vision Brake Control 1 Brake Vehicle Sensor Decision & Radar Dynamics Brake Fusion 1 Control 1 Management 1 Control 2 Brake Lidar Vehicle Sensor Decision & Steering Brake Dynamics Fusion 2 Control 2 Control 1 Management 1 Ultrasonic Steering Control 2 Power steering Cloud Power steering
Interest in ML, related to AD and ITS 100 90 80 70 60 50 40 30 20 10 0 Records in Google Scholar ITSC 2016 (out of 430 papers)
Improving the Results over THE Years • Object detection rate in KITTI • Moderate: Min. bounding box height: 25 Px, Max. occlusion level: Partly occluded, Max. truncation: 30 %
Grouping the ML Methods Training Deployment Where is it trained? Where is it executed? • • Locally On-board • • In the cloud Off-board (e.g., in the cloud) When is it trained? When is it executed? • • Real-time Offline • • Online Offline (or batch processing) How is it trained? Continuous learning? • • Yes Supervised • • Un-supervised (semi-supervised) No Application • System design • Verification
Modular systems vs Holistic Design - Modular systems and ML to design individual components - Holistic and end to end learning for driving Sensor data Vehicle control
Modular systems (1) Well-defined task-specific modules (2) Could have parallel modules Examples: • Semantic segmentation and free space and drivable area detection • Object detection and tracking and information (speed, heading, type, ...) • Road information and geometry of the routes • Sensor fusion • Scene Semantics such as traffic and signs, turn indicators, on-road markings etc • Maps and updating them over time • Positioning and localization • Path planning • Driving policy learning and decision making • Other road user behavior analysis such as intention prediction • Driver monitoring
Holistic and end to end learning • First implemented around 28 years ago in the ALVINN system • Number of parameters<50,000 • Dave-2 • 9 layers (5 convolutional and 3 fully connected) • 250 000 parameters • Dean A. Pomerleau, ”ALVINN, an autonomous land vehicle in a neural network”., No. AIP -77, CMU, 1989. • Mariusz Bojarski, et al. "End to end learning for self- driving cars“, arXiv, Apr. 2016 . • Christopher Innocenti, Henrik Lindén, “Deep Learning for Autonomous Driving: A Holistic Approach for Decision Making”, thesis.
AD Verification • Requirements: setting safety scope for function and AD test scenarios • Test methods • Test track [Kalra, 2016] • Test in real traffic and expeditions • Virtual testing and simulations • Analyzing logged data • Need to have tools such as reference system • Need to have advanced analysis methods • Nidhi Kalra et al. "Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?." Transportation Research Part A: Policy and Practice 94 (2016): 182-193.
Some Examples
Object Detection (1) Accuracy (2) Speed • Ross Girshick et al., “Rich feature hierarchies for accurate object detection and semantic segmentation”, CVPR, 2014. • Shaoqing Ren et al., “Faster R-CNN: Towards real-time object detection with region proposal networks”, NIPS , 2015. • Jifeng Dai et al., “R -FCN: Object Detection via Region-based Fully Convolutional Networks”, arXiv, Jun., 2016. • Donal Scanlan, Lucia Diego, “Robust vehicle detection using convolutional networks“, Master’s thesis, ongoing.
Semantic Segmentation FCN • Jonathan Long et al., “Fully Convolutional Networks for Semantic Segmentation”, CVPR 2015. • Jifeng Dai et al., “Instance -aware Semantic Segmentation via Multi- task Network Cascades”, arXiv, 2015 • Vijay Badrinarayanan et al., “ SegNet: A Deep Convolutional Encoder-Decoder Architecture for Scene Segmentation”, PAMI 2017
Road Information [He 2016] • Christian Lipski et al. "A fast and robust approach to lane marking detection and lane tracking“, SSIAI, 2008 . • Florian Janda et al., "A road edge detection approach for marked and unmarked lanes based on video and radar“, FUSION, 2013. • Jihun Kim et al., “ Robust lane detection based on convolutional neural network and random sample consensus”, ICONIP, 2014. • Bei He et al., “Accurate and Robust Lane Detection based on Dual-View Convolutional Neutral Network”, IV, 2016. • Gabriel L. Oliveira et al., "Efficient deep models for monocular road segmentation“, IROS, 2016.
Traffic Sign Recognition German Traffic Sign Recognition Benchmark (GTSRB) [Stallkamp 2012] • Johannes Stallkamp et al. "Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition“, Neural networks , 2012. • Pierre Sermanet et al., "Traffic sign recognition with multi-scale convolutional networks“, IJCNN, 2011. • Konstantinos Mitritsakis , “Study Real World Traffic Environment Using Street View”, Master’s thesis, 2016.
Interacting with Humans • Eshed Ohn- Bar et al., “Looking at Humans in the Age of Self - Driving and Highly Automated Vehicles”, Trans. on IV, 2015
Driver monitoring • Kevan Yuen et al. ,”Looking at Faces in a Vehicle: A Deep CNN Based Approach and Evaluation”, ITSC, 2016 . • Akshay R. Siddharth et al., “Driver Hand Localization and Grasp Analysis: A Vision-based Real-time Approach ”, ITSC, 2016 . • Tianchi Liu et al. "Driver distraction detection using semi-supervised machine learning." IEEE Transactions on ITS, 2016.
Recommend
More recommend
