Verification of Robotics and Autonomous Deep Learning Verification - PowerPoint PPT Presentation

Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Verification of Robotics and Autonomous Deep Learning Verification Systems Safety Definition Challenges Approaches Experimental Results Verification in Xiaowei Huang, University of Liverpool human-robot interaction Motivation Joint work with Prof. Marta Kwiatkowska, University of Oxford Stochastic Multiplayer Game Cognitive Mechanism Alpine Verification Meeting, November 25, 2017 A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Outline Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Challenges: Robotics and Autonomous Systems Deep Learning Verification of Deep Learning [1] Verification Safety Definition Verification of Human-Robot Interaction [ ? ] Challenges Approaches Experimental Conclusion Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Robotics and Autonomous Systems Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Robotics and Autonomous Systems Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Robotic and autonomous systems (RAS) are interactive, Deep Learning Verification cognitive and interconnected tools that perform useful tasks in Safety Definition Challenges the real world where we live and work. Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Automated Verification, a.k.a. Model Checking Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Systems for Verification: Paradigm Shifting Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

System Properties Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges dependability (or reliability) Deep Learning Verification human values, such as trustworthiness, morality, ethics, Safety Definition Challenges transparency, etc Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Verification of Deep Learning Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Human-Level Intelligence Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Major problems and critiques Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning un-safe, e.g., instability to adversarial examples Verification Safety Definition hard to explain to human users Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Human Driving vs. Autonomous Driving Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Traffic image from “The German Traffic Sign Recognition Cognitive Mechanism A Temporal Benchmark” Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Deep learning verification (DLV) Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Image generated from our tool Deep Learning Verification (DLV) 1 Cognitive Mechanism A Temporal Logic of Trust 1 X. Huang and M. Kwiatkowska. Safety verification of deep neural Complexity networks . CAV-2017. Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Safety Problem: Tesla incident Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Microsoft Chatbot Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction On 23 Mar 2016, Microsoft launched a new artificial Motivation Stochastic Multiplayer intelligence chat bot that it claims will become smarter the Game Cognitive more you talk to it. Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Microsoft Chatbot Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game after 24 hours ... Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Safety Problem: Microsoft Chatbot Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98

Safety Problem: Microsoft Chatbot Verification of Robotics and Autonomous Systems Xiaowei Huang Challenges Deep Learning Verification Safety Definition Challenges Approaches Experimental Results Verification in human-robot interaction Motivation Stochastic Multiplayer Game Cognitive Mechanism A Temporal Logic of Trust Complexity Conclusion Alpine Verification Meeting, November 25, 2017 Xiaowei Huang (Liverpool University) Verification of Robotics and Autonomous Systems / 98