Uncertainty-Centric Safety Assurance of ML- Based Perception for - PowerPoint PPT Presentation

Uncertainty-Centric Safety Assurance of ML- Based Perception for Automated Driving Krzysztof Czarnecki Waterloo Intelligent Systems Engineering (WISE) Lab University of Waterloo 1

Uncertainty-Centric Assurance of ML-Based Perception Misclassifications, Uncertainty Perceptual under-classifications, Influence factors Uncertainty quantitative errors (domain coverage, sensor noise, etc.) Perceptual Uncertainty Aware Uncertainty Responsibility Sensitive Safety Management (PURSS) Safety requirements on perception 2

Uncertainty-Centric Assurance of ML-Based Perception Misclassifications, Uncertainty Perceptual under-classifications, Influence factors Uncertainty quantitative errors (domain coverage, sensor noise, etc.) R. Salay, K. Czarnecki, M. Elli, I. Alvarez, S. Sedwards, J. Weast. PURSS: Towards a Perceptual Uncertainty- Perceptual Uncertainty Aware Aware Responsibility Sensitive Safety . Uncertainty Under submission. Responsibility Sensitive Safety Management (PURSS) Safety requirements on perception 3

Responsible Sensitive Safety (RSS) • Defines responsible behavior to address behavioral uncertainty – Safe actions when safe and proper response when not safe • Guarantees no collision when everyone follows the rules 4

Responsible Sensitive Safety (RSS) RULE 1. Do not hit the car in front (longitudinal distance) RULE 2. Do not cut in recklessly (lateral distance) RULE 3. Right of way is given, not taken RULE 4. Be cautious in areas with limited visibility RULE 5. If you can avoid a crash without causing another one, you must https://arxiv.org/abs/1708.06374 5

RULE 1. Safe Following Distance in RSS Distance traveled Distance traveled due to reaction time Braking distance by front vehicle 6

RULE 1. Safe Following Distance in RSS Problem: Assumes perfect perception 7

Perception Triangle Real-world situation True state Perception (unknowable) Pedestrian Pedestrian speed = 0 speed = 0.1 activity = activity = standing walking Accuracy 8

Safety Argument Decomposition ADS World model Sensing Actuation Planning & Perception control 9

RSS as a Constraint on ADS RSS Sensing World model Planning & Actuation Perception control ADS World model Sensing Actuation Planning & Perception control 10

RSS as a Constraint on ADS RSS Sensing World model Planning & Actuation Perception control ADS World model Sensing Actuation Planning & Perception control 11

Sample RSS-Compliant World Model Schema Safe following distance Safe action set Safe(s) 12

Perception Cases ( s → s’ ) Misperception Correct Perception s → s’ where s ¹ s’ s → s’ where s = s’ Real-world situation Real-world situation True state True state Perception Perception (unknowable) (unknowable) s s’ s s’ Pedestrian Pedestrian Pedestrian Pedestrian s = s’ s ¹ s’ speed = 0 speed = 0 speed = 0.1 speed = 0 activity = activity = activity = activity = standing standing walking standing 13

Safety of Perception Misperception s → s’ potentially causes safety risk iff 14

Safety-Irrelevant Misperceptions Misperception s → s’ where Safe(s) = Safe(s’) 15

Precise World Model Real-world situation True state Perception (unknowable) Pedestrian Pedestrian speed = 0 speed = 0.1 activity = activity = standing walking Accuracy 16

Perceptual Uncertainty Handling via Imprecise World Models Real-world situation Perception True state Imprecise World Model (unknowable) Pedestrian speed = 0 Pedestrian activity = speed = 0 standing activity = standing Accuracy Pedestrian speed = 0.1 … activity = walking Set of credible states at conf. level a 17

Perceptual Uncertainty Aware RSS (PURSS) RSS – rules lifted to imprecise world model Planning & Imprecise Safe Actions Imprecise Planning & Planning & control Perception world model control control ADS World model Imperfect Planning & Action Perception control Situation 18

Lifting World Model Schema to Imprecise World Model Schema Elementwise lifting: • Class entity to superclass • Continuous value to interval • Discrete value to enumerated set • Derived attributes via set operations and interval arithmetic 19

Using Imprecise World Models to Mitigate Misperception Given an under-perception case, where S is an imprecise model of confidence α perceived when the correct model: A safe action in an imprecise model must be safe for every precise model covered by the imprecise model. 20

Different Risk Levels a =10 -4 a =10 -9 a =10 -4 a =10 -9 21

Imprecise Classification when High Integrity Required a =10 -4 a =10 -4 a =10 -9 ? 22

Conservative Action for High Integrity a =10 -4 a =10 -4 a =10 -9 ? 23

Example of Mitigation Any No Lane Obstruction in Front Lane Obstruction in Front (LOF) Static LOF Front Vehicle Actions: continue or stop or follow 24

Safety Requirements on Perception Performance from PURSS Correct LOF/NLOF classification and Any distance ± 5 cm at a LOF = 10 -9 for 100% of time duration within ODD conditions No Lane Obstruction in Front Lane Obstruction in Front (LOF) Static LOF Front Vehicle Correct FV/SLOF classification and distance ± 25 cm and velocity ± 0.5 m/s at a FV = 10 -4 for 90% of time duration within ODD condition 25

Uncertainty-Centric Assurance of ML-Based Perception Misclassifications, Uncertainty Perceptual under-classifications, Influence factors Uncertainty quantitative errors (domain coverage, sensor noise, etc.) K. Czarnecki and R. Salay. Towards a Framework to Manage Perceptual Uncertainty for Safe Perceptual Uncertainty Aware Automated Driving . Uncertainty WAISE’18 Responsibility Sensitive Safety Management (PURSS) Safety requirements on perception 26

Guide to the Expression of Uncertainty in Measurement (GUM) • True accuracy unknowable – Accuracy in ML wrt. test set only • Must estimate uncertainty 27

Perception Triangle (Instance-Level) Real-world situation Sensory True state channel (unknowable) Pedestrian Perception speed = 0 Camera Pedestrian activity = speed = 0 image, standing activity = radar standing Perception Pedestrian data speed = 0.1 algorithm … Accuracy activity = walking Set of credible states (uncertain) 28

Perceptual Triangle Real-world situation Real-world situations Sensory True state channel (unknowable) Sensory Pedestrian Semantics Perception speed = 0 Camera channel Pedestrian activity = speed = 0 image, standing activity = radar Perception Pedestrian standing Perception speed = 0.1 data … algorithm Accuracy activity = Sensory walking Concept data Data Set of credible states interpretation (uncertain) Instance-level Domain-level (generic) 29

Perceptual Triangle When Using Supervised ML Development Operation Operational Development situations and situations and scenarios scenarios Partial Resulting Sensory Sensory semantics perception channel channel (examples) Training Inference & testing Sensory Sensory Concept Concept data Data data Inferred labeling state Trained Model Model class selection, training & testing 30

Factors Influencing Uncertainty (F1-7) Development Operation Operational Development Domain shift F7 situations and situations and scenarios scenarios F3 F2 Partial F3 F2 Resulting Sensory Sensory semantics perception channel channel (examples) F4 F4 Training Inference F1 & testing Sensory Sensory Concept Concept data Data data Inferred labeling state F5 Trained Model Model class selection, training & testing F6 31 K. Czarnecki and R. Salay. Towards a Framework to Manage Perceptual Uncertainty for Safe Automated Driving. WAISE’18

Factors Influencing Uncertainty (F1-7) Development Operation Operational Development Domain shift F7 situations and situations and scenarios scenarios F3 F2 F3 Partial F2 Resulting Sensory Sensory semantics perception channel channel (examples) F4 F4 Training Inference F1 & testing Sensory Sensory Concept Concept data Data data Inferred labeling state F5 Trained Model Model class selection, training & testing F6 32 K. Czarnecki and R. Salay. Towards a Framework to Manage Perceptual Uncertainty for Safe Automated Driving. WAISE’18

F3: Scene Uncertainty 33

F3: Scene Uncertainty • Surrogate measures – range, scale, occlusion level, atmospheric visibility, illumination, clutter and crowding level • Also part of development data set coverage • To determine sufficient coverage, compare these measures with 1. Test set accuracy 2. Estimated uncertainty by the network 34

Synthetic Dataset to Study Scene Influence Factors Samin Khan, Buu Phan, Rick Salay, and Krzysztof Czarnecki. ProcSy: Procedural Synthetic Dataset Generation Towards Influence Factor Studies 35 Of Semantic Segmentation Networks. Workshop on Vision for All Seasons: Bad Weather and Nighttime, associated with CVPR, Long Beach, 2019

Scene Influence Factors -> Accuracy 37

38

39