PURSS: Towards Perceptual Uncertainty Aware Responsibility Sensitive - PowerPoint PPT Presentation

PURSS: Towards Perceptual Uncertainty Aware Responsibility Sensitive Safety with ML Rick Salay, 1 Krzysztof Czarnecki, 1 Ignacio Alvarez, 2 Maria Soledad Elli, 2 Sean Sedwards, 1 Jack Weast 2 1 Dept. Electrical and Computer Engineering, Univ. of Waterloo 2 Intel Corporation, Automated Driving Group 1

Automated Driving Systems (ADS) ADS Sensing Actuation World model Planning & Perception control 2

Automated Driving Systems (ADS) Traditional Safety Assurance ADS Specification ADS verify Sensing Actuation Planning & World model Perception control 3

Automated Driving Systems (ADS) Traditional Safety Assurance ADS Specification RSS Responsibility Sensitive Safety ADS verify Sensing Actuation World model Planning & Perception control 4

Responsible Sensitive Safety (RSS) Formalizes “ common sense safety” e.g., Do not hit the car in front 5

Responsible Sensitive Safety (RSS) Do not hit the car in front - Safe actions maintain distance 𝑒 𝑛𝑗𝑜 If 𝑒 𝑛𝑗𝑜 is breached, “proper - response” is safe action Shalev-Shwartz, Shai, Shaked Shammah, and Amnon Shashua. "On a formal model of safe and scalable self-driving cars." arXiv preprint arXiv:1708.06374 (2017). 6

Responsible Sensitive Safety (RSS) Do not hit the car in front Problem: Assumes perfect perception Misperception -> wrong action -> safety risk! 7

Automated Driving Systems (ADS) Traditional Safety Assurance ADS Specification RSS Responsibility ?? Sensitive Safety ADS verify verify Sensing Actuation Planning & World model Perception control 8

Automated Driving Systems (ADS) Traditional Safety Assurance ADS Specification RSS Responsibility ?? Sensitive Safety ADS Sensing Actuation World model Planning & Perception control 9

Automated Driving Systems (ADS) Traditional Safety Assurance ADS Specification RSS Responsibility ?? Sensitive Safety ADS Sensing Actuation World model Planning & Perception control 10

Automated Driving Systems (ADS) Traditional Safety Assurance ADS Specification RSS Responsibility ?? Is there Sensitive Safety another ADS approach to perceptual Sensing Actuation Planning & Perception safety? control 11

Perceptual Uncertainty • Uncertainty of perceptual component is cause of misperception – many factors * : poor labeling, inadequate dataset coverage, etc. • ML components can report their own uncertainty! – a s long as they are calibrated… * Czarnecki, Krzysztof, and Rick Salay. "Towards a framework to manage perceptual uncertainty for safe automated driving." In International Conference on Computer Safety, Reliability, and Security , pp. 439-445. Springer, Cham, 2018. 12

PURSS PURSS = perceptual uncertainty (PU) + RSS Safety Idea: Use perceptual uncertainty measure to make RSS rules appropriately cautious and limit safety risk PURSS formalizes this idea 13

Precise World Model Real-world situation True state Perception (+ PU) (unknowable) Pedestrian Pedestrian speed = 0 speed = 0.1 activity = activity = standing walking Accuracy Misperception: precise but inaccurate 14

Perceptual Uncertainty Handling via Imprecise World Models Real-world situation Perception (+PU) PU -> Imprecise World Model (𝜷) True state (unknowable) Pedestrian speed = 0 Pedestrian activity = speed = 0 standing activity = standing Accuracy Pedestrian speed = 0.1 … activity = walking Probability 𝛽 that true Covers a “credible set” of world model is in the set world models with conf. level 𝛽 15

Perceptual Uncertainty Handling via Imprecise World Models Real-world situation Perception (+PU) PU -> Imprecise World Model (𝜷) True state (unknowable) Pedestrian speed = 0 Pedestrian activity = Safety parameter 𝛽 is set to speed = 0 standing activity = desired level of safety standing Accuracy Pedestrian speed = 0.1 … activity = walking Probability 𝛽 that true Covers a “credible set” of world model is in the set world models with conf. level 𝛽 16

Perceptual Uncertainty Handling via Imprecise World Models Real-world situation Perception (+PU) RSS rules are “lifted” to accept imprecise PU -> Imprecise World Model (𝜷) True state (unknowable) world models Pedestrian speed = 0 Pedestrian activity = speed = 0 Result: exercises caution by limiting actions standing activity = standing Accuracy Pedestrian to those safe for any covered world model speed = 0.1 … activity = walking Probability 𝛽 that true Covers a “credible set” of world model is in the set world models with conf. level 𝛽 17

Responsible Sensitive Safety (RSS) Do not hit the car in front Lifting: replace values with credible intervals corresponding to 𝛽 18

Responsible Sensitive Safety (RSS) Do not hit the car in front Lifting: replace values with credible intervals corresponding to 𝛽 e.g., precise: 𝑤 𝑔 = 30 𝑛/𝑡 2 = 1 𝑛/𝑡 PU: 𝜏 𝑔 lift to imprecise: 𝛽 = 68% : 𝑤 𝑔 = 29,31 𝑛/𝑡 𝛽 = 95% : 𝑤 𝑔 = 28,32 𝑛/𝑡 19

Responsible Sensitive Safety (RSS) Do not hit the car in front 2 , Given uncertainty 𝜏 Lifting: replace values with credible 𝑔 intervals corresponding to 𝛽 increasing confidence 𝛽 ⇒ e.g., decreasing precision of 𝑤 𝑔 ⇒ precise: 𝑤 𝑔 = 30 𝑛/𝑡 2 = 1 𝑛/𝑡 PU: 𝜏 larger 𝑒 𝑛𝑗𝑜 to be more cautious 𝑔 lift to imprecise: 𝛽 = 68% : 𝑤 𝑔 = 29,31 𝑛/𝑡 𝛽 = 95% : 𝑤 𝑔 = 28,32 𝑛/𝑡 20

Responsible Sensitive Safety (RSS) Do not hit the car in front 2 , Given uncertainty 𝜏 Lifting: replace values with credible 𝑔 intervals corresponding to 𝛽 increasing confidence 𝛽 ⇒ e.g., decreasing precision of 𝑤 𝑔 ⇒ precise: 𝑤 𝑔 = 30 𝑛/𝑡 2 = 1 𝑛/𝑡 PU: 𝜏 larger 𝑒 𝑛𝑗𝑜 to be more cautious 𝑔 lift to imprecise: 𝛽 = 68% : 𝑤 𝑔 = 29,31 𝑛/𝑡 𝛽 = 95% : 𝑤 𝑔 = 28,32 𝑛/𝑡 21

Benefits and Costs • Benefit: Safety parameter 𝛽 can be increased to get as safe as you want – RSS rules become correspondingly more cautious • Cost: More cautious behaviour may negatively impact progress • Important future work: negotiating the trade- off 22

Summary • RSS provides a spec on planning & control – supports traditional safety assurance • Perception is hard to specify and needs ML – different safety approach is needed • PURSS approach to safety – Set desired level of safety ( 𝛽 ) – Perceptual uncertainty → 𝛽 imprecise world models – Lift RSS rules to be correspondingly cautious • Much further work coming! 23