Dagstuhl: December 2016 1 Formal Specification Debugging for Testing and Monitoring of Cyber-Physical Systems Georgios Fainekos Joint work with Adel Dokhanchi and Bardh Hoxha School of Computing, Informatics and Decision System Engineering Arizona State University fainekos at asu.edu CPS PS Lab Lab
Dagstuhl: December 2016 2 Problem: automotive software recalls on the rise Samples of recent recalls from different OEMs: No downshifting from 5 th to 4 th gear Rough idling or stalling due to complicated adaptive ECU Electric motor to rotate in the direction opposite to that selected by the transmission Cruise control does not disengage unless turning off the ignition CPS PS Lab Lab
Dagstuhl: December 2016 3 Challenge: Why there are so many bugs? Correct Software Correct System Behavior Due to the complex interactions between the physical system and software: • Analytical tools and theory cannot provide correct-by-design guarantees • The verification problem is undecidable / complete methods do not scale • Humans cannot predict the conditions that lead to bad behaviors Blood Glucose wearable continuous glucose monitor Time Insulin Infusion Controller Pump CPS PS Lab Lab
Dagstuhl: December 2016 4 Vision: a complete theory for MBD for CPS Transparent from the user perspective: 1. Automated synthesis Informal System Requirements 2. T esting and verification support Deployment with guarantees System Calibration Formal Specifications Hardware In the Loop (HIL) Processor In the Model Design Awards: Loop (PIL) 1017074, 1116136, 1319560, 1350420, 1446730 Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and Autocode Generation do not necessarily reflect the views of (with multi-core in mind) the National Science Foundation. CPS PS Lab Lab
Dagstuhl: December 2016 5 S-Taliro support in the V-process Informal System Requirements Deployment 4 System Calibration Formal 3 5 Specifications Hardware In the 1 2 Loop (HIL) 1 S-Taliro Processor In the Model Design 2 support Loop (PIL) 1. Testing formal specifications and specification mining [TECS 2013, ICTSS 2012, …] 2. Conformance testing: models, HIL/PIL or tuned/calibrated model [MEMOCODE 2014] 3. Testing formal specifications on the HIL/PIL calibrated system [TECS 2013, …] 4. Runtime monitoring of formal requirements [RV 2014] Autocode Generation 5. Specification visualization [IROS 2015] & Debugging [MEMOCODE 2015] (with multi-core in mind) CPS PS Lab Lab
Dagstuhl: December 2016 6 Previous Work: ViSpec • ViSpec helps transforming pre-specified templates in NL • No need for MITL background Hoxha, Bach, Abbas, Dokhanchi, Kobayashi, Fainekos, Towards Formal Specification Visualization for Testing and Monitoring of Cyber-Physical Systems, DIFTS 2014 Hoxha, Mavridis and Fainekos, VISPEC: A graphical tool for easy elicitation of MTL requirements, IROS 2015 CPS PS Lab Lab
Dagstuhl: December 2016 7 ViSpec – Specification Classes Safety: Implication: □ 𝐽 𝜚 𝜚 → 𝜔 Reachability: Reactive Response: ◇ 𝐽 𝜚 □ 𝐽 (𝜚 → 𝑁 𝐽 𝜔) Stabilization: Conjunction: ◇ 𝐽 □ 𝐽 𝜚 𝜚 ∧ 𝜔 Non-strict Sequencing: Recurrence: □ 𝐽 ◇ 𝐽 𝜚 𝑂 𝐽 (𝜚 ∧ 𝑁 𝐽 𝜔) 𝑁, 𝑂 ∈ □ , ◇ CPS PS Lab Lab
Dagstuhl: December 2016 8 Motivating Example: On-Line Survey B. Hoxha, N. Mavridis and G. Fainekos, VISPEC: A graphical tool for easy elicitation of MTL requirements, IROS 2015 We asked: “At some time in the first 30 seconds, the vehicle speed (v) will go over 100 and stay above 100 for 20 seconds ” Response: 𝜒 = ◇ [0,30] ((𝑤 > 100) ⇒ □ [0,20] (𝑤 > 100)) ∃t ∈ [0,30] ( 𝑤 𝑢 > 100 ⇒ ∀𝑢 ′ ∈ [t,t+20] (𝑤(𝑢) > 100)) 𝜒 is a tautology • 𝑤 > 100 =⊥ at any time in [0,30] (𝑤 > 100) ⇒ □ [0,20] (𝑤 > 100) = ⊤ • 𝑤 > 100 = ⊤ for all the time in [0,30] □ [0,20] 𝑤 > 100 = ⊤ between [0,10] (𝑤 > 100) ⇒ □ [0,20] (𝑤 > 100) = ⊤ between [0,10] CPS PS Lab Lab
Dagstuhl: December 2016 9 Somewhere in Switzerland … Not a software bug! From the Tesla Model X Owner’s manual: Invalid requirements: At the presence of stationary vehicles the Traffic-Aware Cruise Control may or may not break! At the absence of vehicles the Traffic-Aware Cruise Control may or may not break! CPS PS Lab Lab
Dagstuhl: December 2016 10 Problem Formulation Problem 1 (System Independent MITL Analysis): Given an MITL formula ϕ, find whether ϕ has any of the following logical issues: • Validity : the specification is unsatisfiable or a tautology. • Redundancy : the formula has redundant conjuncts. • Vacuity : some subformulas do not contribute to the satisfiability of the formula. Problem 2 (System Dependent Vacuity Checking): Given an MITL formula ϕ, and signal µ, check whether µ satisfies the antecedent failure mutation of ϕ. A. Dokhanchi, B. Hoxha, and G. Fainekos, Metric interval temporal logic specification elicitation and debugging . MEMOCODE 2015, Austin, TX, USA CPS PS Lab Lab
Dagstuhl: December 2016 11 Overview • Motivation • Preliminaries • System Independent MITL Analysis • System Dependent Vacuity Checking • Experiments • Conclusion & Future Research CPS PS Lab Lab
Dagstuhl: December 2016 12 Metric Interval Temporal Logic: Semantic Intuition ¬𝜚 𝜚 1 ∨ 𝜚 2 □ 𝐽 𝜚 ◇ 𝐽 𝜚 | 𝜚 1 𝑉 𝐽 𝜚 2 𝜚 ∷= ⊤ | 𝑞 □ 𝑏 - always a a a a a a a ◇ [1,3] a - eventually a * * * a * * 𝑏 𝑉 𝑐 - a until b a a a b * * 𝑏 𝑉 [1,1.5] 𝑐 -a until b 0 0.4 0.7 1.1 1.2 1.7 now time CPS PS Lab Lab
Dagstuhl: December 2016 13 Subset of MITL • Bounded-MITL( ◇ , □ ) with only Always & Eventually operator • Negation Normal Form • Syntax: 𝜚 ∷= ⊤ ⊥ 𝑞 ¬𝑞 𝜚 1 ∨ 𝜚 2 | 𝜚 1 ∧ 𝜚 2 □ 𝐽 𝜚 ◇ 𝐽 𝜚 • No Until, Release, Next operator is used CPS PS Lab Lab
Dagstuhl: December 2016 14 Signal Temporal Logic Specification example: ◇ [1.1,3.2] (𝑦(𝑢) ≥ 𝑦 0 ) 𝑦 𝑢 ∈ R a x 0 Real-Value Signal Time t 3.2 1.1 Boolean abstraction a Boolean Signal Time t 1.1 3.2 Notice example is MITL if we replace the predicate with a proposition: 𝑏 ≡ (𝑦(𝑢) ≥ 𝑦 0 ) CPS PS Lab Lab
Dagstuhl: December 2016 15 Overview • Motivation • Preliminaries • System Independent MITL Analysis • System Dependent Vacuity Checking • Experiments • Conclusion & Future Research CPS PS Lab Lab
Dagstuhl: December 2016 16 Transforming STL to MITL Example: 𝑡𝑞𝑓𝑓𝑒 > 100 ⇒ 𝑡𝑞𝑓𝑓𝑒 > 80 Question: 𝑏 ⟹ 𝑐 ? Speed a b 100 80 t time Boolean abstraction a ≡ 𝑇𝑞𝑓𝑓𝑒 > 100 time b ≡ 𝑇𝑞𝑓𝑓𝑒 > 80 time c 100 ≥ 𝑇𝑞𝑓𝑓𝑒 > 80 time 𝑏 ⇒ 𝑏 ∨ 𝑑 CPS PS Lab Lab
Dagstuhl: December 2016 17 Debugging MITL Specification Specification Elicitation Framework 3-Levels of Specification Debugging MITL Passed CPS PS Lab Lab
Dagstuhl: December 2016 18 Validity Issues Detection Checking whether 𝜒 is unsatisfiable or a tautology A valid specification is one where 𝜒 and ¬𝜒 are satisfiable We asked: “ At some time in the first 30 seconds, the vehicle speed (v) will go over 100 and stay above 100 for 20 seconds” Response: 𝜒 = ◇ [0,30] ( (𝑤 > 100) ⇒ □ [0,20] (𝑤 > 100) ) 𝜒 is a tautology CPS PS Lab Lab
Dagstuhl: December 2016 19 Redundancy Issues Detection 𝑙 Conjunctive formula: Φ =ٿ 𝑘=1 𝜒 𝑘 Removing conjunct: 𝑗−1 𝜒 𝑘 ∧ٿ 𝑘=𝑗+1 𝑙 𝜒 𝑘 ≡ Φ\𝜒 𝑗 ٿ 𝑘=1 If ∃ 𝜒 𝑗 Φ\𝜒 𝑗 ⊨ 𝜒 𝑗 Then 𝜒 𝑗 is redundant Example 𝜒 2 = 𝑞 ∧ □ [0,10] 𝑞 □ [0,10] 𝑞 ⊨ 𝑞 Algorithm 1: Checks Φ\𝜒 𝑗 ⊨ 𝜒 𝑗 for each conjunct Creates a list of redundant conjuncts H. Chockler and O. Strichman, Before and after vacuity. CPS PS Lab Lab Form. Methods Syst. Des., 34(1):37 – 58, Feb. 2009.
Dagstuhl: December 2016 20 Redundancy Example User response to “ At some point in time in the first 30 seconds, vehicle speed will go over 100 and stay above for 20 seconds. ” 𝜒 = ◇ [0,30] (𝑡𝑞𝑓𝑓𝑒 > 100) ∧ ◇ [0,20] (𝑡𝑞𝑓𝑓𝑒 > 100) ◇ [0,30] (𝑤 > 100) is redundant since ◇ [0,20] (𝑤 > 100) ⊨ ◇ [0,30] (𝑤 > 100) CPS PS Lab Lab
Dagstuhl: December 2016 21 Vacuity Issues Detection If sub-formula 𝜔 ∈ 𝜒 does not affect the satisfiability of 𝜒 , then 𝜒 is vacuous Remove 𝜔 Vacuous specifications are equivalent to their mutant H. Chockler and O. Strichman, Before and after vacuity. Form. Methods Syst. Des., 34(1):37 – 58, Feb. 2009. CPS PS Lab Lab
Dagstuhl: December 2016 22 Mutation of MITL for Vacuity Checking Mutation with assigning ⊥ to literal occurrence 𝜒 = (¬𝑞 ∧ 𝑟) ∨ ◇ [0,10] 𝑞 ∨ □ [0,10] 𝑟 𝜒[¬𝑞 ←⊥] = (⊥∧ 𝑟) ∨ ◇ [0,10] 𝑞 ∨ □ [0,10] 𝑟 4 literal occurrence => 4 mutation Algorithm 2: Checks Φ ⊨ 𝜒 𝑗 𝑚 ←⊥ for each mutation Creates a list of mutated sub-formulas CPS PS Lab Lab
Recommend
More recommend
