Testing Real-Time Embedded Systems Using UppAal-TRON -Tool and - PowerPoint PPT Presentation

Testing Real-Time Embedded Systems Using UppAal-TRON -Tool and Application Kim G. Larsen, Marius Mikucionis, Brian Nielsen, Arne Skou Aalborg University, DK {kgl | marius | bnielsen | ask}@cs.aau.dk

Agenda � Automated Model-based Testing � Testing Framework � Timed Automata � Environment Modeling � Relativized I/O conformance � Online Testing Algorithm � Danfoss EKC � Other Issues � Monitoring and Environment Emulation � Coverage Measurement � Demo � Conclusions & Future Work 2

Testing Embedded Software � Testing: Execute actual software (system) with controlled inputs and check responses � To find errors � To determine risk of release � 10-20 errors per 1000 LOC � 30-50 % of development time and cost � Software and complexity increases 3

Automated Model-Based Testing Model Test suite pass DBLclick! Test Test Test Test execution Test execution Gene- x>=2 Test Gene- click? tool Generator tool x:=0 Generator rator rator Event tool fail tool tool tool click? mapping x<2 Driver Selection & optimization I m p l e m e n t a t i o n U n d Implementation Relation e r T e s t Does the behavior of the ( blackbox ) implementation comply to that of the specification? 4

Online Testing Model pass DBLclick! Test Test Test Test execution Test execution Gene- x>=2 Test Gene- click? input input input input tool Generator tool x:=0 Generator rator rator Event tool fail output output output output tool tool tool click? mapping x<2 Driver Selection & optimization I m p l e m e n t a t i o n U n d Implementation Relation e r T e s t • Test generated and executed event-by-event (randomly), reactively • Long Running, deep testing, imaginative 5

Real-Time Systems Real Time System A system where correctness not only depends on the logical order of events but also on their timing sensors Task Task Task Modelling & Task actuators Abstraction Controller Environment Input Environment System Output Model Model Σ 6

Our Framework • UppAal Timed Automata Network: Env || IUT ”Formal Relativized i/o conformance” Relation Correct system behavior •Relevant input event •Test Oracle sequences •Monitor •Load model • Complete and sound algorithm •Efficient symbolic reachability algorithms • UppAal-TRON: Testing Real-Time Systems Online •Release 1.3 http://www.cs.aau.dk/~marius/tron/ 7

Related Work � Formal Testing Frameworks � [Brinksma, Tretmans] � Real-Time Implementation Relations � [Khoumsi’03, Briones’04] � Symbolic Reachability analysis of Timed Automata � [Dill’89, Larsen’97,…] � Online state-set computation � [Tripakis’02] � Online Testing � [Tretmans’99, Peleska’02, Krichen’04] 8

Sample Test Runs highTemp!·3·compressorOn? ⇒ PASS highTemp!·3·compressorOff? ⇒ FAIL highTemp!·13·compressorOn? ⇒ FAIL highTemp!·3·compressorOn?·123 · lowTemp!·3·compressorOff? ⇒ PASS highTemp!·3·compressorOn?·17 · lowTemp!·3·compressorOff? · 3.14 · highTemp!·5·compressorOn?·177 · lowTemp!·3·compressorOff? ⇒ PASS INFINITELY MANY SEQUENCES!!!!!! 10

Sample Cooling Controller C r On! Off! Low? Med? High? IUT-model Env-model 11

Env. Modeling Temp. � Realism and Guiding High! � E M Any action possible at any time Med! � E 1 Only realistic temperature variations � E 2 Temperature never increases when cooling Low! � E L No inputs (completely passive) time E M E 1 E 2 E L E L E 2 E 1 E M 12

Sample Cooling Controller C’ r C r On! Off! Low? E M Med? High? IUT Env-model C ’r rt-ioco EM C r 13

Sample Cooling Controller C’ r On! Off! Low? Med? High? E 1 IUT Env-model C ’r rt-ioco E1 C r , iff 3d<r d.Med?.d.High?.d.Med?.d.Low?. ε .On, ε ≤ r 14

Sample Cooling Controller C’ r On! Off! Low? Med? High? E 2 IUT Env-model C ’r rt-ioco E2 C r 15

Non-Determinism • Modeling Action uncertainty •A controller switches a relay when a control variable crosses ‘around’ threshold value T switchOn! threshold ± err switchOff! time • Modeling Timing uncertainty •A controller switches a relay between 2 and 10 time units 16

Implementation relation Relativized real-time io-conformance ε 0 ,i 0 , ε 1 ,i 1 … e s i System Environment IUT ε 0’ ,o 0 , ε 1’ ,o 1 … Model assumptions • Let P be a set of states • TTr ( P ): the set of timed traces from states in P • P after σ = the set of states reachable after timed trace σ • Out ( P ) = possible outputs and delays in P •i rt-ioco e s =def • ∀σ ∈ TTr(e): Out((e,i) after σ ) ⊆ Out((e,s) after σ ) •i rt-ioco e s iff TTr(i) ∩ TTr(e) ⊆ TTr(s) ∩ TTr(e) • Intuition, for all relevant environment behaviors • never produces illegal output, and • always produces required output in time • ~timed trace inclusion 17

Randomized Online Algorithm Algorithm TestGenExec (TestSpec) returns { pass , fail } Z :={ 〈 l 0 ,0 〉 }, While Z ≠∅ and #iterations ≤ T do choose randomly 1. if EnvOutput ( Z ) ≠∅ // Offer an input choose randomly a ∈ EnvOutput ( Z ) send i to SUT Z :=Z after a choose randomly δ ∈ Delays ( Z ) // Delay and wait for output 2. Wait ( δ ) if o occurred after δ ’ ≤ δ then Z :=Z a fter δ ’ if o ∉ ImpOutpu t( Z ) then return fail Z :=Z after o else // no output within δ time Z :=Z after δ 3. reset IUT Z :={ 〈 l 0 ,0 〉 } • Sound if Z = ∅ then return fail else return pass • Complete as T → ∞ 18

Sound & Complete � TestGenExec is � sound � Fail verdict ⇒ ¬( I ioco e S) � complete � ¬( I ioco e S) ⇒ Prob(Fail) → 1 as T →∞ � (using only unit delays) � Assuming � IUT can be modeled by an input enabled, deterministic, non-blocking IO-TLOTS with isolated outputs � Time unit of IUT is known � TTr(IUT) and TTr(E) are closed under digitization � LTS induced by TA with only non-strict guards � TTr(S) closed under inverse digitization � LTS induced by TA with only strict guards 19

State-set computation � Compute all potential states the model can occupy after the timed trace ε 0 ,i 0 , ε 1 ,o 1 , ε 2 ,i 2 ,o 2 ,… � Let Z be a set of states � Z after a : possible states after executing a (and t*) � Z after ε : possible states after t* and ε i , totaling a delay of ε � o is a legal output from SUT iff O in ImpOutput(Z) � a is a relevant input in Env iff I in EnvOutput(Z) � ε is a permitted delay iff Z after ε ≠∅ � ε is a relevant delay iff Delays (Z) 20

State-set Computation � Compute all potential states the model can occupy after the timed trace ε 0 ,i 0 , ε 1 ,o 1 , ε 2 ,i 2 ,o 2 ,… � Let Z be a set of states � Z after a : possible states after executing a (and τ * ) � Z after ε : possible states after τ * and ε i , totaling a delay of ε l 1 x ≤ 7, a τ , x:=0 τ a l 0 l 2 l 4 l 0 l 1 a, x:=0 l 3 { 〈 l 0 ,x=3 〉 } after a = { 〈 l 0 ,x=0 〉 } after 4 = { 〈 l 0 ,x=4 〉 , 〈 l 1 , 0 ≤ x ≤ 4 〉 } { 〈 l 2 ,x=3 〉 , 〈 l 4 , x=3 〉 , 〈 l 3 , x=0 〉 } � Represent state sets as sets of symbolic states � Use symbolic reachability � (similar to model checkers like UppAal) 21

Symbolic Reachability 22

Real-time Online •Compute all states reachable after timed trace •Maintain a set of symbolic states in real time! Specification Online Tester: TA-network Z 4 i! Z 11 Z 5 Z 3 Z 17 Z 1 Z 7 2.75 System Z 14 Z 8 Z 0 Under O? Z 16 Z 9 Test Z 2 Z 15 Z 18 Z 6 Z 12 [Tripakis’02, Krichen’04] 23

Danfoss EKC Case Electronic Cooling Controller Sensor Input •air temperature sensor •defrost temperature sensor •(door open sensor) Keypad Input •2 buttons (~40 user settable parameters) Output Relays •compressor relay •defrost relay •alarm relay •(fan relay) Display Output •alarm / error indication •mode indication •current calculated temperature •Optional real-time clock or LON network module 25

Industrial Cooling Plants 26

Project Goals � Can we model significant aspects and time constraints? � Can we test in real-time? � Is the tool fast enough? � How do we control and observe target? � Existing product � Documentation � requirements specification � users manuals � equipment and software for real test execution � Meeting and e-mail with Danfoss Engineers � Continued collaboration � Test of new generation controllers being developed � Improved test interface 27

Basic Refrigeration Control highAlarm Limit start start highAlarm alarm Deviation start compressor compressor setpoint +differential differential setpoint stop compressor lowAlarm stop Deviation compressor normal min restart min cooling alarm delay time not elapsed time not elapsed lowAlarm Limit Time 28

EKC Adaptation 1 • Read and write parameter “database” • 47 parameters EKC Software Layering •AK-Online (PC SW) •configuration •supervision Control Software •logging Test Interface Parameter DB (shared variables) Device drivers+kernel Hardware+Physical I/O LON � GW � RS232 win32+OLE+VB 30