Real Tim e TRON TRON TRON TRON Testing Testing using UPPAAL W ith W ith Mariius Mikucionis, Brian Nielsen, Arne Skou, Anders Hessel, Paul Pettersson, Jacob I llum Rasm ussen
Overview Introduction gi knolog Off-line Test Generation Controllable Timed Automata ll bl d CLASSI C CLASSI C CLASSI C CLASSI C CLASSI C CLASSI C CLASSI C CLASSI C CORA CORA CORA CORA CORA CORA CORA CORA onstek Observable Timed Automata TI GA TI GA TI GA TI GA rmatio On-line Test Generation TRON TRON TRON TRON Infor Conclusion and Future Work Summer School on Informatics Kim G Larsen 2 RIO 2012
Testing Primary validation technique used in industry • In general avg. 10-20 errors per 1000 LOC gi • 30 50 % f d 30-50 % of development time and cost in embedded software l t ti d t i b dd d ft knolog To find errors To determine risk of release Part of system development life-cycle Pa t of s stem de elopment life c cle onstek Output p rmatio System Environ- Input Under Test m ent Infor Expensive, error prone, time consuming (for Real-Time Systems) UPPAAL model can be used to generate test specifications Summer School on Informatics Kim G Larsen 3 RIO 2012
Real-tim e Model-Based Testing Plant Controller Program g gi knolog Continuous Discrete sensors onstek actuators Conform s-to? rmatio 1 2 Test generation a inputs 1 2 (offline or 3 3 4 4 Infor online) wrt. b c 3 4 Design Model 1 2 a outputs 1 2 a 3 3 4 b b c c b c 3 4 UPPAAL Model Summer School on Informatics Kim G Larsen 4 RIO 2012
Off-Line Test Generation Controllable Tim ed Autom ata
Model Based Conform ance Testing Model Test suite pass gi Test Test knolog DBLclick! DBLclick! Test execution Test Gene- x>=2 click? tool Generator x:=0 rator Event tool tool fail a tool onstek click? mapping x<2 Driver Selection & rmatio optimization ti i ti Infor Implementation Relation Does the behavior of the ( blackbox ) ( ) implementation comply to that of the specification? Summer School on Informatics Kim G Larsen 6 RIO 2012
Controllable Tim ed Autom ata I nput Enabled : Assumption about all inputs can always be accepted. p y p model of SUT model of SUT gi knolog Output Urgent : enabled outputs will occur immediately enabled outputs will occur immediately. onstek Determ inism : two transitions with same input/output leads to the two transitions with same input/output leads to the rmatio same state. I solated Outputs : I l t d O t t Infor if an output is enabled, no other output is enabled. Summer School on Informatics Kim G Larsen 7 RIO 2012
Test Generation using Verification using Verification gi knolog myGearControl.xml System model System model onstek Uppaal Model- Trace Trace Checker (witness) (witness) rmatio Test purpose Test purpose Some Property Property Random Shortest E<> Gear Gear5 E<> Gear.Gear5 Infor Fastest testGear5.trc Use trace scenario as test case??!! Kim G Larsen Summer School on Informatics RIO 2012 8
Exam ple Light Controller gi knolog onstek rmatio Infor Summer School on Informatics Kim G Larsen 9 RIO 2012
Test Purposes Test Purpose: A specific test objective (or observation) the tester wants to make on SUT gi knolog onstek rmatio TP : Check that the light can become bright: E<> L==10 Infor out(IGrasp);silence(500);in(OSetLevel,0);silence(1000); in(OSetLevel,1);silence(1000);in(OSetLevel,2); silence(1000); in(OSetLevel,3);silence(1000);in(OSetLevel,4);silence(1000); in(OSetLevel 5);silence(1000);in(OSetLevel 6);silence(1000); in(OSetLevel,5);silence(1000);in(OSetLevel,6);silence(1000); in(OSetLevel,7);silence(1000);in(OSetLevel,8);silence(1000); in(OSetLevel,9);silence(1000);in(OSetLevel,10); out(IRelease); Kim G Larsen Summer School on Informatics RIO 2012 10
Coverage Based Test Generation Multi purpose testing u t pu pose test g gi knolog Cover measurement Examples: Examples: onstek Location coverage, Edge coverage, g g , rmatio Definition/use pair coverage Infor Kim G Larsen Summer School on Informatics RIO 2012 11
Coverage Based Test Generation Multi purpose testing u t pu pose test g gi knolog Cover measurement Examples: Examples: onstek Location coverage, Edge coverage, g g , rmatio Definition/use pair coverage Infor Kim G Larsen Summer School on Informatics RIO 2012 12
Coverage Based Test Generation Multi purpose testing u t pu pose test g gi knolog Cover measurement Examples: Examples: onstek Location coverage, Edge coverage, g g , rmatio Definition/use pair coverage Infor Kim G Larsen Summer School on Informatics RIO 2012 13
Coverage Based Test Generation Multi purpose testing u t pu pose test g gi knolog Cover measurement Examples: Examples: onstek Location coverage, Edge coverage, g g , rmatio Definition/use pair coverage Infor Kim G Larsen Summer School on Informatics RIO 2012 14
Edge Coverage Test sequence traversing all edges Encoding: Encoding: gi knolog Enumerate edges e 0 ,…,e n onstek Add auxiliary variable e[i] for each edge Label each edge Label each edge rmatio e[i]:=1 Infor Check: E<>( e[0]=1 E<>( e[0] 1 … e[n] 1 ) e[n]=1 ) Summer School on Informatics RIO 2012 15
Fastest Edge Coverage Time=12600 ms gi knolog onstek rmatio //13 out(IGrasp); //@900 // Bring dimmer PassiveDn->ActiveDN-> silence(500);//hold // ActiveUP+increase to level 10 out(IGrasp); //touch:switch light on silence(1000); in(OSetLevel,1); silence(200); silence(1000); in(OSetLevel,2); out(IRelease); ( ); silence(1000); in(OSetLevel,3); ( ); ( , ); Infor in(OSetLevel,0); silence(1000); in(OSetLevel,4); silence(1000); in(OSetLevel,5); out(IGrasp); //@200 // touch: switch light off silence(1000); in(OSetLevel,6); silence(200); silence(1000); in(OSetLevel,7); out(IRelease);//touch silence(1000); in(OSetLevel,8); in(OSetLevel,0); silence(1000); in(OSetLevel,9); silence(1000); in(OSetLevel,10 // //9 silence(1000); in(OSetLevel,9); //bring dimm State to ActiveDN out(IGrasp); //@400 //Bring dimmer from ActiveUp silence(500); //hold //To Passive DN (level=0) out(IRelease); //check release->grasp is ignored in(OSetLevel,0); out(IGrasp); //@12400 out(IRelease); out(IRelease); Page 1 Page 2 Kim G Larsen silence(dfTolerance); Summer School on Informatics RIO 2012 16
Pow er-Optim al Edge Coverage Cost=320 J gi knolog 1 W onstek 5 · L W 5 · L W 5 L W 5 L W 1 W rmatio //13 out(IGrasp); //@900 // Bring dimmer PassiveDn->ActiveDN-> silence(500);//hold // ActiveUP+increase to level 10 out(IGrasp); //touch:switch light on silence(1000); in(OSetLevel,1); silence(200); silence(1000); in(OSetLevel,2); out(IRelease); ( ); silence(1000); in(OSetLevel,3); ( ); ( , ); Infor in(OSetLevel,0); silence(1000); in(OSetLevel,4); silence(1000); in(OSetLevel,5); out(IGrasp); //@200 // touch: switch light off silence(1000); in(OSetLevel,6); silence(200); silence(1000); in(OSetLevel,7); out(IRelease);//touch silence(1000); in(OSetLevel,8); in(OSetLevel,0); silence(1000); in(OSetLevel,9); silence(1000); in(OSetLevel,10 // //9 silence(1000); in(OSetLevel,9); //bring dimm State to ActiveDN out(IGrasp); //@400 //Bring dimmer from ActiveUp silence(500); //hold //To Passive DN (level=0) out(IRelease); //check release->grasp is ignored in(OSetLevel,0); out(IGrasp); //@12400 out(IRelease); out(IRelease); Page 1 Page 2 Kim G Larsen silence(dfTolerance); Summer School on Informatics RIO 2012 17
V-PLUS: Model-based GUI Testing for Automatic or Manual Execution Automatic or Manual Execution Requirement spec.: gi knolog Create Models Requirement 2.1 Requirement 2.1 When the user presses the button Requirement 2.1 save, all user data is saved to the When the user presses the button Requirement 2.1 database. The values stored in the save, all user data is saved to the When the user presses the button database are output to the screen onstek database The values stored in the database. The values stored in the save, all user data is saved to the save all user data is saved to the When the user presses the button When the user presses the button database are output to the screen save, all user data is saved to the database. The values stored in the database are output to the screen database. The values stored in the database are output to the screen Generate Covering Test Suite: rmatio Output: Output: Infor Quality TestDrive Center/ Excel Excel Gold Gold QTP Kim G Larsen Summer School on Informatics RIO 2012 18
Test Generation from UML Statecharts UML Di UML Diagrams gi knolog onstek Custom Scripts Uppaal Models rmatio Infor Kim G Larsen Summer School on Informatics RIO 2012 19
An I ndustrial Tool Chain… gi knolog onstek rmatio Infor Summer School on Informatics Kim G Larsen 20 RIO 2012
Recommend
More recommend
