i w c t 2 0 1 7 c o m b i n at o r i a l t e s t i n g c t
play

I W C T 2 0 1 7 C O M B I N AT O R I A L T E S T I N G ( C T ) - PowerPoint PPT Presentation

Dec 31, 2023 •8 likes •188 views

U S I N G T I M E D B A S E - C H O I C E C O V E R A G E C R I T E R I O N F O R T E S T I N G I N D U S T R I A L C O N T R O L S O F T WA R E H E N N I N G B E R G S T R M A N D E D U A R D E N O I U I W C T 2 0 1 7 C O M B I N

  1. U S I N G T I M E D B A S E - C H O I C E C O V E R A G E C R I T E R I O N F O R T E S T I N G I N D U S T R I A L C O N T R O L S O F T WA R E H E N N I N G B E R G S T R Ö M A N D E D U A R D E N O I U I W C T 2 0 1 7

  2. C O M B I N AT O R I A L T E S T I N G ( C T ) • CT tries to select test input values: • the test goal is a combination strategy - test criteria. • Test Level: • System: create inputs on user-level interaction • Unit: create inputs for method param. and variables

  3. C O M B I N AT O R I A L T E S T I N G ( C T ) • CT tries to select test input values: • the test goal is a combination strategy - test criteria. • Test Level: T E S T • System: create inputs on user-level interaction C R I T E R I A A P P L I E D T O U N I T S • Unit: create inputs for method param. and variables

  4. P R O G R A M M A B L E L O G I C C O N T R O L L E R S ( P L C ) • Are real-time systems • Found in trains, nuclear power plants, automation • Run on domain-specific operating systems.

  5. I N D U S T R I A L C O N T R O L S O F T WA R E ( U N I T ) P L C W R I T T E N I N I E C 6 1 1 3 1 - 3 G E O R - 5 5 A N D N 1 I T O N O U T L E 5 s 1 2 5 2 I N I N 3 I N 4

  6. T E S T I N G P L C S O F T WA R E E N 5 0 1 2 8 - R A I LWAY I E C 6 2 3 0 4 - M E D I C A L E N 6 2 1 3 8 - N U C L E A R D O - 1 7 8 B - A E R O S PA C E

  7. T E S T I N G P L C S O F T WA R E E N 5 0 1 2 8 - R A I LWAY • Unit (component) level testing • A design is used as expected output (test oracle) • The use of functional testing is mandated • Some level of code coverage is recommended t 0 t 1 t 2 t 3 t 4 t 5 time component Design Test Suite Test Suite Test Result Test Suite Creation Creation Execution Checking Reporting

  8. B A S E C H O I C E C R I T E R I O N AND TON # Tests IN1 IN2 IN3 IN4 1 1 1 5 4 IN1 2 0 1 5 4 OUT1 IN2 5s 3 1 0 5 4 4 1 1 5 3 5 1 1 5 5 SR GT IN3 6 1 1 4 4 OUT2 7 1 1 3 4 IN4 program cycle P=500ms

  9. T I M E D B A S E - C H O I C E C R I T E R I O N # Tests IN1 IN2 IN3 IN4 1. Create the basic input model 1 1 1 5 4 2 0 1 5 4 2. Identify the timing constraint 3 1 0 5 4 4 1 1 5 3 3. Identify a base and time choice test. 5 1 1 5 5 4. Create a test suite. 6 1 1 4 4 7 1 1 3 4 - time choice T = 6s AND TON - (1,1,5,4) is fixed for 6s. IN1 OUT1 IN2 5s - Tests 1 to 7 are fixed for 6s each 
 SR GT IN3 OUT2 IN4

  10. C A S E S T U D Y • Compare timed base-choice with base choice in terms of code coverage and fault detection. I N D U S T R I A L C A S E S T U D Y

  11. C A S E S T U D Y M E T H O D B C T E S T S B R A N C H C O V E R A G E 1 1 P R O G R A M S T B C T E S T S FA U LT R A N D T E S T S D E T E C T I O N

  12. M U TAT I O N A N A LY S I S MUTANT OPERATORS IEC 61131-3 PROGRAM C O M PA R I S O N VA L U E L O G I C A L N E G AT I O N A R I T H M E T I C T I M E R (Yoo et al 2007) (Shin et al. 2012) MUTANTS MUTANTS MUTANTS

  13. M U TAT I O N A N A LY S I S M U TA N T O R I G I N A L M U TA N T M U TA N T P R O G R A M P R O G R A M P R O G R A M P R O G R A M O R I G I N A L F I N D T E S T S W H I C H P R O G R A M W H E N E X E C U T E D O N B O T H O U T P U T / S TAT E I S D I F F E R E N T O R I G I N A L & M U TA N T M U TA N T P R O G R A M M U TA N T M U TA N T P R O G R A M P R O G R A M P R O G R A M

  14. R E S U LT S T B C A C H I E V E S B E T T E R FA U LT D E T E C T I O N S C O R E S T H A N B C O R R A N D . 100 Mutation Score (%) 80 60 40 ● 20 Rand BC TBC

  15. R E S U LT S T B C A C H I E V E S B E T T E R D E C I S I O N C O V E R A G E S C O R E S T H A N B C O R R A N D . 100 Decision Coverage (%) 90 80 70 60 50 Rand BC TBC

  16. R E S U LT S 40 Number of tests 30 20 ● ● 10 0 Rand BC TBC

  17. T H E F U T U R E • multiple base and time choices • evaluate the use of stronger criteria • Use naturally-occurring faults • Use other systems from other domains

  18. U S I N G T I M E D B A S E - C H O I C E C O V E R A G E C R I T E R I O N F O R T E S T I N G I N D U S T R I A L C O N T R O L S O F T WA R E 100 Mutation Score (%) IEC 61131-3 Programs Faulty Programs 80 Timed Base-Choice (TBC) Execution 60 Tests Base-Choice (BC) Framework Random (Rand) 40 ● Coverage Fault Detection 20 Rand BC TBC B C B C T B C T B C C O D E C O V E R A G E FA U LT D E T E C T I O N

Recommend

More recommend