4/9/2018 Topics Defects and Reliability Supplemental Materials: - PowerPoint PPT Presentation

4/9/2018 Topics Defects and Reliability Supplemental Materials: Software Testing Software Defects: are inevitable in a complex Software Testing software system. Test Driven Development – In industry: 10-50 bugs per 1000 lines of code! Black Box Testing – Defects can be obvious or remain hidden. CS2: Data Structures and Algorithms Unit Testing Colorado State University Software Reliability : What is the probability White Box Testing of failure of a software package over time: Coverage Testing Chris Wilcox, Russ Wakefield, Wim Bohm, – Measurements: mean time between failures, crash Dave Matthews Software Debugging statistics, uptime versus downtime. CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – Spring Semester 2018 1 Spring Semester 2018 2 Spring Semester 2018 3 1

4/9/2018 Common faults in algorithms Numerical faults in algorithms Other faults in algorithms Incorrect logical conditions Not using enough bits or digits Poor minimal configuration performance Calculation performed in wrong place Not using enough places before or after the Handling peak loads or missing resources decimal point Non-terminating loop or recursion HW and SW configuration incompatibility Assuming a floating point value will be Incorrect preconditions for an algorithm Crash recovery exactly equal to some other value Not handling null conditions Deadlock, livelock, and critical races Ordering operations poorly so errors build Off-by-one errors Inappropriate resource management up Operator precedence errors CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – Spring Semester 2018 4 Spring Semester 2018 5 Spring Semester 2018 6 2

4/9/2018 Definition Exhaustive Testing? Software Testing Types We consider a program to be correct if it Software Testing is a systematic attempt to – Functional , Configuration, Usability, Reliability, produces the expected output for all inputs . reveal errors in software by running test Performance, Compatibility, Error, Localization, ... Domain of input values can be very large, e.g. programs or scripts (interactively or automated). Processes 2 32 values for an integer or float: – FAILING TEST: an error was demonstrated in the – Test-Driven Development , Coverage Testing , int divide (int operand1, int operand2); Automated Testing, Regression Testing, … software under test. 2 32 * 2 32 = 2 64 , a large number, so we clearly Methods – PASSING TEST: no error was found, at least for this cannot test exhaustively! – Black-box, white-box particular situation. And that is just for one method, in one class, Levels – Theory of testing says you cannot prove absence of in one package, and relatively simple. – Unit (Method), Module (Class), Integration, System all defects in software. CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – Spring Semester 2018 7 Spring Semester 2018 8 Spring Semester 2018 9 3

4/9/2018 Functional Testing Two Kinds of Tests Test Driven Development Goal: Clean code that works! Tests that find defects after they occur Drive development with automated tests – a waste of time – write new code only if tests fail – eliminate duplication Tests that prevent defects Implies a different order of tasks – the only kind to create – Red - write a little test that fails first – Green - make the test work quickly – Refactor - eliminate any resulting duplications Citation: Study of the Toyota Production System, Shigeo Shingo Citation: Test Driven Development, Kent Beck CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – Spring Semester 2018 10 Spring Semester 2018 11 Spring Semester 2018 12 4

4/9/2018 Test Driven Development Program testing methods Black-box Testing Black-box testing Specifications drive test inputs and expected Red outputs – Specifications drive test inputs and expected Write a test outputs Code, design or internal documents that fails – Code, design or internal documents unavailable unavailable White-box testing – Code structure drives test inputs Refactor Green – Specifications used to derive expected outputs – Code, design, and internal documents available Eliminate Make the redundancy code work CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – Spring Semester 2018 13 Spring Semester 2018 14 Spring Semester 2018 15 5

4/9/2018 Equivalence classes Equivalence partition testing Boundary value testing Groups of inputs to be treated similarly Test at least one value of every equivalence Expand equivalence classes to test values at class for each individual input. extremes of each equivalence class. – Numbers: <0, 0, >0 – Numbers: <0, 0..1, >1 Test all combinations where one input is Number ranges: likely to affect the interpretation of another – Months: [- ∞..0], [1..12], [13..∞] – minimum, slightly above minimum, nominal or input. – Months: “Jan”, ”Feb”, ”Mar”, ”Apr”, ”May”, median value, slightly below maximum, and maximum values ”Jun”, ”Jul”, ”Aug”, ”Sep”, ”Oct”, ”Nov”, Test random combinations of equivalence ”Dec”, any other 3 character string. – values slightly and significantly outside the classes. – Years: <0, [0..99], >99 range – Years: <0, [0..9999], >9999 CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – Spring Semester 2018 16 Spring Semester 2018 17 Spring Semester 2018 18 6

4/9/2018 Boundary Value Testing Example Unit Testing JUnit value assertions Test boundaries of the parameter value domain: JUnit is a simple, open source framework to assertTrue( 'a' < 'b' , "message"); write and run repeatable tests. JUnit is assertFalse( 'b' < 'a' ); commonly used in industry for unit testing. Features include: assertEquals( 1+1, 2 ); – Assertions for testing expected results assertEquals( 22.0d/ 7.0d, 3.14159, 0.001 ); – Test fixtures for sharing common test data – Test runners for running tests assertEquals( "cs165" , "cs165" ); Citation: JUnit testing framework (http://www.junit.org/) Citation: JUnit testing framework (http://www.junit.org/) CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – Spring Semester 2018 19 Spring Semester 2018 20 Spring Semester 2018 21 7

4/9/2018 JUnit array assertions JUnit other assertions JUnit test class int[] array1 = { 1, 2, 3 }; import static org.junit.jupiter.api.Assertions.assertEquals; int[] array2 = { 1, 2, 3 }; Stack<Object> stack = new Stack<>(); import org.junit.jupiter.api.Test; assertNull( null ); assertThrows(EmptyStackException.class, class FirstJUnit5Tests { assertNotNull( array1 ); () -> stack.pop()); @Test void myFirstTest() { assertNotSame( array1, array2 ); assertThrows(EmptyStackException.class, assertEquals(2, 1 + 1); () -> stack.peek()); } assertArrayEquals( array1, array2 ); } Citation: JUnit testing framework (http://www.junit.org/) Citation: JUnit testing framework (http://www.junit.org/) Citation: JUnit testing framework (http://www.junit.org/) CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – Spring Semester 2018 22 Spring Semester 2018 23 Spring Semester 2018 24 8

4/9/2018 Test Driven Development RED Green Example Write a test that fails Make it work Red Red Red Write a test Write a test Write a test that fails that fails that fails class Example { class Example { Refactor Green Refactor Green Refactor Green Eliminate Make the Eliminate Make the Eliminate Make the /* Performs requested operations on a value import org.junit.Test; /* Performs requested operations on a value redundancy code work redundancy code work redundancy code work * @param x is an integer input value * @param x is an integer input value import static org.junit.Assert.assertEquals; * @param c1 is a boolean that increments the input value if true * @param c1 is a boolean that increments the input value if true * @param c2 is a boolean that squares the input value if true * @param c2 is a boolean that squares the input value if true /* Test example function*/ * @param c3 is a boolean that negates the input value if true * @param c3 is a boolean that negates the input value if true class TestExample{ * @return the modified input value * @return the modified input value */ */ int xmpl( int x, boolean c1, boolean c2, boolean c3) { int xmpl( int x, boolean c1, boolean c2, boolean c3) { @Test return 0; return x; public void testDoNothing() { } } assertEquals(2, Example.xmpl( 2, false, false, false )); } } } } CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – CS165: Data Structures and Algorithms – Spring Semester 2018 25 Spring Semester 2018 26 Spring Semester 2018 27 9