Logistics Project Software Testing Part 3 (block) due Sunday, - PDF document

Logistics • Project Software Testing – Part 3 (block) due Sunday, Oct 30 – Feedback by Monday Logistics Logistics • Project • Final exam – Part 4 (clock variant) due Sunday, Nov 13 th – Good news…bad news • Individual submission – Good news • Recommended: Submit by Nov 6 th – Scoring • Last day of finals, November 18 th • Functionality (10) – Bad news • Use of framework (5) • 8am-10am • Code Style (5) • Memory Management (5) – Room • 01-3338 • Questions? Plan for this week Before we begin • Yesterday: Exam 2 return • Questions? • Today: Software Testing • Thursday: Pointers to functions 1

Software Testing Software testing • From the software testing FAQ • Some definitions: – TESTING means "quality control" – Error – Improper action of a programmer – QUALITY CONTROL measures the quality – Fault – The result of an error (improper logic). of a product – Failure – Improper action of an executing – QUALITY ASSURANCE measures the program due to a fault. quality of processes used to create a quality product. • No such thing as bug-free code! Software testing Software Development Cycle • Gather Requirements • Programmer writes: – Find out what the user needs char *foo = 0; • System Analysis strcpy (foo, “I smell pointer problems”); – Express these needs formally in system terms – This is an error • Design • The fault is that strcpy accesses a null pointer. – Design a high level solution – Many faults in C++ are pointer problems • Implementation – Bad logic are problems too – Turn solution into code • Testing • The failure: – Verify that the solution works Segmentation fault (Core dumped) • Maintenance – Iterate the cycle Testing Strategies Software Testing • When to test 1. All tests should be traceable to customer requirements. – Incrementally during implementation phase 2. Tests should be planned long before testing begins. • Assure each unit or class meets design and functional specs 3. 80% of errors are traceable to 20% of the modules (Pareto – Limited testing of overall system during Principle) implementation 4. Testing should begin in the small and progress to larger – Formal system test during testing phase (after components. implementation is complete) 5. Exhaustive testing is NOT possible. • Alpha / Beta Testing 6. Testing is more effective when conducted by an • Tests program requirements independent party. 2

Software Testing Software Testing • Levels of testing • Types of Testing – Unit testing – Formal Verification • individual classes • Reduce program to logical assertions and “prove” mathematically that the program is correct – Integration Testing – Empirical Testing • Assembly of one or more classes • Generate Test cases to see where errors exist. – System Test • Most testing that you will do will be empirical • System as a whole Software Testing Programming by Contract • Introduced by Bertrand Meyer, the creator of • Empirical Testing Eiffel. – White Box testing • Creates a contract between the software developer • Assumes access to the code and software user • Test all program flows • Covers all statements and conditions – Every feature, or method, starts with a precondition that must be satisfied by the consumer of the routine. – Black Box Testing – each feature ends with postconditions which the • Assumes no access to code or knowledge of implementation supplier guarantees to be true (if and only if the • Test cases generated based on requirements preconditions were met). • Test valid and invalid input – each class has an invariant which must be satisfied after • Follow the contract any changes to the object represented by the class. Programming by Contract Assertions SomeClass::someFunction ( AnotherClass *fillMeWithData • Debugging mechanism to test condition at ) { any point in the code // check any preconditions here preCondition ( fillMeWithData ); – If condition is false, the program aborts and dumps core. // non-NULL check // do your stuff to add the functionality here ... – Useful for testing preconditions, postconditions // check post conditions postCondition ( fillMeWithData->hasData() ); // did and invariant checks. we do what we said postCondition ( checkInvariant() ); // class invariant check required } 3

Assertions Assertions // constructor #include <cassert> // // Preconditions: // last & first are not empty (emptyString) void foo (int *p) // age is not negative // { Person::Person( string last, string first, int age, string // At this point p should not be null firstJobName ): lastName(last), firstName(first), currentAge(age), currentJob(0) assert (p != 0); { assert( last != emptyString ); … assert( first != emptyString ); } assert( age >= 0 ); if ( firstJobName != noJob ) { currentJob = new Job( firstJobName ); } } Questions? Unit Testing • A unit test tests at a “unit” (in C++, class) level. • Why test classes individually? – Limit the scope of testing – Easier to generate test cases – Bugs found earlier (before integration) are easier to fix. Unit Testing Unit Testing • Black box approach • White Box Approach – Must rely on functional specs and contracts – You have the code, look directly at execution, – Supply inputs, check outputs. • Use debugger to set data member values • Use debugger to get data member values • Call methods with well chosen parameters • Use debugger to check flow of execution • Call methods in various orders • Must test all execution paths • Check object state via access methods. 4

Unit Testing – White Box Unit Testing – White Box • Testing Loops • Loop Testing – Nested Loops – Simple loops (for I=0; I < n; I++) • 1. Start with the innermost loop. Set all other loops to • 1. Skip the loop entirely. minimum values. • 2. Only one pass through the loop. • 2. Conduct simple loop tests for the innermost loop while holding the outer loops at their minimum iteration values. • 3. Two passes through the loop. • 3. Work outward, conducting tests for the next loop, but • 4. m passes through the loop where m<n. keeping all other outer loops at this minimum iteration count. • 5. n-1, n, n+1 passes through the loop • 4. Continue until all loop have been tested. Unit Testing Unit Testing • About writing test cases • Consider all paths – Testing for success • Method gives expected results on good input – Testing for failure • Methods should fail on bad input – Test the contract! – values within ranges • Test boundaries and within range Unit Testing Equivalence Classes • Testing all possible stimuli sequences is • About writing test cases (black box) impossible – A test case should be able to... • ...run completely by itself, without any human input. Unit • Choose “groups” of stimuli that you guess testing is about automation. will cause the same reaction • ...determine by itself whether the function it is testing has passed or failed, without a human interpreting the results. – Based on your knowledge of the code • ...run in isolation, separate from any other test cases (even if • These groups are equivalence classes. they test the same functions). Each test case is an island. – This is what try does for lab submissions! • Choose only one test from each class. • Still a large number of tests to be run 5

Equivalence classes Unit Testing • Two inputs are in the same Equivalence Class if they are handled • Questions? similarly by system – eg. data field valid value in 1-50 – So, 20, 38, 1, 47 belong to the same Equivalence Class – no need to test multiple values from same Equivalent Class – Bounds testing • eg. test 38, then end points 1 and 50 – test valid and invalid equivalence classes – reduces the number of test cases required Example: 3 inputs I 1 has 10 equivalence classes Total tests cases required: I 2 has 10 equivalence classes 10 x 10 x 10 = 1000 test cases. I 3 has 10 equivalence classes Summary • Software Testing • Error / Fault / Failure • Level of Testing • White Box vs. Black Box • Unit Testing • Questions 6