Project Takeaway submitted. Next submission: Software Testing - PDF document

Project • Takeaway – submitted. • Next submission: Software Testing – Nim – Oct 26 th • Remember: okay to change framework – Make sure all games work with framework • Memory checks – Your program should do proper memory management! • Questions Announcement Announcement • Exam 2 • Final exam – October 28 th (next Tuesday) – Tuesday, November 18 th – Will cover: – 12:30pm – 2:30pm • Inheritance – Rm 70-3690 • Memory Management / Problems • STL • Lots to cover…will schedule a review. The Plan Before we begin • Today: Testing • Questions • Tomorrow: More Testing??? • Thursday: Exam 2 Review

Software Testing Software testing • From the software testing FAQ • Some definitions: – TESTING means "quality control" – Error – Improper action of a programmer – QUALITY CONTROL measures the quality – Fault – The result of an error (improper logic). of a product – Failure – Improper action of an executing – QUALITY ASSURANCE measures the program due to a fault. quality of processes used to create a quality product. • No such thing as bug-free code! Software testing Software Development Cycle • Gather Requirements • Programmer writes: – Find out what the user needs char *foo = 0; • System Analysis strcpy (foo, “I smell pointer problems”); – Express these needs formally in system terms – This is an error • Design • The fault is that strcpy accesses a null pointer. – Design a high level solution – Many faults in C++ are pointer problems • Implementation – Bad logic are problems too – Turn solution into code • Testing • The failure: – Verify that the solution works Segmentation fault (Core dumped) • Maintenance – Iterate the cycle Testing Strategies Software Testing • When to test 1. All tests should be traceable to customer requirements. – Incrementally during implementation phase 2. Tests should be planned long before testing begins. • Assure each unit or class meets design and functional specs 3. 80% of errors are traceable to 20% of the modules (Pareto – Limited testing of overall system during Principle) implementation 4. Testing should begin in the small and progress to larger – Formal system test during testing phase (after components. implementation is complete) 5. Exhaustive testing is NOT possible. • Alpha / Beta Testing 6. Testing is more effective when conducted by an • Tests program requirements independent party.

Software Testing Software Testing • Levels of testing • Types of Testing – Unit testing – Formal Verification • individual classes • Reduce program to logical assertions and “prove” mathematically that the program is correct – Integration Testing – Empirical Testing • Assembly of one or more classes • Generate Test cases to see where errors exist. – System Test • Most testing that you will do will be empirical • System as a whole Software Testing Programming by Contract • Introduced by Bertrand Meyer, the creator of • Empirical Testing Eiffel. – White Box testing • Creates a contract between the software developer • Assumes access to the code and software user • Test all program flows • Covers all statements and conditions – Every feature, or method, starts with a precondition that must be satisfied by the consumer of the routine. – Black Box Testing – each feature ends with postconditions which the • Assumes no access to code or knowledge of implementation supplier guarantees to be true (if and only if the • Test cases generated based on requirements preconditions were met). • Test valid and invalid input – each class has an invariant which must be satisfied after • Follow the contract any changes to the object represented by the class. Programming by Contract Assertions SomeClass::someFunction ( AnotherClass *fillMeWithData • Debugging mechanism to test condition at ) any point in the code { // check any preconditions here – If condition is false, the program aborts and preCondition ( fillMeWithData ); dumps core. // non-NULL check // do your stuff to add the functionality here ... – Useful for testing preconditions, postconditions // check post conditions postCondition ( fillMeWithData->hasData() ); // did and invariant checks. we do what we said postCondition ( checkInvariant() ); // class invariant check required }

Assertions Assertions // constructor #include <cassert> // // Preconditions: // last & first are not empty (emptyString) void foo (int *p) // age is not negative { // Person::Person( string last, string first, int age, string // At this point p should not be null firstJobName ): lastName(last), firstName(first), currentAge(age), currentJob(0) assert (p != 0); { assert( last != emptyString ); … assert( first != emptyString ); } assert( age >= 0 ); if ( firstJobName != noJob ) { currentJob = new Job( firstJobName ); } } Questions? Unit Testing • A unit test tests at a “unit” (in C++, class) level. • Why test classes individually? – Limit the scope of testing – Easier to generate test cases – Bugs found earlier (before integration) are easier to fix. Unit Testing Unit Testing • Black box approach • White Box Approach – Must rely on functional specs and contracts – You have the code, look directly at execution, – Supply inputs, check outputs. • Use debugger to set data member values • Use debugger to get data member values • Call methods with well chosen parameters • Use debugger to check flow of execution • Call methods in various orders • Must test all execution paths • Check object state via access methods.

Unit Testing – White Box Unit Testing – White Box • Testing Loops • Loop Testing – Nested Loops – Simple loops (for I=0; I < n; I++) • 1. Start with the innermost loop. Set all other loops to • 1. Skip the loop entirely. minimum values. • 2. Only one pass through the loop. • 2. Conduct simple loop tests for the innermost loop while holding the outer loops at their minimum iteration values. • 3. Two passes through the loop. • 3. Work outward, conducting tests for the next loop, but • 4. m passes through the loop where m<n. keeping all other outer loops at this minimum iteration count. • 5. n-1, n, n+1 passes through the loop • 4. Continue until all loop have been tested. Unit Testing Unit Testing • About writing test cases • Consider all paths – Testing for success • Method gives expected results on good input – Testing for failure • Methods should fail on bad input – Test the contract! – values within ranges • Test boundaries and within range Unit Testing Equivalence Classes • Testing all possible stimuli sequences is • About writing test cases (black box) impossible – A test case should be able to... • ...run completely by itself, without any human input. Unit • Choose “groups” of stimuli that you guess testing is about automation. will cause the same reaction • ...determine by itself whether the function it is testing has passed or failed, without a human interpreting the results. – Based on your knowledge of the code • ...run in isolation, separate from any other test cases (even if • These groups are equivalence classes. they test the same functions). Each test case is an island. – This is what try does for lab submissions! • Choose only one test from each class. • Still a large number of tests to be run

Equivalence classes Unit Testing • Two inputs are in the same Equivalence Class if they are handled • Questions? similarly by system – eg. data field valid value in 1-50 – So, 20, 38, 1, 47 belong to the same Equivalence Class – no need to test multiple values from same Equivalent Class – Bounds testing • eg. test 38, then end points 1 and 50 – test valid and invalid equivalence classes – reduces the number of test cases required Example: 3 inputs I 1 has 10 equivalence classes Total tests cases required: I 2 has 10 equivalence classes 10 x 10 x 10 = 1000 test cases. I 3 has 10 equivalence classes Summary • Software Testing • Error / Fault / Failure • Level of Testing • White Box vs. Black Box • Unit Testing • Questions