Today’s Lecture INF 111 / CSE 121: Software Tools and Methods � More on Testing ● Static Analysis ◘ Code Walkthroughs / Inspections ◘ Code Walkthroughs / Inspections ● Formal Verification Lecture Notes for Fall Quarter, 2007 ● Dynamic Testing Michele Rousseau Lecture Notes 4 - Testing Lecture Notes 4 4 Previous Lecture Typical Testing Process � Continue with XP � No Silver Bullet Expected Subset of Oracle � Testing Output Input Test Results Input Program / Test Compare Spec Strategy Actual Subset of Program / Output Input Spec Lecture Notes 4 2 Lecture Notes 4 5 Different Levels of Testing Quiz #1 Today � Write in Pen if you want it to be � System Testing ● Defined at Requirements -> Run after integration regraded testing � Integration Testing ● Defined at Design -> Run after Unit Testing g g � Unit Testing ● Defined at Implementation -> Run after Implementation of each unit � Regression Testing (testing after Change) ● Defined throughout the process -> Run after modifcations Lecture Notes 4 3 Lecture Notes 4 6 1
V-Model of Development & Testing Motivation (the big picture) � People are not perfect Develop Requirements Execute System Tests ● We make errors in design and code Requirements Review ● Goal of testing: given some code, uncover as Develop Acceptance Tests many errors are possible Acceptance Test Review � Important and expensive activity I t t d i ti it Design Execute Integration Tests Design Review ● Not unusual to spend 30-40% of total project effort on testing Develop Integration Tests Integration Tests Review Code Execute Unit Tests Code Review Lecture Notes 4 7 Develop Unit Tests Unit Tests Review Lecture Notes 4 10 Software Testing The Purpose of Testing � Exercising a system [component] ● on some predetermined input data ● capturing the behavior and output data Design and coding are creative. but… ● comparing with test oracle � Testing is Destructive ● for the purposes of ● The primary goal is to “break” the software ◘ identifying inconsistencies ◘ verifying consistency between actual results and ◘ verifying consistency between actual results and � Very often the same person does � Very often the same person does specification both coding and testing • to provide confidence in consistency with requirements and ● This is not ideal… why? measurable qualities ● Need “split personality”: • to demonstrate subjective qualities ◘ when you start testing, become paranoid and ◘ validating against user needs malicious � Limitations ● Surprisingly hard to do: people don’t like finding out that they made mistakes ● only as good as the test data selected ● subject to capabilities of test oracle Lecture Notes 4 8 Lecture Notes 4 11 Goals of Testing Static Analysis � Examine & analyze source code � Reveal failures/faults/errors � Goal: � Locate failures/faults/errors ● Discovering anomalies and defects � Show system correctness � May be used before implementation � Improve confidence that the system ● Execution is not Required p performs as specified (verification) p ( ) � May be applied to any representation of May be applied to any representation of � Improve confidence that the system the system performs as desired (validation) ● Requirements � Desired Qualities: ● Design ● Accurate ● Test data, etc… ● Complete / thorough ● Repeatable ● Systematic Lecture Notes 4 9 Lecture Notes 4 12 2
Static Analysis Inspection Process � Very effective technique for discovering errors � They reuse domain and programming knowledge Planning ● reviewers are likely to have seen the types of error Overview that commonly arise that commonly arise Individual Prep � Examples: ● Code Reviews & Inspection ● Inspections Rework Re-Inspect Lecture Notes 4 13 Lecture Notes 4 16 Pre-Inspection Stages Code Reviews (“Walk-throughs”) � Planning � Developer presents the code to a small group of ● Select the team colleagues ● Developer describes software ● Organize when and where ● Developer describes how it works ● Ensure code and spec are complete ◘ “Walks through the code” ● Free form commentary/questioning by colleagues ● Free-form commentary/questioning by colleagues � Overview � Overview ● Present general description of the material � Benefits to be inspected ● Many eyes, many minds ● Effective � Individual preparation ● Each member inspects the code and the � Drawbacks spec ● Can lead to problems between developer and colleagues Lecture Notes 4 14 Lecture Notes 4 17 Program Inspection Inspections � Should be short � Small Team � Exclusively focused on defects, ● Author (Programmer) anomalies, & non-compliance with ◘ Silent observer standards ◘ Knows the code too well – might introduce bias ● Reader � Should not recommend changes or ◘ Presents the code ese s e code suggest corrections t ti ◘ May have 1 or 2 ● Tester � Paraphrase code � a few lines at a ◘ Reviews the code “Testing point of view” time ◘ May have 1 or 2 ● Moderator ● Express meaning at a higher level of ◘ Conducts the inspection abstraction ◘ Motivates other participants ◘ Not directly involved with the product being inspected � Code is analyzed using a checklist ◘ Keeps the team focused and together Lecture Notes 4 15 Lecture Notes 4 18 3
Code Checklist Length of Inspection � Wrong use of data � Can cover up to 500 statements per ● Variables not initialized hour ● Array index out of bounds ● Depending on experience of team ● Dangling pointers ● Usually more like 125/hr � Faults in declaration / use of variables ● Duplicate use of variable names � Should not go for more than 2 hours � Faults in computations � Should be done frequently ● Div by 0 ● Type mismatch of variables Lecture Notes 4 19 Lecture Notes 4 22 Code Checklist (2) Inspections � Faults in relational expressions � Cons: ● Incorrect operator use (> instead of >) ● Can be too shallow ● Programmers can be defensive � Faults in Control Flow ● Infinite loops ◘ Evaluations of the programmer should not be ● Off by 1 errors O by e o s determined by reviews determined by reviews ● Team may have insufficient knowledge of the � Faults in Interfaces domain ● Incorrect number of parameters ● Passing the wrong type ● Inconsistent use of global variables Lecture Notes 4 20 Lecture Notes 4 23 Inspections and Testing Rework & Re-inspection � Inspections and testing are � Rework complementary and not opposing verification techniques ● Author corrects code � Both should be used during the V & V process � Re-inspection � Inspections can check conformance with � Inspections can check conformance with ● Can be done by team or moderator a specification ● Can either check for new problems that may ● Can’t check conformance with the customer’s have arisen real requirements ● Can verify errors were corrected Cannot validate dynamic behaviour ● � Inspections cannot check non-functional characteristics such as performance, usability, etc. Lecture Notes 4 21 Lecture Notes 4 24 4
Today’s Lecture Tools for Static Analysis � Scan source text & detect possible faults / anomalies � More on Testing ● Look for possible erroneous situations such as: ● Static Analysis ◘ Unused variables ● Formal Verification ● Formal Verification ◘ Undeclared variables ◘ Undeclared variables ◘ Unreachable code ● Coverage-Based Testing ◘ Variables used before initialization ◘ Parameter type mismatches ◘ Parameter number mismatches ◘ Uncalled functions or procedures ◘ Non-usage of function results ◘ Possible array bound violations ◘ Misuse of pointers Lecture Notes 4 25 Lecture Notes 4 28 Verification & Validation (revisited) Take a break! � Verification � Stretch, Relax “ Are we building the product right?” (Boehm) � Get some water, Use the restroom ● The Software should conform to its specification � Get to know your classmates… ● testing, reviews, walk-throughs, inspections � Etc….. ● internal consistency; consistency with previous step step When we return… � Validation � No Silver Bullet “Are we building the right product?” ● The software should do what the user really � Testing requires ● ascertaining software meets customer’s intent Lecture Notes 3 26 Lecture Notes 4 29 Before the Break Quality Assurance : 5 Problems � Testing #1 : Eliciting the Customer’s Intent ● Static Analysis ● Getting the Specs to meet the “real needs” ◘ Code Walkthroughs ◘ Inspections #2 : QA is inherently difficult ● Systems can be complex making QA difficult to perform ◘ Air Traffic Control � stringent performance ◘ Medical Diagnosis System � Complex processing Lecture Notes 4 27 Lecture Notes 4 30 5
Recommend
More recommend
