From Pressman, “Software Engineering – a practitioner ʼ s approach”, Chapter 13 and Pezze + Young, “Software Testing and Analysis”, Chapters 1–4 Testing Strategies Software Engineering Andreas Zeller • Saarland University 1 2 Perspective of Quality quality di fg ers from one person to another. Also, it di fg ers in the customers’ and developers’ perspectives. 3
From Oxford dictionary T esting • Testing: a procedure intended to establish the quality, performance, or reliability of something, esp. before it is taken into widespread use. 4 From Pressman, “Software Engineering – a practitioner ʼ s approach”, Chapter 13 Software T esting • Software testing : the process of exercising a program with the specific intent of finding errors prior to delivery to the end user. 5 Let ʼ s recall the Waterfall model. Waterfall Model (1968) Communication project initiation requirements gathering Planning estimating scheduling tracking Modeling analysis design Construction code test Deployment delivery support feedback 6
In the second half of the course, we Waterfall Model focus on construction and deployment – essentially, all the activities that take place after the code has been written. (1968) Communication project initiation requirements gathering Planning estimating scheduling tracking Modeling analysis design Construction code test Deployment delivery support feedback 7 So, we simply assume our code is done – We built it! ☺ 8 – but is it ready for release? Shall we deploy it? ☹ 9
It ʼ s not like this is the ultimate horror… 10 …but still, this question causes fear, uncertainty and doubt in managers We built it! 11 Therefore, we focus on the Waterfall Model “construction” stage – and more specifically, on the “test” in here. (1968) Construction code test 12
and the question is: how to make your Waterfall Model code ready for deployment. (1968) Construction code test Deployment delivery support feedback 13 These activities are summarized as V&V – verification and validation V&V See Pressman, ch. 13: “Testing Strategies” • Verification: Are we building the product right? Ensuring that software correctly implements a specific function • Validation: Are we building the Ensuring that software has right product? been built according to customer requirements 14 (from Pezze + Young, “Software Testing and Validation and Verification Analysis”) SW Specs Actual System Requirements Validation Verification Includes usability Includes testing, testing, user feedback inspections, static analysis, proofs 15
Verification or validation depends on the spec – this one is unverifiable, but Validation validatable (from Pezze + Young, “Software Testing and Analysis”) • “if a user presses a request button at floor i, an available elevator must arrive at floor i soon” 16 this one is verifiable. Verification • “if a user presses a request button at floor i, an available elevator must arrive at floor i within 30 seconds” 17 When do V&V start? When are they done? Basic Questions • When do V&V start? When are they done? • Which techniques should be applied? • How do we know a product is ready? • How can we control the quality of successive releases? • How can we improve development? 18
Early descriptions of the waterfall Waterfall Model model separated coding and testing into two different activities (1968) Code Test 19 What do these facts have in common? G They ʼ re all wrong! First Code, then T est N O • Developers on software should do R no testing at all • Software should be “tossed over a wall” to W strangers who will test it mercilessly • T esters should get involved with the project only when testing is about to begin 20 Verification and validation activities occur all over the software process (from Pezze + Young, “Software Testing and Analysis”) 21
This is called the “V”-model of “V&V” activities (because of its shape) V&V Activities (from Pezze + Young, “Software Testing and Analysis”) validation Module Test verification 22 This is called the “V”-model of “V&V” activities (because of its shape) Unit T ests (from Pezze + Young, “Software Testing and Analysis”) • Uncover errors at module boundaries • Typically written by programmer herself • Frequently fully automatic ( → regression) 23 From Pressman, “Software Engineering – a practitioner ʼ s approach”, Chapter 13 Stubs and Drivers • A driver exercises a Driver module’s functions • A stub simulates not-yet- ready modules • Frequently realized as mock objects Stub Stub 24
This is called the “V”-model of “V&V” activities (because of its shape) Integration T ests (from Pezze + Young, “Software Testing and Analysis”) • General idea: Constructing software while conducting tests • Options: Big bang vs. incremental construction 25 From Pressman, “Software Engineering – a practitioner ʼ s approach”, Chapter 13 Big Bang • All components are combined in advance • The entire program is tested as a whole • Chaos results • For every failure, the entire program must be taken into account 26 From Pressman, “Software Engineering – a practitioner ʼ s approach”, Chapter 13 T op-Down Integration • Top module is A tested with stubs (and then used as driver) Stub B Stub Stub • Stubs are replaced one at a time (“depth first”) Stub C Stub • As new modules Stub D Stub are integrated, tests are re-run • Allows for early demonstration of capability 27
From Pressman, “Software Engineering – a practitioner ʼ s approach”, Chapter 13 Bottom-Up Integration • Bottom modules implemented first Driver and combined into clusters Driver C F • Drivers are replaced one at a time D E • Removes the need for complex stubs 28 From Pressman, “Software Engineering – a practitioner ʼ s approach”, Chapter 13 Sandwich Integration • Combines A bottom-up and top-down Stub B Stub Stub integration Driver • Top modules tested with stubs, bottom modules C F with drivers D E • Combines the best of the two approaches 29 Evidence: pragmatic – there is no way a test can ever cover all possible paths TETO Principle through a program Test early, test often 30
From Pressman, “Software Engineering – a practitioner ʼ s approach”, Chapter 13 Who T ests the Software? Developer Independent Tester • understands the system • must learn about system • but will test gently • will attempt to break it • driven by delivery • driven by quality 31 A good tester should be creative and destructive – even sadistic in places. The Ideal T ester – Gerald Weinberg, “The psychology of computer programming” 32 The conflict between developers and testers is usually overstated, though. The Developer 33
Let ʼ s simply say that developers should respect testers – and vice versa. The Developers 34 Theory: As humans want to be honest with themselves, developers are Weinberg’s Law blindfolded with respect to their own mistakes. Evidence: “seen again and again in every project” (Endres/Rombach) From Gerald Weinberg, “The psychology of computer programming” A developer is unsuited to test his or her code. 35 Acceptance T esting • Acceptance testing checks whether the contractual requirements are met • Typically incremental ( alpha test at production site, beta test at user’s site) • Work is over when acceptance testing is done 36
Special System T ests • Recovery testing forces the software to fail in a variety of ways and verifies that recovery is properly performed • Security testing verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration • Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume • Performance testing test the run-time performance of software within the context of an integrated system 37 This is called the “V”-model of “V&V” activities (because of its shape) V&V Activities (from Pezze + Young, “Software Testing and Analysis”) validation verification 38 Which techniques should be applied? Basic Questions • When do V&V start? When are they done? • Which techniques should be applied? • How do we know a product is ready? • How can we control the quality of successive releases? • How can we improve development? 39
There is a multitude of activities (dynamic ones execute the software, static ones don ʼ t) – and we ʼ d like them to end when the software is 100% Program correct. Testing Analysis Unfortunately, none of them is perfect. (dynamic verification) (static or dynamic) Inspections Proofs (static verification) (static verification) 40 Why V&V is hard (on software) • Many different quality requirements • Evolving (and deteriorating) structure • Inherent non-linearity • Uneven distribution of faults 41 If an elevator can safely carry a load of 1000 kg, it can also safely carry any Compare smaller load; If a procedure correctly sorts a set of 256 elements, it may fail on a set of 255 or 53 or 12 elements, as well as on 257 or 1023. (from Pezze + Young, “Software Testing and Analysis”) can load 1,000 kg can sort 256 elements 42
Recommend
More recommend
