Learning objectives Understand the role of quality is the - PowerPoint PPT Presentation

Learning objectives • Understand the role of quality is the development process Test and Analysis Activities • Build an overall picture of the quality process within a Software Process • Identify the main characteristics of a quality process – visibility – anticipation of activities – feedback (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 1 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 2 Software Qualities and Process The Quality Process • Qualities cannot be added after development • Quality process: set of activities and – Quality results from a set of inter-dependent activities responsibilities – Analysis and testing are crucial but far from sufficient. – focused primarily on ensuring adequate • Testing is not a phase, but a lifestyle dependability – Testing and analysis activities occur from early in requirements – concerned with project schedule or with product engineering through delivery and subsequent evolution. usability – Quality depends on every part of the software process • The quality process provides a framework for • An essential feature of software processes is that software test and analysis is thoroughly integrated and – selecting and arranging activities not an afterthought – considering interactions and trade-offs with other important goals. (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 3 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 4

Interactions and tradeoffs Properties of the Quality Process example • Completeness : Appropriate activities are planned to detect each important class of high dependability vs. time to market faults. • Mass market products: – better to achieve a reasonably high degree of dependability on • Timeliness : Faults are detected at a point of a tight schedule than to achieve ultra-high dependability on a high leverage (as early as possible) much longer schedule • Critical medical devices: • Cost-effectiveness : Activities are chosen – better to achieve ultra-high dependability on a much longer depending on cost and effectiveness schedule than a reasonably high degree of dependability on a – cost must be considered over the whole tight schedule development cycle and product life – the dominant factor is usually the cost of repeating an activity through many change cycles. (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 5 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 6 Planning and Monitoring Process Visibility • A process is visible to the extent that one can answer • The quality process the question – Balances several activities across the whole – How does our progress compare to our plan? development process – Example: Are we on schedule? How far ahead or behind? – Selects and arranges them to be as cost-effective as • The quality process has not achieved adequate visibility possible if one cannot gain strong confidence in the quality of the software system before it reaches final testing – Improves early visibility – quality activities are usually placed as early as possible • Quality goals can be achieved only through • design test cases at the earliest opportunity (not ``just in time'') careful planning • uses analysis techniques on software artifacts produced before actual code. • Planning is integral to the quality process – motivates the use of “proxy” measures • Ex: the number of faults in design or code is not a true measure of reliability, but we may count faults discovered in design inspections as an early indicator of potential quality problems (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 7 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 8

A&T Strategy A&T Plan • A comprehensive description of the quality process that • Identifies company- or project-wide standards includes: that must be satisfied – objectives and scope of A&T activities – documents and other items that must be available – procedures required, e.g., for obtaining quality – items to be tested certificates – features to be tested and not to be tested – techniques and tools that must be used – analysis and test activities – staff involved in A&T – documents that must be produced – constraints – pass and fail criteria – schedule – deliverables – hardware and software requirements – risks and contingencies (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 9 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 10 Quality Goals Dependability Qualities • Process qualities (visibility,....) • Correctness: – A program is correct if it is consistent with its specification • Product qualities • seldom practical for non-trivial systems • Reliability: – internal qualities (maintainability,....) – likelihood of correct function for some ``unit'' of behavior – external qualities • relative to a specification and usage profile • usefulness qualities: • statistical approximation to correctness (100% reliable = correct) – usability, performance, security, portability, • Safety: interoperability – preventing hazards • dependability • Robustness – correctness, reliability, safety, robustness – acceptable (degraded) behavior under extreme conditions (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 11 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 12

Example of Dependability Qualities Relation among Dependability Qualites 12 11 1 reliable but robust but not 10 2 • Correctness, reliability: 9 3 not correct: safe: catastrophic 8 4 let traffic pass according 7 5 6 failures failures can occur to correct pattern and occur rarely Robust Reliable central scheduling • Robustness, safety: Correct Safe Provide degraded function when possible; never signal conflicting correct but safe but not greens. not safe or correct: robust: the • Blinking red / blinking annoying specification yellow is better than no failures can is inadequate lights; no lights is better occur than conflicting greens (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 13 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 14 Analysis Inspection • can be applied to essentially any document • analysis includes – requirements statements – manual inspection techniques – architectural and detailed design documents – test plans and test cases – automated analyses – program source code • can be applied at any development stage • may also have secondary benefits – spreading good practices • particularly well suited at the early stages of – instilling shared standards of quality. specifications an design • takes a considerable amount of time • re-inspecting a changed component can be expensive • used primarily – where other techniques are inapplicable – where other techniques do not provide sufficient coverage (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 15 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 16

Automatic Static Analysis Testing • More limited in applicability • Executed late in development – can be applied to some formal representations of • Start as early as possible requirements models • Early test generation has several advantages – not to natural language documents – Tests generated independently from code, when the • are selected when available specifications are fresh in the mind of analysts – substituting machine cycles for human effort makes – The generation of test cases may highlight them particularly cost-effective. inconsistencies and incompleteness of the corresponding specifications – tests may be used as compendium of the specifications by the programmers (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 17 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 18 Improving the Process Organizational factors • Long lasting errors are common • Different teams for development and quality? • It is important to structure the process for – separate development and quality teams is common in large organizations – Identifying the most critical persistent faults – indistinguishable roles is postulated by some – tracking them to frequent errors methodologies (extreme programming) – adjusting the development and quality processes to • Different roles for development and quality? eliminate errors – test designer is a specific role in many organizations • Feedback mechanisms are the main ingredient – mobility of people and roles by rotating engineers of the quality process for identifying and over development and testing tasks among different removing errors projects is a possible option (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 19 (c) 2007 Mauro Pezzè & Michal Young Ch 4, slide 20