BIO PRESENTATION W12 September 29, 2004 3:00 PM A M ANAGER ' S G UIDE TO G ETTING THE M OST O UT OF T ESTING AND QA Brian Warren Ceridian Corporation Better Software Conference & EXPO September 27-30, 2004 San Jose, CA USA
Brian Warren Brian Warren is a QA Manager in Ceridian Corporation’s Human Resource Solutions division managing testing of benefits management tools and HR information systems. He has eleven years of experience in hardware and software testing and development, working in Dell’s Enterprise Systems Group, IBM’s Netfinity Group, a dot-com startup, and a niche software company that developed heads-down data entry products prior to joining Ceridian in 2003. In addition to his test management experience, Brian has worked in a wide range of development, testing, and IT roles, having served as test methodologist for Dell's Server Engineering organization. He also managed a software development and tools team responsible for test automation, load testing, and functional test tools, and worked as a database and web developer.
A Manager’s Guide to Getting the Most Out of Testing and Quality Assurance Brian Warren, Ceridian Corp Better Software 2004
Audience Poll: Test and QA Organizations � Which of the organizations below do you have in your company? � Quality Assurance � Quality Control � Testing � Verification & Validation � More than one of the above � None of the above
Audience Poll: Test and QA Roles � How widely understood are the roles of your Test/QA departments � Role clearly understood at all levels of the company � Role clearly understood within Test/QA and the teams they serve, others not so clear � Test/QA staff understand, others may not � Some Test/QA staff still unclear � Non-issue: No Test/QA to worry about
Understanding the role of Test and QA � Understanding of the role Test/QA play in the corporation impacts… � How much value they are perceived to add (and how their performance is evaluated) � How other teams interact with them � How the Test/QA organizations operate � In the absence of real understanding, the labels used bring their own expectations… � “Quality Police” or intelligence gatherers? � Product experts or process gurus?
Why those expectations matter � Consider what happens then the following groups are working off incorrect assumptions � Executive Management � Control how much is invested in Test/QA � Evaluate performance of departments � Other teams (e.g.: development, marketing) � Supply critical information to Test/QA � Primary customers of Test/QA outputs � Individuals within the organization � Perform the actual work � Customers � Final consumers of product and providers of corporate revenue � Job-seekers � Provide talent pool to support organizational growth
So what SHOULD it be called? � It depends on what the team is doing and what it is trying to accomplish… � There is no single “industry standard” answer � IEEE, CMMI, RUP, and others each have their own perspective
Views of testing and quality assurance � IEEE covers all the bases with two definitions for “software quality assurance”… � (1) “a planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements � (2) a set of activities designed to evaluate the process by which products are developed or manufactured”
Views of testing and quality assurance � … and definitions for “testing” and “verification and validation” � Testing: “the process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component.” (IEEE, 1990) � [Verification and Validation] determines whether development products of a given activity conform to the requirements of that activity, and whether the software satisfies its intended use and user needs. (IEEE, 1998)
Views of testing and quality assurance � CMMI provides a definition (unsurprisingly) focused on process management… � “The purpose of SQA is to provide management with appropriate visibility into the process being used by the software project and of the projects being built” (Paulk et al, 1995)
Views of testing and quality assurance � While RUP provides separate definitions for “testing” and quality assurance � Test: A discipline in the software-engineering process whose purpose is to integrate and test the system. (RUP 2003) � Quality Assurance: All those planned and systematic actions necessary to provide adequate confidence that a product or service will satisfy given requirements for quality. (RUP 2003)
Where these definitions fail � Successful communication depends on expressing concepts such that the entire audience understands � Technical professionals � Non-technical professionals � Executives and managers � Organizations, their objectives, and their goals must be definable in business terms, not technical jargon
A more practical example… � Johanna Rothman provides two very tangible definitions… � QA staff assess the product and the system that produced the product, to see if there are improvements the project team can make for this project or the next one. � Testers test the product, performing verification activities. They discover where the product works and where the product doesn’t work. They test in whatever ways the product requires, whether that’s for functionality, internationalization, performance, reliability, maintenance, usability, or some other attribute of the system.
When all else fails, draw a picture…
3-Dimensional Model of Testing and QA Reporting Internal Role Advisory Focus Product Process Authoritarian External Based on 2-D model from Christensen and Thayer, 2001
Dimensions of Testing and QA: Focus Product Production Process Focused Focused Focused Testing Quality Quality Control Assurance
Characteristics of Organizational Focus � A question of “how much” you can do, not “which” you should do � All options are important and each contributes to the overall ability to deliver the right quality to your customers � The demand for support in each of these areas varies based on business context � Solutions are not one-size-fits-all � Each focus represents a different set of problems and may require different skill sets and structures
Skill Requirements by Focus Skill Set Process-focused Production-focused Product-focused Technical • To the degree needed to • Production tools, • Product architectures, Knowledge review work products and infrastructure, and design, implementation, and Analytical determine process operations and usage Skills adherence or deviation • Production monitoring and • Test design and planning • Process characterization, testing tools and techniques techniques development, and • Statistical measurement • Test (and automation) tools measurement and trending analysis • Issue characterization, and • Issue characterization and troubleshooting skills troubleshooting skills Requirements • System design lifecycle • Manufacturing or production • Understanding of customer and development processes and operations and market demands and and Business Analysis Skills processes product usage • Regulatory and compliance policies Communicatio • Presentation of process • Presentation of technical • Presentation of technical ns Skills adherence and and issue data to technical and issue data to technical compliance data to audiences and business audiences and business primarily non-technical owners owners audiences (management)
Dimensions of Testing and QA: Role Advisory Authoritative Makes Makes decisions recommendations (gatekeeper)
Advisory Role Characteristics � Advisory Pro’s � Non-adversarial approach fosters cooperation and communication � Places ownership of release decisions with those best suited to handle them -- the business decision makers � Advisory Con’s � Can’t stop the business from making a “bad” decision should it decide to do so � Success is heavily reliant upon the advocacy and data presentation skills of Test/QA staff
Authoritative Role Characteristics � Authoritative Pro’s � Ability to halt a project prevents Test/QA from being overrun by stronger personalities from other organizations � Authoritative Con’s � Sets up an adversarial relationship that can stifle communication, cooperation, and trust � Quality of release decisions dependent upon Test/QA organization’s understanding of the business – not just the product � Test/QA organization can now be blamed regardless of the decision made
A quote from the experts… � “Some testers dream of having veto control over the release of the product. They deserve to be punished by having their wish granted.” Kaner, Bach, and Pettichord in Lessons Learned in Software Testing
Recommend
More recommend