BIO PRESENTATION T16 Thursday, October 30, 2003 3:00 PM E NSURING R EQUIREMENTS T RACEABILITY IN F UNCTIONAL AND P ERFORMANCE T ESTING Marc Bloom Capital One Financial Corp International Conference On Software Testing Analysis & Review October 27-31, 2003 San Jose, CA USA
Marc S. Bloom Marc S. Bloom heads up End to End Systems Integration Testing for Capital One. He is an experienced IT Project Manager and Quality Assurance professional with expertise designing, implementing, as well as testing enterprise-level network environments. Accomplishments include integrating MVS, Unisys, UNIX, Oracle, and NT solutions for core financial services and E-Commerce systems. Before joining Capital One in 1998, Marc spent 10 years as a certified systems engineering consultant deploying and supporting large-scale network solutions for Fortune 500 corporations. Marc holds a Bachelor’s degree from James Madison University and a Master’s certificate in Project Management from George Washington.
Requirements Traceability in Functional and Performance Testing Marc S. Bloom QA Test Manager marc.bloom@capitalone.com
Agenda Requirements Definition Requirements Traceability Requirements Management Functional vs. Performance Requirements Key Similarities Key Differences 2
Capital One at a Glance � A leading financial services Numerous awards including: company • Top 100 training organization – Training magazine 6th largest credit card � issuer in the U.S. • 20/20 Vision Award – CIO � -- $60.7 billion in magazine managed loans � -- 45.8 million managed • One of the “Best Companies accounts to Work for” in the U.K.-The Sunday Times Located in 8 U.S. cities, � Canada, U.K., France and South Africa • “A top 100 company in Customer Relationship Management” A FORTUNE 500 Company - � • - CIO magazine 3 #191!
Requirements Definition
What are Requirements? “ Business requirements represent high-level objectives of the organization or customer requesting the system or product. They are captured in a document describing the project’s vision and scope.” “ User requirements describe tasks the user must be able to accomplish with the product. These are captured in use cases or scenario descriptions.” “ Functional requirements define the software functionality the developers must build into the product to enable users to accomplish their tasks, thereby satisfying the business requirements. Functional requirements are documented in a software requirements specification (SRS).” 5 - Karl Wiegers 1999. Software Requirements
When are Requirements Defined? By whom? Analysis Phase of the Software Development Life Cycle (SDLC). IT Business Analysts meet with various stakeholders and document needs. All stakeholders verify requirements documentation as correct and unambiguous before Development begins. 6
What are the Sources of Requirements Documentation? Business Rules Business Models System Process Flow Use Cases System Specifications User Interface Specifications Standards – Internal & External 7
What are Some Characteristics of Good Requirements? Be SMART Complete S imple Consistent M easurable Correct A ccurate Unambiguous R easonable Non-compound T estable Feasible Necessary Prioritized 8
What are Testable Requirements? To be most useful in providing a solid basis for a Test Plan, requirements must be verifiable . A "Testable Requirement" is one that can be verified through the execution of a defined test case. 9
Requirements Traceability
Requirements Traceability Defined The degree to which a relationship or link can be established between two or more products of the development process – particularly when having a predecessor-successor or master-subordinate relationship to one another. * IEEE Std 829 11
What are Some Traceability Benefits? Certification of requirements coverage Change impact analysis Maintenance throughout SDLC Project tracking and metrics Re-engineering support Reuse of code in other projects Testing linkage 12
What is a Requirements Traceability Process? The process that ensures Development has designed the right software product in the right way, i.e. that it meets the stated needs of the customer. C U S T O M E R R E Q U I R E M E N T S S 1. Requirements E S A CASES C 2. Use Cases USE T S E 3. Prioritization T 4. Code 5. Test Cases PRIORITY CODE MODULES 13
Requirements Traceability Model Legend: Gray – Artifacts Green – Actors 14 Red - Processes
Requirements Traceability Matrix Key method for associating individual requirements to test cases Dynamic artifact in Requirements Process Test cases can represent single or multiple requirements 15
Requirements Traceability Sample Matrix Use Case Functional Design Code (Lines) Test Case Requirement Element P3 Query COAR AccountLookup FindCOAR P3-001 Acct# Servlet P3 Query Credit AccountLookup Query.code2 P3-002 Acct# Servlet CAA1 Query JAVA Q Query.code2 CAA1-003 Transfer 3 Acct# P3 Return COAR AccountStat Find_stat.44 P3-002 Status Servelet CAA2 Connect to EJB ABCD_CAA1 CAA2-001 PCAAS CAA3 Add CAA23 DBM Add_CAA3 CAA3-001 16
Requirements Management
Requirements Management Defined The purpose of Requirements Management is to establish a common understanding and agreement between the customer and the software project of the customer’s requirements that will be addressed by the software. * CMM L2-1 18
Requirements Management Process It's rarely the case that the set of requirements with which a project originally starts is the same set of requirements that the final product fulfills. The Requirements Management Process insures the accurate traceability of individual requirements throughout the SDLC. 19
Requirements Management Process (cont.) Formal Change Control and Configuration Management are essential to documenting and tracking system or application requirements. A Change Control Board, with a defined CCB process, should approve all changes, additions or modifications. 20
Downstream Impact of Requirements Changes Architecture Design Code Development Use Cases User Documentation & Manuals Help Screens/Error Messages Test Cases 21
Functional vs. Performance Testing Requirements
Stakeholder Identification Ask your team the question, “Who are we testing for?” The number of answers received may surprise you. Understanding the differences and overlaps in the beginning of the process insures that testing is properly targeted to each stakeholder’s needs. 23
Stakeholders (cont.) Functional: Performance: Business Customer Business Customer Marketing IT Platform Management Operations Systems Engineering Customer Relations Capacity Planning IT Development IT Infrastructure Mgt. Internal Audit Future Technologies R&D Internal Audit 24
Focus of Functional Requirements Positive-based Process Flow Alternate or Exception Flows Data Integrity Verification GUI Interface Validation Information Security Risk Mitigation Legal and Compliance Verification 25
Focus of Performance Requirements Increased Priority on IT-defined Requirements Positive-based Process Flow Excludes Boundary and Exception Tests System Monitoring Resource Utilization Platform Capacity/Scalability Infrastructure Stability 26
Key to Breaking out Performance Requirements Knowing the Demographics of the users What are their high percentage activities? How many total users in the system at any given time? Concurrent Load Tests What are the peaks during the day? Elasticity (Spike/Bounce) Tests What are acceptable response times? Stress and Endurance Tests 27
Summary Requirements drive the entire testing process. Use the Traceability Process to provide the necessary linkage to your Testing Process. Testing Tools provide excellent support for Requirements Management and Traceability. Ensure that you know your stakeholders to properly define the Functional and Performance testing needs. 28
Bibliography Rodger Drabick, “On-Track Requirements ”, STQE Magazine, 1999 Johanna Rothman, Brian Lawrence, “ Testing in the Dark ”, STQE Magazine, 1999 Paul Gerrard, “Testing Requirements”, EuroSTAR ’94 Bill and Carol Councill, “Automating Requirements Traceability”, STQE Magazine, 2000 Karl E. Wiegers, “Requirements When the Field Isn’t Green”, STQE Magazine, 2001 Software Requirements, Second Edition, Karl E. Wiegers, 2003. 29
Questions? 30
Recommend
More recommend