T6 Concurrent Session Thursday 10/25/2007 11:15 AM JUMP TO: Biographical Information The Presentation Related Paper A “Framework for Testing” for A “Framework for Testing” for Repeatable Success Repeatable Success Presented by: Randy Slade, Kaiser Permanente HMO Presented at: The International Conference on Software Testing Analysis and Review October 22-26, 2007; Anaheim, CA, USA 330 Corporate Way, Suite 300 , Orange Park, FL 32043 888-268-8770 � 904-278-0524 � sqeinfo@sqe.com � www.sqe.com
Randy Slade Randy Slade Randy Slade and Viktoriya Kozlova are Test Managers at Kaiser Permanente HMO. Together they led the development and implementation of the “Framework For Testing” for their testing groups. They jointly developed this presentation and white paper. Viktoriya graduated from Kiev National University and was previously a teacher. She has 15 years experience in Quality Assurance and Testing, with 9 years at Kaiser and prior to that was with IBM. Viktoriya describes herself as pro-active, not re-active. She believes in: managing chaos; tracking deliverables; and using repeatable processes. Randy has 30 years experience in software development, Quality Assurance, and Testing across various industries. He is a graduate of California State University at Chico, and attended Duke University Fuqua Graduate School’s Management Development Program. Randy has been a Test Lead, Test Manager, Testing and Quality Assurance Consultant at various companies including: Pacific Bell, BellCore Communications; Kaiser Permanente; California State Automobile Association; Pacific Gas & Electric; Wells Fargo Bank; and Patotech. Randy is an advocate for continuous improvement, repeatable processes, building quality into a product from the beginning, and implementing lessons learned.
A “Framework for Testing” for Repeatable Success by: Randy Slade Viktoriya Kozlova From Kaiser Permanente HMO 08.07.2007
The History � Testing Team of 60 to 125 resources � Resource pool management � Delivery of 60 – 80 projects per year � Deployments of in-house developed, off-shelf stand alone, and hybrid integrated applications � Collaborative effort with applications development and business team � Collaborative effort with applications vendors
The Challenge � No standards or repeatable processes � Incoherent approach and inconsistent results � Communication gaps � Dependencies on individual expertise, knowledge, and heroic efforts � Manage workload � Manage peaks and valleys
Desired Goals � Develop clearly defined, standardized, repeatable processes � Identify set of critical deliverables for each testing phase � Develop standardized, re-usable templates � Define guidelines to manage resource pool and projects workload � Develop guideline to maintain project historical data
Benefits_Processes � Implementation of “best practices” on all projects, regardless of resource staffing � Achievement of predictable and manageable results including historical data collection � Development of Subject Matter Expertise for applications and utilization of Cross Knowledge practices for all projects � Leverage new improvements across all projects using unified approach
Benefits_Resource Management � Develop pool staffing model of highly professional testing resources with proficiency in testing systems/components/areas � Flexibility with resources allocation assignments and re-assignments to the projects with ability to sustain commitments continuity � Leverage workload with re-usability of testing artifacts � Motivate resources professional and leadership development � Support individual initiatives for processes improvements
Approach_Initiation � Identify key stakeholders from impacted areas to get multiple perspectives based on different experiences � Clearly define goals and benefits � Formulate desired outcome statement � Obtain overall group commitment � Form a Core Team
Approach_Core Team � Review existing processes for common phases (engagement, planning, preparation, execution, deployment) � Conduct gap analysis and identify missing processes � Prioritize based on work impact/criticality/effectiveness � Define process development/improvement guidelines � Assign Lead for each sub-team for process development � Develop overall workplan
Approach_Process Development/Improvement Sub-Team � Form a Sub-Team � Schedule an introduction meeting for Sub-Team with clearly define agenda � Outline process development tasks and milestones � Develop implementation strategy and delivery timeline � Develop process with designated Sub-Team � Send for review to other Sub-Teams � Collect and incorporate feedback
Approach_Process Development/Improvement Sub-Team � Identify testing artifact(s) that should support new/improved process implementation � Assign sub-team resource for artifact(s) template development � Conduct sub-team internal and Core Team external artifact(s) template reviews. Incorporate feedback � Pilot new/improved process/artifact(s) template on selected projects � Obtain approval from the Core Team on new/improved process and/or artifact(s) template � Publish and implement new/improved process and/or artifact(s) template
Implementation Requirements � Process should be documented � Process might have a supporting presentation and/or supporting workflow � Process might have a supporting artifact(s) template � Process should be piloted � Process and/or artifact(s) should be approved for implementation
Process Workflow_ Example (Overview)
How To Read A Process � Start at the top � Each numbered box has an associated detailed sub-process � Bracket on a side with blue colored text indicates pertinent template location for the designated box � Bracket on a side with black colored text indicates pertinent document for the designated box � Step by step road map from the beginning to the end of the specific process
Sub-Process Workflow_ Example
Framework For Testing � “Framework For Testing” is an outcome of six month Process Development/Improvement effort. � Implementation of this framework has resulted in high quality processes, templates, tools and presentations that support the implementation of a standardized testing methodology � Testing deliverables were met and have exceeded customer satisfaction. This framework supported timely deployments within the defined budget � This methodology worked effectively in the current and planned information technology infrastructure
Methodology Workbook � Consolidate new/improved processes organized by five phases of a software testing life-cycle � Guided resources through the “what, when, how and why” steps � Develop Entrance and Exit Criteria for each phase as a check point � Provide step by step workflows for each process and sub- process � Provide narratives for each workflow and reference for supporting templates
Testing Life Cycle Testing Life Cycle consists of five phases: � Engagement � Planning � Preparation � Execution � End of Engagement
Engagement Phase Collections of: � Workflows – Overall Phase Workflow, Sub-flows for each sub- process � Presentations – Testing Services Introduction � Templates – Engagement Form Charge Agreement Testing WorkPlan Project Status Report � Tools – Early Engagement Estimation Time Reporting Tracking
Planning Phase Collections of: � Workflows – Overall Phase Workflow, Sub-flows for each sub- process � Presentations – Readiness Assessment For Testing (RAFT) Offshore Readiness Assessment for Testing (ORAFT) � Templates – RAFT ORAFT Requirements Review Log Test Plan Test Coverage & Traceability Matrix
Planning Phase � Tools – RAFT Score Calculation and Recommendations ORAFT Score Calculation and Recommendations Requirements Upload into Quality Center Repository Risk Reporting Lessons Leaned Data Collection
Preparation Phase Collections of: � Workflows – Overall Phase Workflow, Sub-flows for each sub- process � Presentations – Cross Knowledge � Templates – Test Cases Peer Review Business & Development Review Cross Knowledge � Tools – Test Cases Upload into Quality Center Repository
Execution Phase Collections of: � Workflows – Overall Phase Workflow, Sub-flows for each sub- process � Templates – Execution Status Report Defect Status Report Consolidated Status Report Test Summary Report � Tools – Test Readiness
End Of Engagement Phase Collections of: � Workflows – Overall Phase Workflow, Sub-flows for each sub- process � Presentations – Lessons Learned � Templates – Lessons Leaned End of Engagement Plan
Standards Outside of Testing Life Cycle � Standard Interview Approach � New Employees Testing Orientation � Project Transition � Resource Transition � Projects & Resources Allocation Tracking � Cross Area Resources Allocation
Key Leaning � Continues and consistent processes maintenance – you are never done � Constant Improvement can be driven across the organization with common processes � The best way to an organization is to develop and implement unified processes, tools and templates
Recommend
More recommend
