wg1
play

WG1 5/15/2002 9:15:00 AM I DENTIFYING T ESTING P RIORITIES T HROUGH - PDF document

Jan 05, 2024 •189 likes •511 views

P R E S E N T A T I O N BIO PRESENTATION WG1 5/15/2002 9:15:00 AM I DENTIFYING T ESTING P RIORITIES T HROUGH R ISK A NALYSIS Rick Craig Softw are Quality Engineering International Conference On Software Testing Analysis & Review May 13-

  1. P R E S E N T A T I O N BIO PRESENTATION WG1 5/15/2002 9:15:00 AM I DENTIFYING T ESTING P RIORITIES T HROUGH R ISK A NALYSIS Rick Craig Softw are Quality Engineering International Conference On Software Testing Analysis & Review May 13- May 17, 2002 Orlando, FL USA

  2. STQE Publishing STQE Publishing Rick D Craig Rick D. Craig is an experienced test manager, consultant, and lecturer who's helped hundreds of companies throughout Europe, Asia, Australia, and the Americas improve their testing. He's been a featured speaker at testing conferences since 1983, and is the former American editor of Software Quality Management Magazine. Rick is also a colonel in the United States Marine Corps Reserve and a technical editor for the StickyMinds.com Web site.

  3. Identifying Testing Priorities Through Risk Analysis By Rick D. Craig Software Quality Engineering 1

  4. RISK Likelihood Impact RISK – “the chance of injury, damage or loss; A dangerous chance, a hazard 2

  5. • Purpose of Software Risk Analysis – To determine what to test, the testing priority and the depth of testing • Purpose of Planning Risk Analysis – To identify potential (test) schedule disruptions and to formulate and agree upon contingencies 3

  6. Risk Analysis Planning Risks Software Risk Analysis Type of Risk And (Business and Technical) Analysis Contingencies Identify Potential Identify Identify Key Disruptions to Likelihood Impact of Activities Schedule Of failure Failure (Planning Risks) Testing Priorities Results Contingencies 4

  7. Software Risk Analysis Process Overview 1. Form a 5. Assign 8. Prioritize Brainstorming Numerical The Team Values Features 2. Compile a 6. Compute 9. Determine List of The Risk The Features Priority “Cut line” 7. Review/ 3. Determine 10. Consider Modify the The Mitigation Values Likelihood 4. Determine The Impact 5

  8. Process Overview …. Include end-users, developers, testers, Form a Step marketers and business analysts. Brainstorming 1 Team Compile a system-wide list of features Compile a Step and attributes. List of 2 Features 6

  9. List Features of an ATM • Withdraw cash • Deposit cash • Check account balance Features • Transfer funds • Purchase stamps • Make a loan payment • Usability • Performance Attributes • Security 7

  10. Process Overview …. Assign a relative value for Likelihood of Failure. What is the likelihood that this Determine Step feature or attribute will fail to operate The 3 Likelihood correctly? 8

  11. Based on our current knowledge of the system what is the likelihood that this feature or attribute will fail or fail to operate correctly? Likelihood of Failure for ATM Features/Attributes ATM Software Likelihood Features Attributes Withdraw cash High Deposit cash Medium Check account Low Balance Transfer funds Medium Purchase stamps High Make a loan Low payment Usability Medium Performance Low Security Medium 9

  12. Process Overview …. Assign a relative value for the impact. Determine Step What would be the impact on the user if The 4 this feature or attribute failed? Impact 10

  13. What would be the impact on the user if this feature or attribute failed to operate correctly? Impact of Failure for ATM Features/Attributes ATM Software Likelihood Impact Features Attributes Withdraw cash High High Deposit cash Medium High Check account Low Medium Balance Transfer funds Medium Medium Purchase stamps High Low Make a loan Low Medium payment Usability Medium High Performance Low Medium Security Medium High 11

  14. Process Overview …. Assign numerical values corresponding Assign Step to the relative values assigned in steps 3 Numerical 5 and 4 above. Values Add together the values assigned to Compute Step likelihood of failure and impact of failure. The Risk 6 Priority 12

  15. Risk Priority Likelihood Of Failure High (3) Likelihood + Impact = Risk Priority Medium (2) Low (1) Impact Of Low (1) Medium (2) High (3) Failure 13

  16. Summed Priorities for ATM Features/Attributes ATM Software Likelihood Impact Priority Features Attributes Withdraw cash High High 6 Deposit cash Medium High 5 Check account Low Medium 3 Balance Transfer funds Medium Medium 4 Purchase stamps High Low 4 Make a loan Low Medium 3 payment Usability Medium High 5 Performance Low Medium 3 Security Medium High 5 14

  17. Process Overview …. Review/modify priorities based on complexity, Pareto analysis, new or Review/ modified features, development Step Modify the methodology, environmental accessibility, 7 Values usability and team history, etc. 15

  18. Modify Likelihood using “Failure Indicators” • Team history • Complexity • Usability • New or modified features • New techniques or technology • Pareto analysis • Environmental accessibility 16

  19. Process Overview …. Reorganize the list of features and Prioritize Step attributes in order of risk priority. The 8 Features 17

  20. Sorted Priorities for ATM Features/Attributes ATM Software Likelihood Impact Priority Features Attributes Withdraw cash High High 6 Deposit cash Medium High 5 Usability Medium High 5 Security Medium High 5 Transfer funds Medium Medium 4 Purchase stamps High Low 4 Make a loan Low Medium 3 payment Check account Low Medium 3 Balance Performance Low Medium 3 18

  21. Process Overview …. Establish a “cut line” which separates Determine Step features “to be tested” and features “not The 9 to be tested”. “Cut line” 19

  22. “Cut-Line” for ATM Features/Attributes ATM Software Likelihood Impact Priority Features Attributes Withdraw cash High High 6 Deposit cash Medium High 5 Usability Medium High 5 Security Medium High 5 To Be Tested Transfer funds Medium Medium 4 Purchase stamps High Low 4 Make a loan Low Medium 3 payment Not to be tested Check account Low Medium 3 (or tested less) Balance Performance Low Medium 3 20

  23. Process Overview …. Decide which risks (if any) could be Consider Step reduced by adding resources, changing Mitigation 10 the development methodology, etc. 21

  24. Mitigated List of Priorities for ATM Features/Attributes ATM Software Likelihood Impact Priority Mitigation Features Attributes Withdraw cash High High 6 Code Inspection Deposit cash Medium High 5 Early Prototype Usability Medium High 5 Early User Feedback Security Medium High 5 Transfer funds Medium Medium 4 Purchase stamps High Low 4 Make a loan Low Medium 3 payment Check account Low Medium 3 Balance Performance Low Medium 3 22

  25. Planning Risks and Contingencies 23

  26. Planning Risks • Planning risks are unscheduled events or late activities that may jeopardize the testing schedule. • Some common planning risks include: – Unrealistic delivery dates – Scope of testing – Staff availability – Lack of requirements – Budget shortfall – Risk assumptions – Environmental options – Usage assumptions – Tool inventory – Resources – Acquisition schedule – Feature creep – Participant buy-in – Poor quality s/w – Training needs 24

  27. Contingencies • Reduce the scope • Delay implementation • Add resources • Reduce quality processes 25

  28. Sample Planning Risk and Contingencies • Sample Planning Risk – The user introduces a major requirements change late in the software life-cycle. • Sample Contingency #1 – Ask the user group to contribute more users to the testing effort (i.e., add more resources). • Sample Contingency #2 – Decide not to implement a low priority feature until a later release (e.g., reduce the scope). • Sample Contingency #3 – Delay Implementation • Sample Contingency #4 – Decide not to test (or at least to test less) some of the low risk features identified in the course of the software risk analysis (i.e., reduce quality processes). 26

  29. Contingency #4 - Do not test some low risk features, i.e. move the cut line up. “Cut-Line” Adjustments ATM Software Likelihood Impact Priority Features Attributes Withdraw cash High High 6 Deposit cash Medium High 5 Usability Medium High 5 Security Medium High 5 To Be Tested Transfer funds Medium Medium 4 Purchase stamps High Low 4 Make a loan Low Medium 3 Not to be tested payment (or tested less) Check account Low Medium 3 Balance Performance Low Medium 3 27

  30. Bottom Line: • Software risk analysis: – You can’t test everything – Software risk analysis helps determine where to focus the testing • Planning Risk – The best laid plans of mice and men… – Identifying planning risks early allows the entire engineering staff the opportunity to develop and agree on contingencies for the inevitable changes that occur in every project. 28

  31. “If you do not actively attack risks, they will actively attack you Tom Gilb 29

Recommend

More recommend