BEWARE THE DARK SIDE OF METRICS! Agile Arizona 2019 Barbara - PowerPoint PPT Presentation

BEWARE THE DARK SIDE OF METRICS! • Agile Arizona 2019 • Barbara Anderson Unrestricted Content 11/27/19 1 This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Scrum@Scale Trainer 2

Team Working Agreement Unrestricted Content 11/27/19 3 This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Why do we collect metrics? • Group exercise: • At your tables, take 5 minutes to create a list of agile metrics & why each is useful Unrestricted Content 4 This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Performance Tracking Mechanisms • Burndown Charts • Utilized to predict when all work in an increment will be completed • Remember that story points are relative estimates and therefore burndown will be relative estimate of scope left Burndown Charts Provide Quick Visual of Planned Work Remaining 5 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Performance Tracking Mechanisms • Velocity Charts • Measures amount of work a team can tackle during a single sprint • Velocities trending downwards indicate problems within a team • Continuously improving teams should see upward trends as processes are streamlined and barriers are removed • Average velocity and standard deviation of velocity are useful metrics for predicting amount of scope team can achieve in future sprints Velocity Charts Provide Visual of Team Performance Trends 6 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Sprint Tracking Mechanisms • Using a burnup chart allows ease of visualizing situation where team finishes early and brings in additional work Sprint Burnup shows progress in meeting iteration goals 7 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Example burn-up charts 50 40 45 35 40 30 35 30 25 25 20 20 15 15 10 10 5 5 0 Day 10 Day 10 Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day 8 Day 9 Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day 8 Day 9 0 50 30 25 40 20 30 15 20 10 10 5 0 0 Day 10 Day 10 Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day 8 Day 9 Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day 8 Day 9 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Sprint Tracking Mechanisms • Story and Task Boards • Communicates progress in the current sprint to stakeholders • Focuses team on highest priority work • Promotes transparency and visibility of work • Signs of healthy sprints: • Highest priority items completing early • Limited work in progress • Warning signs during a sprint: • No items completed, high quantity of items in progress Lower priority items completed, higher • priority items not started Story and Task Boards Provide Daily Visual Communication 9 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

How is this team doing? Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations. 1 0

Managing WIP • Cumulative Flow Diagram • Clearly visualizes work in progress (vertical line) • Shows lead time & cycle time (horizontal lines) Cumulative Flow Diagram provides ability to visualize & manage WIP 11 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

What is the context behind this Wells Fargo message? https://welcome.wf.com/renew/ Unrestricted Content 12 This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Metrics and Transparency • Metrics must make people have conversations and to provide frequent and regular feedback. • Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure. • Metrics can be used as part of Kaizen, continuous improvement. • Beware of the “dark side” of Metrics. • The misuse of a metric, often by leadership, is likely to promote the intentional manipulation of the metric, a dysfunctional behavior usually referred as “gaming.” Unrestricted Content 1 This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations. 3

Spurious Correlations https://www.tylervigen.com/spurious-correlations Unrestricted Content 14 This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Spurious Correlations https://www.tylervigen.com/spurious-correlations Unrestricted Content 15 This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Spurious Correlations https://www.tylervigen.com/spurious-correlations Unrestricted Content 16 This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Four Lenses for Measuring Progress Productivity Value Delivery Quality Sustainability Are we Are we Are we Are we working in a improving our prioritizing the meeting our way we can ability to deliver right features to quality continue for product over deliver first? standards? the long run? time? Unrestricted Content 17 This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Discussion: What metrics should we track? Discuss: Brain storm metrics • Leading/Lagging for your table’s topic • Pros/Cons • Side effects • Be ready to present a good metric and a bad metric. Describe why. Productivity Value Delivery Quality Sustainability 18 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Looking at Metrics Together Reveals Insight A B C Velocity Velocity Velocity V (points) (points) (points) Sprint Sprint Sprint Rev $/point Rev Rev $ $/point $/point Quarter Quarter Quarter Happiness Happi- Happi- H rating ness ness Sprint Sprint Sprint Team is driving velocity by Team is burning out to The Team is Conclusion focusing on easy, low-value deliver results working well stories 19 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Program Velocity - Grouped by Value High Value Legend: Medium Value Low Value Unassigned 20 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.

Discussion / Questions? 21 Unrestricted Content This document does not contain technology or technical data controlled under either the U.S. International Traffic in Arms Regulations or the U.S. Export Administration Regulations.