Dev "Programming" Ops for DevOps Success Damon Edwards - PowerPoint PPT Presentation

If you want freedom, take responsibility New Way Old Way • Developers own their application • Developers owned feature requirements • Operations owns infrastructure and common tooling • Operations owned performance and uptime • QA owned quality • Security owned security • etc...

If you want freedom, take responsibility New Way Old Way • Developers own their application • Developers owned feature requirements • Operations owns infrastructure and common tooling • Operations owned performance and uptime • Everybody owns quality, availability, security for the thing • QA owned quality they produce and shared • Security owned security responsibility for things that • etc... consume it

If you want freedom, take responsibility New Way Old Way • Developers own their application • Developers owned feature requirements • Operations owns infrastructure and common tooling • Operations owned performance and uptime • Everybody owns quality, availability, security for the thing • QA owned quality they produce and shared • Security owned security responsibility for things that • etc... consume it

Dev initiated DevOps Transformation 1. Take an “operations first” mindset 2. Build organizational alignment

What is organizational alignment?

What is organizational alignment? • 1000’s of small decisions made daily • How do we harness that?

What is organizational alignment? • 1000’s of small decisions made daily • How do we harness that? • Top down is brittle • Conditions, people, tools are always changing... how do we handle that?

What is organizational alignment? • 1000’s of small decisions made daily • How do we harness that? • Top down is brittle • Conditions, people, tools are always changing... how do we handle that? • Alignment is when you know that different individuals would independently see a set of conditions and arrive at the same decision that is correct for the company’s goal

What is organizational alignment? • 1000’s of small decisions made daily • How do we harness that? • Top down is brittle • Conditions, people, tools are always changing... how do we handle that? • Alignment is when you know that different individuals would independently see a set of conditions and arrive at the same decision that is correct for the company’s goal

What does an aligned organization “see”?

What does an aligned organization “see”? expanded from

What does an aligned organization “see”? 1. See the system expanded from

What does an aligned organization “see”? 1. See the system expanded from 2. Focus on flow

What does an aligned organization “see”? 1. See the system expanded from 2. Focus on flow 3. Recognize feedback loops

What does an aligned organization “see”? 1. See the system expanded from 2. Focus on flow 3. Recognize feedback loops 4. Look for continuous improvement opportunities

Sure... but how do you do that?

Building organizational alignment

Building organizational alignment 1. Socialize the concepts and vocabulary

Building organizational alignment 1. Socialize the concepts and vocabulary 2. Visualize the system

Building organizational alignment 1. Socialize the concepts and vocabulary 2. Visualize the system � a. value stream mapping

Building organizational alignment 1. Socialize the concepts and vocabulary 2. Visualize the system � a. value stream mapping � b. timeline analysis

Building organizational alignment 1. Socialize the concepts and vocabulary 2. Visualize the system � a. value stream mapping � b. timeline analysis � c. waste analysis

Value Stream Mapping Core business service delivery process Current state value stream map Steering Committee Erica Switzer Full L/T Lead time Projects / P/T Process time Expedites H/C Head count S/R Scrap rate Business Analyst Erica Switzer D Defects Estimates EP Extra processes and requirements M Motion PD Partially done Project Manager TS Task switching W Waiting Erica Switzer Business Change Creative request Change Daily Scrum Program Management Control Forms meeting Request QA Mx3 build Development Technical Dx3 Daily Scrum Develop Deployment Specification QA web assets meeting Meeting Business Systems Jira issues RT tickets Release Engineering Support L/T 6 weeks Web Assets, Deployment P/T 4 weeks Technical Operations flows and Document Jira issues Stan Walters H/C 12 wireframes S/R 5% Dx2 Wx2 Wx2 Bob McNulty PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Integration testing Build and deploy fixes Set of EP service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours Cut project branch W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D Mx2 D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 L/T 15 minutes Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD EP P/T 5 minutes Deploy to Merge project Tested Acceptance testing Feature testing Deploy to Staging Staging testing H/C 1 Production branch packages S/R 0% L/T 5 days L/T 1 week L/T 4 weeks L/T 6 days L/T 9 hours L/T 30 minutes Chris Holmes Development P/T 5 hours P/T 3.5 days P/T 2.5 weeks P/T 90 minutes P/T 8 hours P/T 30 minutes Environments H/C 2 H/C 4 H/C 4 H/C 3 H/C 6 H/C 1 Documented S/R 10% S/R 66% S/R 20% S/R 66% S/R 66% S/R 33% test plans Raj Lee Frances Middleton Raj Lee Frances Middleton Allen Cannata Fernando Gomez and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environment L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Deploy to Performance Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment Process Time = 62 days 4 weeks 4 weeks 4 hours 1 day 7 hours 5 hours 2.5 weeks 90 minutes 3.5 days 8 hours Lead Time = 2 weeks 4 weeks 1.5 days 1 hours 4.5 days 1.5 weeks 5.75 days 1.5 days 1 hour 113 days

Value Stream Mapping Core business service delivery process Current state value stream map Steering Committee Erica Switzer Full L/T Lead time Projects / P/T Process time Expedites H/C Head count S/R Scrap rate Business Analyst Erica Switzer D Defects Estimates EP Extra processes and requirements M Motion PD Partially done Project Manager TS Task switching W Waiting Erica Switzer Business Change Creative request Change Daily Scrum Program Management Control Forms meeting Request QA Mx3 build Development Technical Dx3 Daily Scrum Develop Deployment Specification QA web assets meeting Meeting Business Systems Jira issues RT tickets Release Engineering Support L/T 6 weeks Web Assets, Deployment P/T 4 weeks Technical Operations flows and Document Jira issues Stan Walters H/C 12 wireframes S/R 5% Dx2 Wx2 Wx2 Bob McNulty PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Integration testing Build and deploy fixes Set of EP service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours Cut project branch W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D Mx2 D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 L/T 15 minutes Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD EP P/T 5 minutes Deploy to Merge project Tested Acceptance testing Feature testing Deploy to Staging Staging testing H/C 1 Production branch packages S/R 0% L/T 5 days L/T 1 week L/T 4 weeks L/T 6 days L/T 9 hours L/T 30 minutes Chris Holmes Development P/T 5 hours P/T 3.5 days P/T 2.5 weeks P/T 90 minutes P/T 8 hours P/T 30 minutes Environments H/C 2 H/C 4 H/C 4 H/C 3 H/C 6 H/C 1 Documented S/R 10% S/R 66% S/R 20% S/R 66% S/R 66% S/R 33% test plans Raj Lee Frances Middleton Raj Lee Frances Middleton Allen Cannata Fernando Gomez and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environment L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Deploy to Performance Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment Process Time = 62 days 4 weeks 4 weeks 4 hours 1 day 7 hours 5 hours 2.5 weeks 90 minutes 3.5 days 8 hours Lead Time = 2 weeks 4 weeks 1.5 days 1 hours 4.5 days 1.5 weeks 5.75 days 1.5 days 1 hour 113 days

Value Stream Mapping M Service pack review Current state value stream map email Customer L/T = 56d communication L/T Lead time P/T = 7d H/C = 6 P/T Process time L/T = 28d S/R = 100% P/T = 7d H/C Head count Suresh Wu H/C = 1 crit bugs S/R Scrap rate S/R = Customer M(2) W Stephen / Xi derived Release Program D Defects reqs. Management Release EP Extra processes M W Schedule L/T = M Motion Estimates PRD Product Program P/T = PD Partially done Planning H/C = S/R = TS Task switching Erica Smith L/T = 105d W Waiting P/T = 46d H/C = 15 PD S/R = 100% Lockdown Technical Support M(2) EP John Robert control Product Management Engineering Engineering Planning Cloud Services Design Process Specs PRD PRD L/T = 45d EP(2) TS(2) Documentum P/T = 18d H/C = 23 M EP M(3) W(2) PD S/R = ERR Preliminary Bob Smith Cloud Services M(3) W Development Release BRD PRD Release ERR Server L/T = 60d L/T = 45d Memos Provisioning P/T = 16d P/T = 21d MOP, SOP H/C = 3 H/C = 140 QA D L/T = 24d S/R = 3% S/R = Environment P/T = 4d Remedy Reggie / Carlos Bob Smith M(3) EP QA Forum H/C = 3 Ticket Ticket S/R = 50% QA forum Engineering BRD Jen Garza checklist Release EP Patch M W L/T = 60d Calendar P/T = 1d Change Control EP D README H/C = 1 Server QA Forum Rollout S/R = >5% M EP(3) L/T = 42d Acceptance Ticket Schedule Victoria Doe P/T = Tasks QA Test H/C = L/T = 14d S/R = P/T = 1d L/T = 105d Peter Lee MOP H/C = 4.5 MOP ERR P/T = 11d S/R = 15% PD(3) H/C = 42 Lynn A. etc Selective S/R = Sam Young Promotion L/T = 90d Remedy P/T = 15d New Targets Ticket H/C = 5 S/R = Steve Young BTS README Deploy Release M W(2) EP Full M(2) PD(3) Development Shared Build Commits L/T = 90d Release Drive Prod Packages P/T = 8d Shared Drive Promotion L/T = 75d H/C = 8 L/T = 1d Test P/T = 43d S/R = 2% P/T = 0.3d H/C = 130 L/T = 60d Lewis S./Peter Y. H/C = 2 S/R = P/T = 0.2d XML Production S/R = 33% Bob Smith H/C = 1 M W(2) EP John Doe S/R = >5% Single Victoria Doe Image PD D(3) TS Server

Value Stream Mapping M Service pack review Current state value stream map email Customer L/T = 56d communication L/T Lead time P/T = 7d H/C = 6 P/T Process time L/T = 28d S/R = 100% P/T = 7d H/C Head count Suresh Wu H/C = 1 crit bugs S/R Scrap rate S/R = Customer M(2) W Stephen / Xi derived Release Program D Defects reqs. Management Release EP Extra processes M W Schedule L/T = M Motion Estimates PRD Product Program P/T = PD Partially done Planning H/C = S/R = TS Task switching Erica Smith L/T = 105d W Waiting P/T = 46d H/C = 15 PD S/R = 100% Lockdown Technical Support M(2) EP John Robert control Product Management Engineering Engineering Planning Cloud Services Design Process Specs PRD PRD L/T = 45d EP(2) TS(2) Documentum P/T = 18d H/C = 23 M EP M(3) W(2) PD S/R = ERR Preliminary Bob Smith Cloud Services M(3) W Development Release BRD PRD Release ERR Server L/T = 60d L/T = 45d Memos Provisioning P/T = 16d P/T = 21d MOP, SOP H/C = 3 H/C = 140 QA D L/T = 24d S/R = 3% S/R = Environment P/T = 4d Remedy Reggie / Carlos Bob Smith M(3) EP QA Forum H/C = 3 Ticket Ticket S/R = 50% QA forum Engineering BRD Jen Garza checklist Release EP Patch M W L/T = 60d Calendar P/T = 1d Change Control EP D README H/C = 1 Server QA Forum Rollout S/R = >5% M EP(3) L/T = 42d Acceptance Ticket Schedule Victoria Doe P/T = Tasks QA Test H/C = L/T = 14d S/R = P/T = 1d L/T = 105d Peter Lee MOP H/C = 4.5 MOP ERR P/T = 11d S/R = 15% PD(3) H/C = 42 Lynn A. etc Selective S/R = Sam Young Promotion L/T = 90d Remedy P/T = 15d New Targets Ticket H/C = 5 S/R = Steve Young BTS README Deploy Release M W(2) EP Full M(2) PD(3) Development Shared Build Commits L/T = 90d Release Drive Prod Packages P/T = 8d Shared Drive Promotion L/T = 75d H/C = 8 L/T = 1d Test P/T = 43d S/R = 2% P/T = 0.3d H/C = 130 L/T = 60d Lewis S./Peter Y. H/C = 2 S/R = P/T = 0.2d XML Production S/R = 33% Bob Smith H/C = 1 M W(2) EP John Doe S/R = >5% Single Victoria Doe Image PD D(3) TS Server

Timeline Analysis Core business service delivery process Current state value stream map Steering Committee Erica Switzer Full L/T Lead time Projects / P/T Process time Expedites H/C Head count S/R Scrap rate Business Analyst Erica Switzer D Defects Estimates EP Extra processes and requirements M Motion PD Partially done Project Manager TS Task switching W Waiting Erica Switzer Business Change Creative request Change Daily Scrum Program Management Control Forms meeting Request QA Mx3 build Development Technical Dx3 Daily Scrum Develop Deployment Specification QA web assets meeting Meeting Business Systems Jira issues RT tickets Release Engineering Support L/T 6 weeks Web Assets, Deployment P/T 4 weeks Technical Operations flows and Document Jira issues Stan Walters H/C 12 wireframes S/R 5% Dx2 Wx2 Wx2 Bob McNulty PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Integration testing Build and deploy fixes Set of EP service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours Cut project branch W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D Mx2 D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 L/T 15 minutes Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD EP P/T 5 minutes Deploy to Merge project Tested Acceptance testing Feature testing Deploy to Staging Staging testing H/C 1 Production branch packages S/R 0% L/T 5 days L/T 1 week L/T 4 weeks L/T 6 days L/T 9 hours L/T 30 minutes Chris Holmes Development P/T 5 hours P/T 3.5 days P/T 2.5 weeks P/T 90 minutes P/T 8 hours P/T 30 minutes Environments H/C 2 H/C 4 H/C 4 H/C 3 H/C 6 H/C 1 Documented S/R 10% S/R 66% S/R 20% S/R 66% S/R 66% S/R 33% test plans Raj Lee Frances Middleton Raj Lee Frances Middleton Allen Cannata Fernando Gomez and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environment L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Deploy to Performance Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment Process Time = 62 days 4 weeks 4 weeks 4 hours 1 day 7 hours 5 hours 2.5 weeks 90 minutes 3.5 days 8 hours Lead Time = 2 weeks 4 weeks 1.5 days 1 hours 4.5 days 1.5 weeks 5.75 days 1.5 days 1 hour 113 days

Waste Analysis Core business service delivery process Current state value stream map Steering Committee Erica Switzer Full L/T Lead time Projects / P/T Process time Expedites H/C Head count S/R Scrap rate Business Analyst Erica Switzer D Defects Estimates EP Extra processes and requirements M Motion PD Partially done Project Manager TS Task switching W Waiting Erica Switzer Business Change Creative request Change Daily Scrum Program Management Control Forms meeting Request QA Mx3 build Development Technical Dx3 Daily Scrum Develop Deployment Specification QA web assets meeting Meeting Business Systems Jira issues RT tickets Release Engineering Support L/T 6 weeks Web Assets, Deployment P/T 4 weeks Technical Operations flows and Document Jira issues Stan Walters H/C 12 wireframes S/R 5% Dx2 Wx2 Wx2 Bob McNulty PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Integration testing Build and deploy fixes Set of EP service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours Cut project branch W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D Mx2 D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 L/T 15 minutes Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD EP P/T 5 minutes Deploy to Merge project Tested Acceptance testing Feature testing Deploy to Staging Staging testing H/C 1 Production branch packages S/R 0% L/T 5 days L/T 1 week L/T 4 weeks L/T 6 days L/T 9 hours L/T 30 minutes Chris Holmes Development P/T 5 hours P/T 3.5 days P/T 2.5 weeks P/T 90 minutes P/T 8 hours P/T 30 minutes Environments H/C 2 H/C 4 H/C 4 H/C 3 H/C 6 H/C 1 Documented S/R 10% S/R 66% S/R 20% S/R 66% S/R 66% S/R 33% test plans Raj Lee Frances Middleton Raj Lee Frances Middleton Allen Cannata Fernando Gomez and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environment L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Deploy to Performance Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment Process Time = 62 days 4 weeks 4 weeks 4 hours 1 day 7 hours 5 hours 2.5 weeks 90 minutes 3.5 days 8 hours Lead Time = 2 weeks 4 weeks 1.5 days 1 hours 4.5 days 1.5 weeks 5.75 days 1.5 days 1 hour 113 days

Waste Analysis (Mary Poppendick’s “Seven Wastes of Software Development”)

What should we be looking for? Core business service delivery process Current state value stream ma Steering Committee Erica Switzer Full Projects / Expedites Business Analyst Erica Switzer Estimates and requirements Project Manager Erica Switzer Change request Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee Erica Switzer Full Projects / Expedites Business Analyst Erica Switzer Estimates and requirements Project Manager Erica Switzer Change request Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites Business Analyst Erica Switzer Estimates and requirements Project Manager Erica Switzer Change request Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites • Large batch sizes Business Analyst Erica Switzer Estimates and requirements Project Manager Erica Switzer Change request Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites • Large batch sizes Business Analyst Erica Switzer • Human to human information transfer Estimates and requirements Project Manager Erica Switzer Change request Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites • Large batch sizes Business Analyst Erica Switzer • Human to human information transfer Estimates and requirements Project Manager • Scrap / Rework / “Backwash” Erica Switzer Change request Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites • Large batch sizes Business Analyst Erica Switzer • Human to human information transfer Estimates and requirements Project Manager • Scrap / Rework / “Backwash” Erica Switzer Change request • Process or tooling inconsistency Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites • Large batch sizes Business Analyst Erica Switzer • Human to human information transfer Estimates and requirements Project Manager • Scrap / Rework / “Backwash” Erica Switzer Change request • Process or tooling inconsistency Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support • Manual deployment / configuration eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites • Large batch sizes Business Analyst Erica Switzer • Human to human information transfer Estimates and requirements Project Manager • Scrap / Rework / “Backwash” Erica Switzer Change request • Process or tooling inconsistency Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support • Manual deployment / configuration eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy • Reliance on manual testing / verification Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites • Large batch sizes Business Analyst Erica Switzer • Human to human information transfer Estimates and requirements Project Manager • Scrap / Rework / “Backwash” Erica Switzer Change request • Process or tooling inconsistency Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support • Manual deployment / configuration eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy • Reliance on manual testing / verification Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD • Injecting requirements late in the process Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites • Large batch sizes Business Analyst Erica Switzer • Human to human information transfer Estimates and requirements Project Manager • Scrap / Rework / “Backwash” Erica Switzer Change request • Process or tooling inconsistency Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support • Manual deployment / configuration eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy • Reliance on manual testing / verification Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD • Injecting requirements late in the process Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% • Fixing in place (vs rebuilding) test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment

What should we be looking for? • Long lead times (vs processing time) Core business service delivery process Current state value stream ma Steering Committee • Request and approval queues Erica Switzer Full Projects / Expedites • Large batch sizes Business Analyst Erica Switzer • Human to human information transfer Estimates and requirements Project Manager • Scrap / Rework / “Backwash” Erica Switzer Change request • Process or tooling inconsistency Change Daily Scrum Control Forms Request QA meeting build Technical Daily Scrum Deployment Specification meeting Meeting Business Systems Jira issues RT tickets Support • Manual deployment / configuration eb Assets, Deployment flows and Document Stan Walters Jira issues ireframes Dx2 Wx2 Wx2 PDx4 Change TSx5 RT tickets D Control Delta Development Development Release Development features and Build and deploy • Reliance on manual testing / verification Build and deploy Integration testing fixes Set of service L/T 8 weeks L/T 2 days L/T 1 day L/T 8 hours branches P/T 4 weeks P/T 4 hours P/T 1 day P/T 7 hours W H/C 5 H/C 1 H/C 5 H/C 1 D PDx3 D D S/R 25% S/R 10% S/R 95% S/R 2% Mx8 Dx2 Joe Harbaugh Joe Harbaugh Joe Harbaugh Fernando Gomez EP PD • Injecting requirements late in the process Deploy to Tested Acceptance testing Feature testing Deploy to Staging Staging testing Production packages L/T 5 days L/T 6 days L/T 1 week L/T 9 hours L/T 4 weeks P/T 5 hours P/T 3.5 days Development P/T 2.5 weeks P/T 90 minutes P/T 8 hours H/C 2 H/C 4 Environments H/C 4 H/C 3 H/C 6 Documented S/R 10% S/R 20% S/R 66% S/R 66% S/R 66% • Fixing in place (vs rebuilding) test plans Raj Lee Frances Middleton Frances Middleton Allen Canna Raj Lee and cases PDx2 QA Staging Production Build test plan Build test cases Environments Environment Environmen • “Snowflake” servers and environments L/T 5 days L/T 2 weeks P/T 4 days P/T 8 days H/C 1 H/C 2 S/R 10% S/R 20% Performance Deploy to Raj Lee Raj Lee testing Performance L/T 6 days L/T 1 week P/T 90 minutes P/T 3 days H/C 3 H/C 1 S/R 66% S/R 66% Frances Middleton Allen Cannata Performance Environment