The Bottom Line on ROI: Measuring the Impact and ROI of Organization Development Programs, Projects, and Initiatives Timothy R. Brock, PhD, CPT, CRP Director of Consulting Services ROI Institute
The BOTTOMLINE on ROI Reaction Objectives The intent is for you to perceive this 90-minute session content as: Relevant to their work Important to their success Immediately applicable 2
The BOTTOMLINE on ROI Learning Objectives After completing this session, you should be able to: Identify the drivers for ROI Make the business case for ROI Explain how business alignment is achieved Identify 7 of the 12 guiding principles Describe the 10 steps in the ROI Methodology 3
The BOTTOMLINE on ROI Application Objectives Within two weeks of this session, you should: Conduct a briefing on ROI. Develop Level 3 and Level 4 objectives for a specific program. 4
The BOTTOMLINE on ROI There is greater demand to show the value of investments in OD programs, projects, & initiatives at my organization than in the past. A. Yes B. No C. I don’t know 5
The BOTTOMLINE on ROI What is Value? Exercise: Think about a project or program that you recently implemented. How would you define the value delivered by the program? 6
The BOTTOMLINE on ROI The New Definition of Value Value Must: Be balanced, with qualitative and quantitative data Contain financial and nonfinancial perspectives Reflect strategic and tactical issues Satisfy all key stakeholders Be consistent in collection and analysis Be grounded in conservative standards Come from credible sources 7
The BOTTOMLINE on ROI The “Show Me” Evolution 8
The BOTTOMLINE on ROI Shifting Paradigms Activity Based Results Based No business need for the program Program linked to specific business needs No assessment of performance issues Assessment of performance effectiveness Specific objectives for behavior and No specific measurable objectives business impact No effort to prepare program participants to Results expectations communicated to achieve results participants No effort to prepare the work environment Environment prepared to support transfer to support transfer No efforts to build partnerships with key Partnerships established with key managers managers & clients No measurement of results or benefit-cost Measurement of results and benefit-cost analysis analysis Planning and reporting is input focused Planning and reporting is outcome focused 9 Source: Measuring ROI in Healthcare: Tools and Techniques to Measure the Impact and ROI in Healthcare Improvement Projects and Programs , 2013, V. V. Buzachero, J. Phillips, P. P. Phillips, and Z. L. Phillips.
The BOTTOMLINE on ROI The Executive View * Current Should Measure Measure Measure Measure Importance Last year, 78,000 employees received 1. Inputs 94% 85% 6 formal learning. Formal learning costs $2.15 per hour of 2. Efficiency 78% 82% 7 learning consumed. Employees rated our training very high, 3. Reaction 53% 22% 8 averaging 4.2 out of 5. 92% of participants increased knowledge 4. Learning 32% 28% 5 and skills At least 78% of employees are using the 5. Application 11% 61% 4 skills on the job. Our programs are driving our top 5 1 6. Impact 8% 96% business measures in the organization. Five ROI studies were conducted on 7. ROI major programs yielding an average of 4% 74% 2 68% ROI. Our learning and development program 8. Awards won an award from the Association for 40% 44% 3 Talent Development. * CEO Survey—Fortune ne 500 and nd Large Private Company , ROI Institu tute 10
The BOTTOMLINE on ROI 2011 Global Leadership Development Practices Survey Results Measurement Measurement 2006 2011 Levels Category Status Status 0 Inputs/Indicators 100% 100% Number of programs, participants, costs, efficiencies 1 Reaction 92% 89% Reaction and satisfaction with experience, content, value of program 2 Learning 48% 59% Acquisition of information, knowledge, skills, contacts 3 Application 11% 34% Use of information, knowledge, skill, insights 4 Impact 8% 21% Changes in business impact measures such as output, quality, time, costs 5 ROI 2% 11% Compares monetary benefits to cost of program, project, or initiative N=232; large organizations with leadership development budgets averaging $3 million annually.
The BOTTOMLINE on ROI The ROI Methodology Generates Six Types of Measures: • Reaction and Planned Action • Learning • Application • Business Impact • Return on Investment • Intangible Measures The ROI Methodology also includes a technique to isolate the effects of the program or solution. 12
The BOTTOMLINE on ROI An Evaluation System Must Have Five Elements 13
The BOTTOMLINE on ROI ROI Framework Level Measurement Focus Measures participant satisfaction with 1. Reaction and Planned the program/project and captures Action planned action Measures changes in knowledge and 2. Learning skills Measures changes in on-the-job 3. Application behavior or actions Captures changes in business impact 4. Business Impact measures Compares monetary benefits to the 5. Return on Investment costs 14
The BOTTOMLINE on ROI An Evaluation System Must Have Five Elements 15
The BOTTOMLINE on ROI The ROI Methodology, Part 1 Evaluation Planning Data Collection LEVEL 1: REACTION AND LEVEL 3: APPLICATION PLANNED ACTION Collect Collect Data Develop Develop/ Data During After Evaluation Review Program Program Plans and Objective of Implementation Implementation Baseline Data Program LEVEL 2: LEVEL 4: BUSINESS IMPACT LEARNING 16
The BOTTOMLINE on ROI The ROI Methodology, Part 2 Capture Costs of Solution Reporting Data Analysis Calculate Convert Generate Isolate the Return Data To Impact the Effects of On Monetary Study Program Investment Value LEVEL 5: ROI Identify Intangible Measures Intangible Benefits 17
The BOTTOMLINE on ROI An Evaluation System Must Have Five Elements 18
The BOTTOMLINE on ROI Guiding Principles, Part 1 When conducting a higher-level evaluation, collect data at 1. lower levels. When planning a higher level evaluation, the previous level 2. of evaluation is not required to be comprehensive. When collecting and analyzing data, use only the most 3. credible sources. When analyzing data, select the most conservative 4. alternatives for calculations. Use at least one method to isolate the effects of the 5. program or project. If no improvement data are available for a population or 6. from a specific source, assume that no improvement has occurred. 19
The BOTTOMLINE on ROI Guiding Principles, Part 2 Adjust estimates of improvements for the potential error of 7. the estimates. Avoid use of extreme data items and unsupported 8. claims when calculating ROI calculations. Use only the first year of annual benefits in the ROI analysis 9. of short-term solutions. 10. Fully load all costs of the solution, project, or program when analyzing ROI. Intangible measures are defined as measures that are 11. purposely not converted to monetary values. 12. Communicate the results of the ROI Methodology to all key stakeholders. 20
The BOTTOMLINE on ROI An Evaluation System Must Have Five Elements 21
The BOTTOMLINE on ROI Example Application Case Studies 22
The BOTTOMLINE on ROI An Evaluation System Must Have Five Elements 23
The BOTTOMLINE on ROI ROI Myths • ROI is too complex for most users. • ROI is too expensive, consuming too many critical resources. • If senior management does not require ROI, there is no need to pursue it. • ROI is a passing fad. • ROI is too subjective • ROI is for post analysis only 24
The BOTTOMLINE on ROI 25
The BOTTOMLINE on ROI The ROI Calculation Program Benefits $300,000 Benefit Cost Ratio (BCR) = Program Costs $200,000 = 1.50:1 (Program Benefits – Program Costs) $300,000 - $200,000 ROI = X 100 Net Program Benefits $100,000 Program Costs $200,000 = 0.50 X 100% = 50% 26
The BOTTOMLINE on ROI It’s That Easy! 27
The BOTTOMLINE on ROI The ROI Methodology, Part 1 Evaluation Planning Data Collection LEVEL 1: REACTION AND LEVEL 3: APPLICATION PLANNED ACTION Collect Collect Data Develop Develop/ Data During After Evaluation Review Program Program Plans and Objective of Implementation Implementation Baseline Data Program LEVEL 2: LEVEL 4: BUSINESS IMPACT LEARNING 28
The BOTTOMLINE on ROI Reaction & Planned Action Chain of Impact Learning Application & Implementation Isolate the Effects of the Program Impact Intangible Benefits 29
The BOTTOMLINE on ROI Five Levels of Measurement: Examples Level 0 Input and Indicators Level 1 Reaction and Planned Action • Number of programs • Relevance* • Participants • Importance* • Hours • Usefulness • Requests • Recommend to Others • Efficiencies • Necessary • Costs • Practical • Time to Deliver • Uniqueness • Concreteness • New Information* • Motivation • Appropriateness • Intent to Use* *Correlates with Application 30
Recommend
More recommend
