PRACTICAL STRATEGIES TO SUPPORT DATA COLLECTION AND ANALYSIS 1
WE ARE GLAD YOU ARE HERE! 2 ITACC: 2017 AIDD Technical Assistance Institute
LEARNING OBJECTIVES • Participants will recognize basic concepts related to data collection; • Participants will understand data needed to report on new performance measures; • Participants will explore data collection activities and strategies. 3 ITACC: 2017 AIDD Technical Assistance Institute
4 THINGS TO KNOW • Know why you want to evaluate a project/activity. • Process of evaluating progress towards goals • Collect and Use Data • Accountability for results • Clarity of outcomes 4 ITACC: 2017 AIDD Technical Assistance Institute
o • o • o o • • Notes for Slide 4: Before collecting data – Know why you want to evaluate a project/activity. Logic model identifies desired outcomes, resources, and activities necessary to accomplish the outcomes. Process of evaluating progress towards goals: Align structures, processes, and routines to enable a methodical and routine evaluation process between work and desired outcomes. Collect and Use Data Establish and implement routines and processes for collecting, analyzing, and monitoring data Accountability for results Making decisions to continue, improve, or end practices based on data; communicating results with internal and external stakeholders Clarity of outcomes Communicate priorities, set ambitious, clear, and measurable goals and outcomes with aligned strategies and activities. 5
CONNECTIONS WITH STRUCTURES AND PROCESSES Systematic approach to ensure quality and progress towards goals Aligning structures, processes, and routines Logic Model Evaluation Plan annual work plan – data collection 6 ITACC: 2017 AIDD Technical Assistance Institute
COLLECT AND USE DATA The collection of data can inform continuous improvement, provide feedback, and help guide decisions. 7 ITACC: 2017 AIDD Technical Assistance Institute
Notes for Slide 6: Collect and Use Data Establish and implement routines and processes for collecting, analyzing, and monitoring data. There is a resource on the itacchelp.org website - http://itacchelp.org/wp-content/uploads/2015/12/2015-TAI-Evaluation-FAQ-3.pdf 8
ACCOUNTABILITY FOR RESULTS • Decisions based on results • Improve • Adjust • Terminate • Incentives • Additional or continuation funding • Communicating results • Internal and external stakeholders 9 ITACC: 2017 AIDD Technical Assistance Institute
Notes for Slide 7: Making decisions to continue, improve or end practices based on data; implementing incentives tied to performance; and engaging and communicating results with internal and external stakeholders. 10
CLARITY OF OUTCOMES • Establish Priorities • Communicate priorities • Set ambitious, clear, measurable goals and outcomes with aligned strategies and activities. 11 ITACC: 2017 AIDD Technical Assistance Institute
Notes for Slide 8: To ensure the Council is getting what they want from projects and activities, it is important to establish priorities for the activity and to communicate the expectations. Of course, setting clear, measurable goals and outcomes that align well with the strategies and activities will help 12
HAVE YOU EXPERIENCED THIS WITH GRANTEES? 13 ITACC: 2017 AIDD Technical Assistance Institute
Notes for Slide 9: When councils engage other entities to help accomplish goals and objectives – and changes are made to the initial plans, don’t forget to adjust the evaluation component as well – ITACC staff frequently hears about data being provided to Councils that does not speak to the overall outcomes of the funded project/activity – having a process in place that helps grantees understand the data that is needed to measure impact is critical! 14
MEASURING IMPACTS AND RESULTS 15 ITACC: 2017 AIDD Technical Assistance Institute
Notes for Slide 10: Let’s take a look at the new performance measures – the new measures call for connection between investments, outputs, and the outcomes of the activity – this translates into results and short-term impact, intermediate impact, and ultimate impacts of Council activities 16
PM INFO DATA = IT’S NOT JUST NUMBERS • Data = facts and statistics collected together for reference or analysis. • Do you know what you need to know? • Think Impact, Impact, Impact!! • Lessons learned: • Training for sub grantees is very important • Councils need to take the lead • Request more than you think you’ll need • Continuous monitoring and follow up for clarification • Self reporting and incomplete survey’s • Use examples and tools from states who’ve been doing it 17 ITACC: 2017 AIDD Technical Assistance Institute
Notes from Slide 11: Data: Good data isn’t just a bunch of raw numbers – it’s a compilation of facts based on a clear understanding of WHAT you are tying to find out. The results should tell a story. Do you know what you need to know? These PM’s aren’t new. Although they keep changing slightly, the basic concept of what you need to collect isn’t. Make sure you really read and think through these PM’s so you know what you need to collect. Lessons Learned: Initial training is vital. Talk to the grantees about the new way we need to collect and report on data. Put the work in up front - look ahead at what you need to report in the PPR, then track back to figure how you will get that information from the grantees in a way that’s clear and concise. For us it was creating our own reporting templates which will get to in a min. Take the lead in telling sub-grantee’s what you need to know and how you want it reported. I tell the grantees if they question whether info is relevant – put it in and I will determine that because I’d rather have more than not enough. Continuous monitoring = allows you to make sure that you’re getting the data, and that it’s relevant to what you need to report in the PPR. Follow up with sub-grantees is very important. Make sure you get clarification. Looking at surveys = not everyone completes every question on a survey! You have to calculate percentages based on the number of people who answered a question – not based on the number of people who completed the survey! 18
CAN ' T HAVE ONE WITHOUT THE OTHER Data Collection • Narrative • Performance Measures • Excel spreadsheets • Logic Model • DD Suite • Work plan • Surveys • Sub-grantee reports PPR/IMPACT • Impact Statements • Personal Stories How We Show State Plan Our Worth • Sign in sheets • Focus Groups • Conversations with sub-grantees 19 ITACC: 2017 AIDD Technical Assistance Institute
THE OREO EFFECT STATE PLAN DATA PPR OUR IMPACT 20 ITACC: 2017 AIDD Technical Assistance Institute
Notes from Slide 13: The Oreo effect. The filling is the data collection, it’s what holds the top and bottom wafers together. Without it, you’d just have chocolate wafers, which are ok, but it’s just not an Oreo. 21
LET’S LOOK AT SOME TOOLS 22 ITACC: 2017 AIDD Technical Assistance Institute
Notes from Slide 14: With change – Council’s have the opportunity to revisit the evaluation tools that will best meet the Councils needs… 23
SOUTH CAROLINA • Quarterly Progress Reports • Surveying Process • Monitoring Visits • Grantees can enhance minimum reporting requirements to best-fit project/activity • Ultimate question from Council staff is: “how are you measuring the impact of your programming ?” 24 ITACC: 2017 AIDD Technical Assistance Institute
Recommend
More recommend
