Evaluation: Hot tips from The Ian Potter Foundation Dr Squirrel Main, Research and Evaluation Manager
Before we begin… Please note that IPF staff may be taking photos/videos during this workshop for social media purposes. If you do not wish for your image to be included in the publicity, please let any staff member know.
Presentation Overview Standards of evidence Level 1: Grantee tips and KPIs Level 2: Endgame and sustainability Level 3: Resources and ideas Dissemination: Infographics
Evaluation vs. chasing unicorns…
Standards of Evidence
Level 1: Top tips 1. Train the volunteers conducting surveys 2. Ensure redundancy in the key knowledge holders (bikes- cancer-hearts) 3. Hold immediate staff-wide dissemination-training as soon as possible 4. Start early: consultations with stakeholders take a long time 5. Capture baseline data 6. Use pre-existing, validated surveys (North Carolina Family Assessment Scales) 7. Ensure all council permits, etc. are in order
Level 1: SMART KPIs Take a moment to review your goals in pairs. Listen to your partner’s goals and provide feedback - Are they SMART ? S pecific M easurable A chievable R ewarding/Relevant T ime-bound How can your partner improve their measurements? Have time? Take a moment to review your long-term outcomes in pairs…. How will your partner begin to measure/collect data for their longer-term outcomes? Challenges? Solutions?
Level 2: Evaluation key steps Determine Involve Circulate Finalise Review first Disseminate end-game stakeholders tender proposal draft
Level 2: Decide your endgame Source: Alice Guglev and Andrew Stern. What’s Your End -game? Global Development Incubator. 30 January 2014.
Then tailor your evaluation… Short- Medium- Long- Inputs Activities Outputs term term term outcomes outcomes outcomes • How does the program work? • What outcomes resulted from the • What did the program program? accomplish? • When and to whom? • What has influence how the • What is the program’s program works? effectiveness? (Findings, not Rec’s) • How do changes compare to other programs/sectors?
Level 2: Who would you invite?
Level 3: Evaluation resources • TIPFEP • Social impact hub
Level 3: Data rehearsal
Level 3: Data sources
Hot tip: Simplicity
Table 2: Statistical Analysis of Pre-Post Workshop Questionnaire - Paired t-test Table 2 outlines the statistical analysis of the data presented in Tables 3 to 7. Pre & Post Test n= Pre-test Mean Post test-Mean Difference Significance Level Question 1 59 2.64 3.98 1.33 P< 0.001 P< 0.001 Question 2 59 3.64 4.17 0.52 P< 0.001 Question 3 59 2.76 4,20 1.44 P< 0.001 Question 4 59 3.37 4.17 0.79 Question 5 59 3.19 4.20 1.01 P< 0.001 Table 2 shows the mean differences between pre workshop and post workshop responses for each question from 59 participants. Data was subject to a paired t-test to determine the p value for each question. This indicated the outcome to be highly significant at p<0.01 as the difference in mean is significant and not due to chance, which indicates attitudinal change and knowledge transfer.
Or this? Growth in financial knowledge: Responses from 59 workshop participants 5 4.2 4.2 4.0 4 3.4 3.2 3 2.6 2 1 It's a waste of time to think In order to avoid debts and The only way to pay bills about managing money financial stress it’s and saving is to spend less important to understand than you earn how loans, credit cards, interest and mobile phone plans work Pre-test Mean Post test-Mean
Read it Pull out one big quote and one big stat for people to remember Find other ‘pull’ quotes and numbers Think about the best way to represent each item White space!
Secret sauce to communicating outcomes: creativity?
Or is the secret sauce a good webpage? https://thenounproject.com/ http://piktochart.com/ https://infogr.am/
Recommend
More recommend
