Asking the “right” questions for IMPACT Jonathan Ferris Purdue Extension – Fayette County Purdue University is an Equal Opportunity/Equal Access institution.
The Goal “ Data collection is worthless unless people are able to use the findings and use them well. The methodological area that presents the most challenge …. is that of helping the people who should use information from program studies to interpret it such a way that it actually is used in a positive manner benefiting people and communities.” Sara Steele, Univ. Wisconsin Extension Purdue University is an Equal Opportunity/Equal Access institution.
Or, “ How to keep this whole IRB/Human Subjects evaluation thing as simple as possible so as to not pull your hair out and hopefully end up with a simple yet effective impact statement that will blow away your district director and program leader as well as wow the folks in DC.” Purdue University is an Equal Opportunity/Equal Access institution.
Where I’m coming from… • In the past few years, I’ve reviewed 500+ IRB applications • Many survey instruments are overly complicated and detailed • We all could use help in crafting questions that get the types of answers we want. (None of us are trained in this!) Even with IRB stuff, our jobs aren’t that bad… Purdue University is an Equal Opportunity/Equal Access institution.
So how do we start thinking about this? (My humble opinion) 1. (Bear with me as I get philosophical for a minute…) Think of a program you really enjoy doing in your county/area. • Why do you do it?? What drives you? • What do you really want your participants to take away from it? Purdue University is an Equal Opportunity/Equal Access institution.
More… 2. What would you REALLY like for the impact statement to say? Imagine writing it now…what would it say? Don’t forget – you need to be realistic about what you can accomplish in (for example) a 3 hour program. Be concise (this isn’t thesis research). Your efforts will probably be summed up into a couple of sentences anyway! Purdue University is an Equal Opportunity/Equal Access institution.
My personal pet peeve… 3. Only ask questions you NEED to ask! -In other words, make sure there is a reason for asking the question. Will the data it generates be part of your impact statement? Example: What is your age? (Check one) __ Under 18 __ 18-24 __24-32 __32-40 __Over 40 Purdue University is an Equal Opportunity/Equal Access institution.
Some types of questions • Program Improvement What if they had low Did this program… expectations? a) Meet your expectations b) Exceed your expectations c) Not meet your expectations Program improvement questions should be kept to a minimum. Maybe even just poll a few folks afterwards? Purdue University is an Equal Opportunity/Equal Access institution.
“Scale questions” The information on weed As a result of this control was: workshop, I am more knowledgeable about a) Not beneficial at all (XYZ) (Circle one) b) Slightly beneficial 4) Strongly agree c) Moderately beneficial 3) Agree d) Very beneficial 2) Disagree 1) Strongly Disagree Purdue University is an Equal Opportunity/Equal Access institution.
More Scale questions… To what extent was this meeting worth your time? (Circle one) Not at all A great deal 1 2 3 4 5 6 Please rate your knowledge of (XYZ) before and after the program by placing a (B) or (A) on the scale below Least Most 1 2 3 4 5 6 7 8 9 10 Purdue University is an Equal Opportunity/Equal Access institution.
Strongly Agree/Disagree • The problem with this The resources provided sort of question (in my were useful (check one) opinion), is that when Strongly Agree you try to report the Agree findings, it becomes Undecided “wordy”. Also, in this Disagree specific case, the Strongly Disagree question could have been worded better. (“Will you use the resources provided?”) Purdue University is an Equal Opportunity/Equal Access institution.
Irrelevant? Gender: Male__ • Some of these sorts Female__ of questions may have merit, if you How old are you? plan to break the data out differently What size farming depending on their operation do you have? answers. If not, do you need to know? Purdue University is an Equal Opportunity/Equal Access institution.
Enough negative…let’s practice making questions better! • How about… Did this program… “Would you like to see a) Meet your similar workshops expectations offered in the b) Exceed your future?” (Yes/No) expectations c) Not meet your expectations Purdue University is an Equal Opportunity/Equal Access institution.
• How about… The information on weed control was: “Do you plan to change your weed a) Not beneficial at all management b) Slightly beneficial practices as a result c) Moderately of what you learned beneficial today?” (Yes/No) d) Very beneficial Purdue University is an Equal Opportunity/Equal Access institution.
As a result of this • How about… workshop, I am more “Do you plan to offer knowledgeable about more healthy snacks healthy snacks (Circle to your children as a one) result of today’s 4) Strongly agree program?” (Yes/No) 3) Agree 2) Disagree 1) Strongly Disagree Purdue University is an Equal Opportunity/Equal Access institution.
To what extent was this meeting worth your time? (Circle one) Not at all A great deal 1 2 3 4 5 6 • How about… “Would you recommend this program to your friends?” (Yes/No) Purdue University is an Equal Opportunity/Equal Access institution.
Please rate your knowledge of pond weed management before and after the program by placing a (B) or (A) on the scale below Least Most 1 2 3 4 5 6 7 8 9 10 • How about… “Did you learn something about managing your pond weeds that will help you enjoy your pond more this year?” (Yes/No) Purdue University is an Equal Opportunity/Equal Access institution.
Surveys – how can we “ramp it up” and make them more transformational ? • Think about audiences you work with who attend your programs every year. • Why not simply add a few questions for returning attendees to your survey instruments? • What could you ask them to see if practices/behaviors have changed? Purdue University is an Equal Opportunity/Equal Access institution.
(An Example of how this might work…) If you have attended DPAC Field Days in the past , please take a moment to tell us about how the information you received at the field days has helped you in your farming operations. 1. How many years would you estimate you have attended the DPAC field day? _______________ 2. What keeps you coming back? 3. Please tell us about one or two specific things that you are doing differently in your operation as a result of something you learned at a DPAC Field Day. 4. Would you say that the information you’ve received at DPAC field days in the past has helped you be more profitable? (Circle one) YES NO 5. Can you think of one thing you wish we’d change about this program? If so, what is it? Purdue University is an Equal Opportunity/Equal Access institution.
What results did we get? 3. Please tell us about one or two specific things that you are doing differently in your operation as a result of something you learned at a DPAC Field Day. Herbicide use, corn stalk management, better herbicide selection (9x), weed management (8x), scout more before making a decision, conservation farming, GPS signal on front of tractor, changing weight of tractor, research more before investing time and $, drainage & pesticide safety, GPS, tillage, plant populations, insect resistance management, drift control, more pre- emergent herbicide usage, scouting 4. Would you say the information you received at DPAC field days in the past has helped you be more profitable? – Yes: 95% – No: 5% Purdue University is an Equal Opportunity/Equal Access institution.
Another Example: (From Wild Game Workshops) 4. As a result of attending our workshops, would you say that you tend to keep or bring home more fish & game? (Circle one) YES NO 5. As a result of attending our workshops, would you say that you hunt/fish more now that you are better prepared to utilize the game you harvest? (Circle one) YES NO (Keep in mind – sometimes you craft questions with ulterior motives in mind, like future sponsors/partners…) Purdue University is an Equal Opportunity/Equal Access institution.
Results… 89% of returning respondents said they actually hunt or fish more (as a result of the workshops), now that they can better utilize the fish/game they harvest. 95% said they keep or bring home more fish/game as a result of attending our programs. Do you think the DNR might be interested in hearing about this? Future sponsor/partner? Purdue University is an Equal Opportunity/Equal Access institution.
Recommend
More recommend
