Checklists for Improving Evaluation Practice June 10, 2015 - PDF document

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 Checklists for Improving Evaluation Practice June 10, 2015 Introductions Lori Wingate Emma Perk Goldie MacDonald This material is based upon work supported by the National Science Foundation under grant number 1204683. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the presenters and do not necessarily reflect the views of NSF. evalu ‐ ate.org 1

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 Behind the Scenes Janet Pinhorn Mike Lesiecki Webinar Materials Jason Recording Slides Handout www.evalu-ate.org/webinars/2015-june/ evalu ‐ ate.org 2

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015  1. Checklist Form & Function  2. Using Checklists to Improve Evaluation Practice Checklists help us remember what we need to do evalu ‐ ate.org 3

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 Have you read this book? We need a different strategy for overcoming failure… Photo courtesy of the MacArthur Foundation www.macfound.org evalu ‐ ate.org 4

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 evalu ‐ ate.org 5

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 “ It is a checklist. …It is a checklist. Checklists Save Lives bit.ly/WHO-check evalu ‐ ate.org 6

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015  Checklist Form and Function Lori Wingate  Memory Task Decision Evaluation Aid Completion Making Laundry  Checklist Form & Function list Procedural   checklist Diagnostic    checklist Criteria     of merit checklist based on Scriven’s (2007) ”The Logic and Methodology of Checklists” bit.ly/check-logic evalu ‐ ate.org 7

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015  Laundry List A nonordered list of items or tasks, Checklist Form & Function sometimes grouped into categories evalu ‐ ate.org 8

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 bit.ly/fema-check bit.ly/fema-check Is there anything on this list you wouldn’t have thought of on your own? Get ready to use your markers… evalu ‐ ate.org 9

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 bit.ly/fema-check  Procedural Checklist A list of tasks or issues that must be Checklist Form & Function addressed in a particular order evalu ‐ ate.org 10

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 bit.ly/WHO-check evalu ‐ ate.org 11

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015  Diagnostic Checklist A series of questions or considerations Checklist Form & Function that lead the user to a descriptive conclusion about the status of something bit.ly/think-ebola [example – PTSD checklist? Auto diagnostic?] evalu ‐ ate.org 12

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 bit.ly/think-ebola [example – PTSD checklist? Auto diagnostic?]  Criteria of Merit Checklist A list of characteristics that define Checklist Form & Function quality for a particular product or phenomena to aid the user in reaching an evaluative judgment evalu ‐ ate.org 13

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 This is a checklist to be utilized by authors, report developers, and other interested parties to aid in the development and maximization of readability of written documents. This is a checklist to be utilized by authors, report developers, and other interested parties to aid in the development and maximization of readability of written documents. (Will help (Will help you you avoid writing avoid writing terribl terrible sentences e sentences like this one!) like this one!) evalu ‐ ate.org 14

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 This is a checklist to be utilized by authors, report developers, and other interested parties to aid in the development and maximization of readability of written documents. This is a checklist to be utilized by authors, report developers, and other interested parties to aid in the development and maximization of readability of written documents. evalu ‐ ate.org 15

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 1-page checklist is supported by 5 pages of explanation  MAIN FUNCTIONS Aid Memory Checklist Form & Function Guide Task Completion CHECKLISTS TS Guide Decision Making CHECKLIS Reach Evaluative Conclusions MAIN BENEFITS Distill Complex Content Transfer Knowledge Quality Control evalu ‐ ate.org 16

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015  Checklists Distill Complex Content Checklist Form & Function Photo from NSF Flickr stream evalu ‐ ate.org 17

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 bit.ly/cg-checklists 15 pages 52 pages (Six 1-3 page checklists) Photo from NSF Flickr stream bit.ly/cg-checklists 52 pages 15 pages (Six 1-3 page checklists) Photo from NSF Flickr stream evalu ‐ ate.org 18

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015  Checklists Support Quality Control Checklist Form & Function evalu ‐ ate.org 19

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 EvaluATE’s Webinar Checklist bit.ly/webinar-check  Checklists Aid in Knowledge Transfer Checklist Form & Function evalu ‐ ate.org 20

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 Michael Scriven Michael Quinn Patton Daniel Stufflebeam  Key Evaluation Checklist  Utilization-Focused Evaluation  Evaluation Plans and Checklist Operations Checklist  Qualitative Evaluation  CIPP Evaluation Model Checklist Checklist  Evaluation Design Checklist  Evaluation Contracts Checklist “ Daniel Stufflebeam Students began pressing me to explain what exactly I was doing in designing project evaluations. Answering their questions proved difficult. In retrospect, I was developing and exercising a kind of “ personal art of evaluation design rather than laying out and following any particular systematic approach. When the students persisted in pressuring me to give them an evaluation planning protocol, I decided to try to respond. Thus, I developed my first evaluation checklist.  Evaluation Plans and  Evaluation Plans and Daniel Stufflebeam (2000) Operations Checklist Operations Checklist “Guidelines for Developing Evaluation Checklists:  CIPP Evaluation Model The Checklists Development Checklist” Checklist www.wmich.edu/evalctr/checklists  Evaluation Design Checklist  Evaluation Contracts Checklist evalu ‐ ate.org 21

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 www.wmich.edu/evaluation/checklists  Discussion Goldie MacDonald evalu ‐ ate.org 22

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015  Emma Perk Lori Wingate Goldie MacDonald Questions Type them Type them in in the chat box now the chat box now evalu ‐ ate.org 23

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 Join us on Social Media EvaluATE WMUEvaluATE @EvaluATE_WMU #evalchecklists EvaluATE Group WE’RE HIRING! WE’RE HIRING! Assistant Director of The Evaluation Center bit.ly/wmu-ec-job Kalamazoo evalu ‐ ate.org 24

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015  Using Checklists to Improve Evaluation Practice Lori Wingate www.wmich.edu/evaluation/checklists Categories  Evaluation Capacity Building  Evaluation Design and Management  Evaluation Models  Evaluation Values and Criteria  Metaevaluation  Checklist Development evalu ‐ ate.org 25

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 Evaluation Contracts Checklist by Daniel Stufflebeam bit.ly/eval-contracts Evaluation Evaluation Contracts Contracts Checklist Checklist by Daniel Stufflebeam by Daniel Stufflebeam bit.ly/eval-contracts bit.ly/eval-contracts evalu ‐ ate.org 26

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 Evaluation Contracts Checklist by Daniel Stufflebeam bit.ly/eval-contracts Evaluation Contracts Checklist by Daniel Stufflebeam bit.ly/eval-contracts evalu ‐ ate.org 27

EvaluATE Webinar: Checklists for Improving Evaluation Practice 6/10/2015 Evaluation Budget Checklist by Jerry Horn bit.ly/eval-budget Photo courtesy of www.seniorliving.org Evaluation  Basic Considerations Budget  Travel Checklist  Supplies and Materials  Communications  Copying Printing 6 pages of checkpoints across 10 categories  Equipment  Consultants by Jerry Horn  Subcontracts bit.ly/eval-budget  In-kind or Cost-sharing  Indirect/Overhead Photo courtesy of www.seniorliving.org evalu ‐ ate.org 28