Grantee Logic Models: Application through Implementation Evaluation and Logic Model Initiative CSP Project Directors’ Meeting April 2018
This presentation was produced in whole with Federal funds from the U.S. Department of Education under contract with WestEd, number ED-OII-15-C- 0051. Patricia Kilby- Robb is the Contract Officer’s Representative (COR) for this project, patricia.kilby-robb@ed.gov. The content of this presentation does not necessarily reflect the views or policies of the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. 2
Objectives 1. Introduce Logic Model Review tool 2. Identify strategies to strengthen use and implementation of logic models 3
Agenda • Logic model review • Introduce and discuss logic model review tool • Identify strategies to strengthen logic models • Group activity 4
Who is WestEd? is a research, development, and service agency that works with education and other communities to promote excellence, achieve equity, and improve learning for children, youth, and adults. has monitored several U.S. Department of Education (ED) discretionary grant programs including the Charter Schools Program, Magnet Schools Assistance Program, and Voluntary Public School Choice . 5
Evaluation and Logic Model Initiative • Convene semi-regular expert panel to review logic models, identify criteria, develop rubric and review tool • Validated review tools with panel and CSP staff • Reviewed all grantee logic models against rubric • Providing on-going resources, TA, and guidance 6
Expert Panel Members Candice Bocola – Melvin Mark – Penn State Harvard/WestEd University Thomas Brush – University of Joanna Smith – University of Indiana Oregon Ann Doucette – Claremont Elena Polush – Iowa State University/Evaluator’s Institute University 7
Evaluation and Logic Model Initiative New this year: • Reviewed FY17 grantees • Developed implementation/usage and end of grant phases of tool 8
FY16 Notice Inviting Applicants Application Requirements: (d) Logic model: Provide a complete logic model (as defined in this notice) for the project. The logic model must address the role of the grant in promoting the State-level strategy for expanding the number of high-quality charter schools through startup subgrants, optional dissemination subgrants, optional revolving loan funds, and other strategies. 9
FY17 Notice Inviting Applicants (g) Quality of the Management Plan and Theory of Action (up to 15 points): ( 1) The quality, including the cohesiveness and strength of reasoning, of the ‘‘logic model’’ (as defined in this notice), and the extent to which it addresses the role of the grant in promoting the State-level strategy for using charter schools to improve educational outcomes for students through CSP subgrants for planning, program design, and initial implementation and other strategies; 10
Definition in NIAs Logic model (also referred to as a theory of action) means a well- specified conceptual framework that identifies key components of the proposed process, product, strategy, or practice (i.e., the active ‘‘ingredients’’ that are hypothesized to be critical to achieving the relevant outcomes) and describes the relationships among the key components and outcomes, theoretically and operationally. (34 CFR 77.1) 11
What is in a logic model? Resources Activities Outputs Outcomes • Staff, • Project • Products • Short-, partners, work mid-, funding, long- other term Assumptions External Factors External Evaluation 12
Why use a logic model? “…to help generate the clarity and specificity required for success and often demanded by funders and constituents.” “Logic modeling offers an organized way of defining your program goals, services, and measurable outcomes by producing: • An inventory of what you have and what you need to operate your program • A strong case for how and why your program will produce desired results • A method for program management and assessment 14
What makes a good logic model? • Clarity – Are the connections among elements clear and logical? Is level of detail appropriate? • Accuracy – Does the model reflect the project? Is content complete? • Logic – Does it make sense? 15
CSP Logic Model Expectations • Stand alone document • Adequately characterizes grant project • Used in on-going implementation of grant project 17
Logic Model Review and Implementation Tool Implementation Progress tool Initial Review tool is designed is designed to facilitate on-going to systematically review the usage of the logic model during content of the logic models to life span of grant ensure common features and relevant content 18
Roles and Responsibilities Application Content • CSP provides application content Expert Panel • Expert Panel conducts the initial Review review on application content • Grantees makes revisions, as Grantee needed Input • Grantees’ annually provide updates on progress • CSP annually reviews grantee CSP Review progress, provides feedback 19
Logic Model Review Tool – Structural Components • Are all components included (resources, activities, outputs, outcomes, assumptions, and external factors)? • Are there explicit relationships? • Are outcomes defined within the grant time frame? 20
Logic Model Review Tool – Content Review • Is there an alignment with the state strategy? • Are the project objectives/ performance measures included? • Are projected outcomes feasible? • Do outcomes demonstrate meaningful value/public benefit? 21
Logic Model Review Tool – Rubric • Alignment/relationship between grant goals, inputs, activities, outputs, and outcomes • Reasonable timeframe for activities and outcomes within the period of the grant • Connects grant project to state strategy • Adequacy of resources 22
Logic Model Progress Tracker • Designed to support use of logic models by grantees in regular grant administration • Intended to help grantees capture major project implementation and changes • Expected to be reviewed annually by Program Officers in conjunction with APR reviews 23
Logic Model Progress Tracker • What is current implementation status? Has there been any changes? Any additional information? • Resources • Activities • Outputs • Outcomes • External Factors 24
Getting from here to there • Know where you start • Review your current logic model • Know what you can adjust (and what you can’t) • These are a part of your approved application package • Revisions to clarify and strengthen are preferred • Major changes should be done in conjunction with Program Officer 25
Tips for developing a stronger logic model • Components in the right order • Elements under the correct components • Defined and detailed elements • Inclusion of project objectives and/or performance measures in outcomes • Articulated relationships • Defined assumptions, external factors 26
Tips for using logic models for project implementation • Using a logic model should be part of an ongoing process of discussion, commitment, and reflection as part of a grantee’s dedication to program improvement and accountability . • The process serves as the starting point… and can later be considered the “report card”… to understand whether or not you are meeting objectives. • They should be revisited on regular basis and should closely reflect the goals of your [grant project] and evaluation plan. 27
Group Activity Split the room by grant year. • FY15 and FY16 grantees • FY17 grantees FY15 Then…(see handout) FY16 • Review initial logic model feedback and identify ways to strengthen logic model OR FY17 • Review current logic model and identify implementation status and changes 28
Homework FY15 and FY16: FY17: • • Summarize current Clarify timeframes implementation status • Identify assumptions and • Identify changes (as needed) external factors • Talk with Program Officers 29
Wrap-up/Discussion • Questions? • What’s next? 30
Contact us! Sara Allender – sallend@wested.org John Flaherty – jflaher@wested.org Sharon Herpin – sherpin@wested.org WestEd.org
Recommend
More recommend