STARTING WITH THE END IN MIND How evaluation can support you in achieving program goals October 23, 2017 Sophia Mansori Sheila Rodriguez Gabriela Garcia
Today’s agenda Introduce our evaluation team 1 Review evaluation activities and expectations 2 Strategies to start with the end in mind 3 2 21st CCLC Fall Program Director’s Meeting | edc.org
Meet the EDC team EDC has been working with ISBE on the 21 st CCLC evaluation since 2013! v Leslie Goodyear: Our fearless leader v Sophia Mansori: Making sure things go v Josh Cox: Keeping track of our data v Sheila Rodriguez: Connecting with you v Gabriela Garcia: Connecting with you 3 21st CCLC Fall Program Director’s Meeting | edc.org
Overview of the statewide evaluation Our goals: § To provide ISBE instructive, relevant, and actionable data and information on the progress of the 21 st CCLC program and its grantees toward meeting the state’s program objectives. § To provide grantees feedback regarding their performance with respect to program objectives, as well as support and feedback on their evaluation of implementation and progress. 4 21st CCLC Fall Program Director’s Meeting | edc.org
Statewide Evaluation: Activities and Expectations EDC Activities Grantee Expectations v Administer and support data collection ü Submit an annual local evaluation from grantees (local evaluation report, report for each grant (Due Dec 1, 2017) annual survey) ü Submit an annual survey for each grant v Conduct site visits for evaluation (Spring 2018) studies ü Submit APR data and student roster v Provide annual reports on grantee information to the Benchmarking Tool activities and progress ü Participate in site visits v Provide technical assistance on evaluation topics 5 21st CCLC Fall Program Director’s Meeting | edc.org
Local Evaluation Report ü Each grantee must complete an annual local evaluation. ü Grantees submit one report per grant . The report should address all of the sites under the grant. Organizations with multiple grants submit multiple reports. ü The report template was emailed and will be available online. ü Reports are due December 1, 2017. ü Send reports to 21stCCLC.Evaluation@edc.org. 6 21st CCLC Fall Program Director’s Meeting | edc.org
Local Evaluation Report Goals § Emphasis of evaluation is on learning from the data you have collected. What do the data tell you about your program? What recommendations do you have for program improvement? § The template outlines the minimum information that should be included in your local evaluation report. 21st CCLC Fall Program Director’s Meeting | edc.org 7
Because EVERYTHING starts EARLY start early with the end in mind 8 21st CCLC Fall Program Director’s Meeting | edc.org
Strategies to help you start with the end in mind Develop a program logic Evaluation questions and model methods matrix Data collection W’s Plan for evaluation use 9 21st CCLC Fall Program Director’s Meeting | edc.org
Logic Models I NPUTS A CTIVITIES O UTPUTS S HORT T ERM O UTCOMES L ONG T ERM O UTCOMES Federal Funding Provide technical assistance # Trained professionals in 21st CCLC programs develop plans for Long term sustainability of CCLC afterschool programming sustainability programs via funding and resources 21 st CCLC program Offer professional guidelines development # Networked professionals in 10% or more increase in academic Strong partnerships for families, afterschool programing achievement for students who schools, and communities Statewide objectives Funding participate in 21st CCLC programs Guidelines and examples of 21 st Program staff, families, and Technical assistance Reviewing grants and CCLC programs in the State of 21st CCLC program staff increase communities increase awareness and provider approving program plans Illinois understanding of key areas of the knowledge of 21 st CCLC and statewide objectives afterschool programs, statewide Professional development Provide opportunities for # Programs serving students with objectives and NCLB and training in State grantees and ISBE to interact the greatest need in all parts of state 21st CCLC program staff have increased ISBE Standards and NCLB (e.g., site visits, spring professional development opportunities Students who participate in 21 st conferences, regular Data on grantee activities CCLC’s graduate from high school Staff who are knowledgeable communication) Increase in staff trained to meet the about the 21 st CCLC State-wide program evaluation needs of 21st CCLC programs, staff, Students who participate in 21 st program and issues in Reporting and feedback to report and students CCLC’s apply to and enter college afterschool Department of Education, State of Illinois, and CCLC Increase in program opportunities for Students who participate in 21 st programs students who are in the greatest of need CCLC’s are prepared for post secondary and/or career readiness Monitoring ISBE increases understanding of program activities, successes and Students who participate in 21 st State-wide evaluation challenges CCLC’s and their families demonstrate a strong knowledge base in the common core state standards and Illinois learning standards Funding Professional development # of staff who receive professional Reported increases in school development attendance rates for students Students who participate in 21 st Educator expertise Activities that strengthen the CCLC’s demonstrate positive behavior academic and social skills of # and dosage of activities that Students improve attitude towards and strong socio-emotional skills Individual program guidelines students strengthen academics, college and school and goals career readiness, socio-emotional Activities that develop college skills Students engage in and demonstrate Localized resources and career readiness positive behavior Grantees # Students who participate in Professional development and Activities that address behavior activities that strengthen academics, Students improve academic training and socio-emotional skills college and career readiness, socio- achievement emotional skills Activities that encourage family Students develop college and career and community involvement # of activities offered by programs readiness skills that engage families and community Local Evaluation Families and community members # of families and community increase engagement in program members involved in activities activities and develop relevant skills and knowledge E XTERNAL C ONTEXTUAL F ACTORS 10 21st CCLC Fall Program Director’s Meeting | edc.org
Evaluation questions and methods matrix Objective Evaluation Questions Spring Local Site APR Survey evaluation Visits data Do 21 st CCLC programs provide opportunities for participants Objective #1: Participants will demonstrate X X X an increased involvement in school to increase participation in activities and subjects such as activities and participation in other subject technology, the theatre and arts, and extracurricular activities areas such as technology, arts, music, such as sports and clubs? In what ways? For whom? theater, sports and other activities. To what extent do program participants increase participation X X X X in activities and subjects such as technology, the theatre and arts, and extracurricular activities such as sports and clubs? Objective #2: Participants in the program To what extent do program goals and activities address and X X X will demonstrate increased academic support increased academic achievement for program achievement. participants? Have 21 st CCLC program activities and services positively X X influenced student achievement outcomes (i.e., increased student test scores, grade promotion rates)? 11 21st CCLC Fall Program Director’s Meeting | edc.org
Data Collection W’s (and an H) What Why When Who How What are the Why are you When is the best Who needs to How will you kinds of collecting these or most be included in use these information and data? appropriate time the process of data? data that will to collect this both collecting help you What will they information? and What understand this tell you? interpreting processes are goal? When do you data? there to need data in incorporate order to make findings and decisions? make program changes? 12 21st CCLC Fall Program Director’s Meeting | edc.org
Plan to use your evaluation findings early and often Evaluation use is what happens when you don’t just put a report in a file cabinet... Let funders Share student know the outcomes with extent of your school day Communicate Inform parents services. teachers. with Partners of how children about program benefit from activities and our program. connections. Discuss results with staff, including challenges they face and what worked well. 13 21st CCLC Fall Program Director’s Meeting | edc.org
Example: Using attendance and achievement data Issue Data Response Program wanted to § 59% of participants who Ø Recommendation in evaluation: understand attended 60+ days made Develop a retention plan. effectiveness and gains in at least one subject Ø Communicate with parents about participant area. the benefits of regular outcomes. § 30% of participants participation. attended 60 days or more Ø Engage staff in developing retention strategies Ø Collect more data to understand barriers to regular attendance 14 21st CCLC Fall Program Director’s Meeting | edc.org
Recommend
More recommend
