Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Dr. Nicolas Antony Valcik, Director Dr. Kimberly Scruton, Professor Office of Institutional Research of Business Management West Virginia University Dean Business Administration, Marketing, and Management Methodist University August 19, 2015 AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Acknowledgements The authors would like to extend our thanks for all of the support given to us by the personnel at the Kentucky Campus Compact, Berea College, the Kentucky Higher Education Assistance Authority, and the Kentucky Department of Education Office of Career and Technical Education in compiling data when we requested it and putting the information in a usable format. The authors would also like to thank the Kentucky College Coach program participants who were interviewed for this project, who took time and effort out of their schedules to provide the authors with valuable qualitative information. The authors were impressed by the professionalism of all of the administrators of the Kentucky College Coach program. Without the support of all of these individuals, performing an accurate assessment of the Kentucky College Coach program would not have been possible. AmeriCorps funded this external assessment. AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Introduction and Purpose This assessment is to determine the state of the KCC program as well as delve deeper into characteristics that make the program successful with the “ignored cohort” that the evaluators discussed in the last assessment report. To restate the purpose of this assessment, the program evaluators were contracted through the Kentucky Campus Compact (paid by AmeriCorps) to provide an evaluation of the KCC program and to assess whether the KCC program has been effective in achieving its three main program goals: 1. To increase the number of Kentucky high school students graduating from high school; 2. To increase the number of Kentucky high school graduates choosing to pursue postsecondary education; 3. To help build a college-going culture among Kentucky high school students. AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Introduction and Purpose (Continued) The evaluation of the first two goals uses Quantitative data from KCC and KHEAA. 1) First goal measured: Increasing high school graduation rates 2) Second goal measured: The rate of high school students going to postsecondary education The evaluation of the third goal uses quantitative and qualitative data collected from a variety of sources. 3) Third goal measured: Building of a college-going culture in the Commonwealth of Kentucky. The data collected and provided has allowed the program evaluators to determine the effectiveness and impact of the KCC program. AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Organization and Structure of the KCC Program Kentucky Campus Compact (KCC) Berea College (2011 – 2014) Office of Career and Technical Education (OCTE) (2011 – 2013) Kentucky Higher Education Assistance Authority (KHEAA) AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Research Design and Methodology 1) Qualitative Methodology 2) Quantitative Methodology 3) Spatial Data Analysis – Added for the 2012 – 2014 Evaluation 4) Site Visits – Added for the 2012 – 2014 Evaluation AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Quantitative Data – Information Provided by AmeriCorps and KHEAA The empirical portion of this study uses a quasi-experimental design to estimate the causal impact of the KCC intervention on the target population because random assignment of the population was not available. • The population consisted specifically of high school students. • The “core” group students represent the experimental group in each high school individually coached/mentored by the programs. • The non-core group students did not receive any individual coaching or intervention services. • A control group was used to provide valid measurements to which the results of the KCC program could be compared. • This data was also used for the Geospatial Analysis for this study. AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Quantitative Statistical Tests Used in the Evaluation • Analyses of Variance (ANOVA) and t-tests were used to compare differences in means for statistical significance and to determine comparability of the program and control group. • Chi-square statistics were used to evaluate the success of the program by analyzing the enrollment status in college of the KCC program and comparison group during the follow-up school year. • The study also further categorized the students by the three different administering agencies — Berea College, KHEAA and ATC’s — and compared the results, since the main goal of this study was to capture as much detail as possible about the program and its participants. AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Quantitative Statistical Tests Used in the Evaluation ( Continued) • In order to estimate the effect of the KCC program on students’ academic achievement as measured by GPA, a regression analysis was applied. • Through the use of this approach, it was determined that the KCC program had a positive impact upon high school students graduating from targeted high schools during the 2012-2014 school years after controlling for confounding variables. • Statistical significance was assessed at the 5% level of significance (p-value < 0.05). • All analyses were performed using IBM SPSS statistics version 20. AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Results of Analysis • A population of 1,705 students participated in this study from 29 schools. • Overall, 1,194 students received either individual or group mentoring. • There were 1,073 female (62.9%) and 632 male (37.1%). • Of the 1,194 students who received either group or individualized mentoring 63.7% were female and 36.3% were male. • A total of 511 students did not receive any mentoring. Of those, 61.3% were female and 38.7% were male. AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Results of Analysis ( Continued) Ethnicity Breakdown of Total Population • 80.8% of the students are White/Caucasian • 17.2% are Black or African American • 1.1% are Asian • 0.8% are American Indian • 0.1% are Pacific Islander/Native Hawaiian. • 3.8% identified themselves as being of Hispanic or Latino origin or descent. • Of those who were mentored, the majority was white (79.6%). • The remaining were either black/African American (17.9%), or Asian, American Indian and Pacific Islander/Native Hawaiian (2.5%). • Of those who were not mentored, 83.7% of the participants were white and 15.5% were black, while the remaining 0.8% were Asian, American Indian and Pacific Islander/Native Hawaiian. AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Results of Analysis ( Continued) FRPL (free- and reduced-price lunch program) 87.9% of the mentored students also participated in free-and reduced- price lunch program (FRPL), while 80.4% of non-mentored students participated in FRPL program. AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Results of Analysis ( Continued) Mother’s Education • 38.2% of mothers have a high school education and/or GED graduate • 15.5% have attended college but did not graduate • 8.5% have attended high school but did not graduate • 10.8% have an associate’s degree • 3.3% of mothers have less than a high school education • 6.4% have just a post-secondary certificate or technical diploma • 3.9% have up to an baccalaureate degree • 2.2% have obtained a master’s degree or higher AmeriCorps funds paid for this assessment and report .
Investing in the Future: Evaluating the Kentucky College Coaching Program 2012-2014 Results of Analysis ( Continued) Father’s Education • 38.1% have a high school education and/or GED • 9.3% have attended college but did not graduate • 9.3% have attended high school but did not graduate • 5.3% have post secondary certificate or technical diploma • 2.9% of fathers have an Associate's degree • 2.6 % have up to a baccalaureate degree • 1.6% have obtained a master’s degree or higher AmeriCorps funds paid for this assessment and report .
Recommend
More recommend
