Immediate and Longitudinal Effects of the Immediate and Longitudinal Effects of the Tennessee Voluntary Pre-K Program Tennessee Voluntary Pre-K Program on on Student Student Cognitive and Non-Cognitive Outcomes Cognitive and Non-Cognitive Outcomes Mark W. Lipsey, Kerry G. Hofer, Dale C. Farran, Mark W. Lipsey, Kerry G. Hofer, Dale C. Farran, Nianbo Nianbo Dong, Carol Bilbr Dong, Carol Bilbrey ey Peabody Research Institute Vanderbilt University SREE, Washington DC, March 2014
Research Team • Principal Investigators Senior Research Associate • – Mark Lipsey – Kerry Hofer – Dale Farran Research Associate • • TN Dept of Education Partner – Nianbo Dong – Connie Casha Research Analysts • • Project Manager – Chris Ham – Rick Feldser – Carol Bilbrey – Ilknur Sekmen • Project Coordinator TN Education Consultant – Janie Hughart • – Bob Taylor • Regional Coordinator Child assessors across the state – Diane Hughes • Funded by the Funded by the Institute for Ed Institute for Education Sciences Grant #R305E090009 ucation Sciences Grant #R305E090009
The Tennessee Voluntary Pre-K Program (TN-VPK) • 4-year old children – priority for children who qualify for the FRPL programs – after that, children at-risk for other reasons. • Full day program – 5.5 hours per day; minimum of 180 days per year. • State licensed teachers with early childhood education endorsements. • Aide in each classroom; class size maximum of 20. • Approved age-appropriate curriculum aligned with the Tennessee ECE Developmental Standards. • 934 pre-k classrooms in 133 of the 136 Tennessee school systems across all 95 Tennessee counties; serves more than 18,000 children.
Two Components of the Evaluation Focus of this • RCT (Randomized Control Trial) presentation – Full Sample • Randomization to wait lists in oversubscribed schools • 2 cohorts; 3025 students across TN – Cohort 1: Pre-k year = 2009-10; N=1764 – Cohort 2: Pre-k year = 2010-11; N=1261 • Tracking through the state data system to 3 rd grade – Intensive Substudy of Consented Children (ISS) • Assessed each year by the research team • 1076 students in the analysis sample • 773 in TN-VPK; 303 not in TN-VPK (control) • RDD (Age-Cutoff Regression Discontinuity Design) – Probability sample of TN-VPK classrooms in 4 regional groups – Over 3400 students to date – potential of 2400 more – Directly assessed at the beginning of PK/K as applicable
RCT Participants Cohort 1: Pre-k year 2009-10 Cohort 2: Pre-k year 2010-11 Full sample: 79 participating schools in 29 school districts across TN Intensive substudy sample: 76 schools in 21 districts
Achievement (Cognitive) Outcomes [Available only for the Intensive Substudy Sample] Woodcock Johnson III Scales • Literacy : Letter-Word Identification, Spelling • Language : Picture Vocabulary, Oral Comprehension, Passage Comprehension (K and 1 st grade only) • Math : Applied Problems, Quantitative Concepts, Calculation (K & 1 st grade only) • Overall WJ Composite – Mean W score across all WJ scales • Composite of 6 scales used at end of pre-k and beyond • Composite of 8 scales used at end of K and beyond
Characteristics of the Children in the ISS Analysis Sample Charact Characteristic ristic Mean Mean Age start of pre-k year 4.3 Male 45% FRPL 100% Race/ethnicity White 57% Black 23% Hispanic 18% Asian and other 2% Non-native English 20% speaker
Preschool Experience of ISS Control Children 49% 50% 45% 60% at home 40% 35% Percent of Nonparticipants 15% private 30% childcare; 25% 11% Head Start 20% 10% mixed 15% 15% 11% 10% 6% 6% 5% 5% 4% 5% 0% Home with Home with Homebased Private Child Head Start Mix (some Mix (no Unknown parent Other Care Care formal) formal)
Analysis Plan for ISS Data: Propensity Scores • Variables included in the Propensity Score • Gender • Working Parents • Race/ Ethnicity • Library Card Use • ESL • Newspaper Subscription • Age at Pretest • Magazine Subscription • Test Lag • Mother’s Education • Test Interval • Pretest • Cohor t • Propensity Score as a Level 1 covariate in all analyses • Consent rate for T and C groups on each randomized list, and interaction, used as Level 2 covariates • Multiple imputation for missing data.
VPK Effects at End of Pre-k on the WJIII Composite W-Scores VPK effect size = .33 1.2 .33 SD 1.0 (45% TN‐VPK Participants increase) Nonparticipants 0.8 Standard Deviation 0.6 .73 SD 0.4 Control 0.2 group pre- post gain in 0.0 SD units Pretest Posttest ‐0.2
VPK Effects at End of Pre-k on the WJII Subscales % Im provem ent Outcom e Measure Effect Size for Pre-K Overall Com posite .33* 45% Literacy Letter-Word ID .46* 89% Spelling .25* 30% Language Picture Vocabulary .20* 83% Oral Comprehension .10* 26% Math Applied Problems .14* 21% Quantitative Concepts .31* 49% * p <.05
Effects at End of PK, K, & 1 st on WJII Subscales 1 st ES Outcom e Measure PK ES K ES Overall Com posite (6) .33* .04 -.09 Overall Com posite (8) -- .00 -.09 Literacy Letter-Word ID .46* .04 -.06 Spelling .25* .01 -.09 Language Picture Vocabulary .20* .08 .02 Oral Comprehension .10* .09 -.07 Passage Comprehension -- -.11† -.07 Math Applied Problems .14* .02 -.05 Quantitative Concepts .31* -.07 -.21* * p <.05 Calculation -- -.07 -.06 † p <.10
WJIII Composite Achievement: Native English Speakers in Pre-K Classrooms with the Largest Gains
WJIII Composite Achievement: Native English Speakers in Pre-K Classrooms with the Smallest Gains
WJIII Composite Achievement: Non-native English Speakers in Pre-K Classrooms with the Largest Gains
WJIII Composite Achievement: Non-native English Speakers in Pre-K Classrooms with the Smallest Gains
Non-Cognitive Outcomes [Available for ISS sample & portions of full sample] Teacher ratings at beginning of K and end of 1 st grade • Cooper-Farran Behavioral Rating Scales (ISS sample) – Work-Related Skills – Social Behavior • Academic Child Behavior Record (ISS sample) – Preparation for Grade – Peer Relations – Behavior Problems – Feelings About School Measures from the State Education Information System • Retention in grade • Attendance • Disciplinary Action • ELL status (too early to assess; still collecting data) • IEPs (too early to assess; still collecting data)
Effects on Teacher Ratings at Beginning of K and End of First Grade (ISS sample) End of 1 st Beginning Outcom e Measure of K ES Grade ES CF Social Skills .30 .06 CF Work-Related Skills .37* -.03 ACBR Preparation for Grade .44* -.10 ACBR Peer Relations .19* .08 ACBR Behavior Problems -.39* -.05 ACBR Feelings About School .05 -.06 * p <.05
Grade Retention in Kindergarten 10% 9% 8% Percent Retained in K No VPK 6.8% 7% No VPK 6.2% 6% VPK 5.0% 5% VPK 4.1% 4% 3% 2% 1% 0% ISS Sample ( p <.05) Full Sample ( p <.05)
Attendance in Kindergarten & 1 st Grade TN-VPK TN-VPK Participants Nonparticipants p for Grade and Sam ple Mean Days Mean Days difference K attendance, ISS sample 160.0 160.4 .534 K attendance, Full sample 158.0 156.6 .0 8 4 1 st grade attendance, Cohort 1 164.0 160.6 .0 17
Recorded Disciplinary Action in Kindergarten & 1 st Grade TN-VPK TN-VPK Participants Nonparticipants p for Grade and Sam ple % with Record % with Record difference Kindergarten, ISS sample 0.9 0.8 .252 Kindergarten, Full sample 1.1 1.5 .196 1 st grade, Cohort 1 2.6 2.2 .633
Summary of TN-VPK Effects Found So Far (1) Achievement (Cognitive) Outcomes Achievement (Cognitive) Outcomes • Nontrivial positive effects at the end of pre-k on all measures; largest on literacy scales • Largest gains were for non-native English speakers • Effects were not sustained through kindergarten and first grade
Summary of TN-VPK Effects Found So Far (2) Non-Cognitive Outcomes Non-Cognitive Outcomes • Teacher ratings – Nontrivial positive effects at beginning of K on ratings of preparedness for grade, work related skills, and classroom behavior – Effects were not sustained through end of first grade • School records – Fewer TN-VPK participants retained in kindergarten – More days attended by TN-VPK participants in K and 1 st grade – No difference for recorded disciplinary actions in K and 1 st grade – Sufficient data on ELL and IEP status are not yet available
Thanks! More information available at: http://peabody.vanderbilt.edu/research/pri/projects/by_content_area/tennessee_state_pre-k_evaluation/ or Google “Peabody Research Institute” Contact email: mark.lipsey@vanderbilt.edu
Recommend
More recommend
