Student Learning By Elizabeth Dhuey (UofT) and Justin Smith (WLU) - PowerPoint PPT Presentation

How School Principals Influence Student Learning By Elizabeth Dhuey (UofT) and Justin Smith (WLU)

Why Examine Principals? • Understanding how students learn and what makes some students learn more than others is important. • Large differences between student achievement between different schools, districts/boards, states/provinces and countries. • Important to understand why these differences exist so that we can improve education. • One avenue to explore is the school leadership.

Compare 2 schools

Compare 2 schools • Blake Street Public School • Pape Avenue Public School

What Causes Differences in Achievement? • Resources • Socioeconomic status of students • Culture • Family and neighborhood influences • Teacher quality • What else? …school leadership?

Motivation • Principals perform vital tasks in schools – Hiring/firing, supervision, scheduling, matching students to teachers, discipline, atmosphere • Recent policy interest in using student achievement data to measure contributions of principals in the U.S. – Race to the Top – 2011 FL Senate Bill 736 “Student Success Act” – Similar laws passed in LA and TN. School boards also using test score growth in evaluations.

Goals of this Project 1. Estimate the “quality” of principals – Using test score data and value added modeling 2. Estimate the impact of switching principals on school inputs and outcomes – Compare incoming to outgoing principals – Effect of incoming principal being higher, lower, or same quality – Academic and non-academic inputs and outcomes

Value added Modeling • Statistical method that estimates principal effectiveness at raising student performance on standardized math and reading tests. • Tries to make apples-to-apples comparisons. • Project a student’s future performance by using past scores. • Compare projection to the student’s actual results. • Difference is the “value” that the principal added or subtracted.

Principal Value-added Modeling • To estimate value added for grade t, obtain t-1 and t-2 grade test scores as well as information about backgrounds of the students. • The t-1 and t-2 grade test scores and background info are used to predict what the student’s grade t math scores will be. • Student A’s actual score in grade t test is 8 points higher than predicted. Student B’s actual score is 2 points lower. Etc. • The value-added estimate is the average of the differences between the actual and predicted scores of all the students in grade t.

Value-added Modeling Value added by a less effective principal Value added by an effective principal

Data • North Carolina administrative data 1998-2009 • 5,388,543 student-year observations • 4415 principals • 1954 schools

Summary Statistics: Students

Summary Statistics: Principals

Estimating Principal Quality ' b 2 + z s ( i , t ) t y it = b 0 + y it - 1 b 1 + x it b 3 + p p ( i , t ) t b 4 + d p ( i , t ) + j s ( i , t ) + h t + e it ' ' y it – math or reading score x it – student-level demographic characteristics ' z s ( i , t ) t ' – school-level demographic characteristics p p ( i , t ) t – principal-level, time-changing characteristics ' d p ( i , t ) – principal effects j s ( i , t ) – school effects h t – year effects e it – error term

Standard Deviation of Principal Quality • The standard deviation shows how much variation or “dispersion” exists in principal quality from the average principal. • Suppose the average principal has a quality of 0 and the standard deviation of principal quality is 0.1. This means that ~68% of all principals have a quality between -0.1 and 0.1 and ~95% have a quality between -0.2 and 0.2.

Variation in Principal Quality

What does this variation mean? • The average student gains ~0.9 standard deviations per year • “Great” principal vs. “average” principal – equivalent to ~1 month more schooling each year. • 90 th percentile vs. 10 th percentile – equivalent to ~5 months more schooling each year

Match Effects • The analysis thus far provides estimates of principal effects that are fixed across schools (and time) • It is plausible that a principal’s effectiveness varies across schools • “Match effects” might arise if a principal’s effect on student achievement depends on – Interactions with the teaching staff – Demographic composition of the student body – Location preference of the principal – Any complementarity between principal and school

Match Effects

Effect of Principal Changes • What happens when a new principal enters a school? – Does this depend on principal quality? • Estimate the effect of a change of principal on school inputs and outcomes.

Effect of Principal Changes What happens when a new principal enters a school? What happens the 1 st year? What happens the first 3 years? 2003-04 2007-08 1998-99 2001-02 2006-07 2008-09 1999-00 2000-01 2002-03 2004-05 2005-06 Principal B – value added .2 Principal A – value added .1 Principal C – value added .1 Principal D – value added .1 Principal E – value added .1 Principal E – value added .05 Principal F – value added .1 Principal G – value added ?

Results • Test Scores: – Brand new principals decrease math and reading scores during the 1 st year and during the first 3 years. • School outcomes: – No effect on crime – Lower value added principals decrease suspension rates – New principals decrease % daily attendance

Results • School inputs: – Higher value added principals decrease % of Nationally Board Certified Teachers and increase % of new teachers – New principals increase teacher turnover rate, increase % of new teachers – New and lower value added principals decrease % of highly experienced teachers.

Overall Implications • Schools are looking for ways to improve school outcomes. • Changing principals is relatively simple and inexpensive • Some evidence that it could work – May not be a tide that lifts all boats – Could consider replacing bottom X% of principals – Offer financial incentives based on value added