Student Growth in Accountability: A Colorado Perspective Marie - PowerPoint PPT Presentation

Student Growth in Accountability: A Colorado Perspective Marie Huchton, Supervisor of Accountability Analytics June 2019 1

Origins of Growth & State Accountability in Colorado 2

Historical Context The No Child Left Behind Act (NCLB) of 2001: • Set unrealistic expectations for all students to score proficient, de facto punishing diverse schools with high-impact student populations • Subsequent focus on “bubble kids” who were close to the proficient cut meant students far below or above proficient were receiving less attention Colorado educators championed the need for a measure of student progress that could be used to more fairly evaluate school and district effectiveness 3

Historical Context Education stakeholders wanted a growth measure that: • Reflected the learning gains made by students • Allowed both low and high scoring students to show progress • Took into account multiple years of student’s previous score histories • Was not correlated with student demographics • Could measure student progress towards meeting achievement targets • Reflected the efficacy of school instructional practices in improving student learning 4

Historical Context To fulfill these demands and create a more meaningful state accountability system: • Legislation was passed in 2008 directing the Colorado Department of Education (CDE) to build a measure of student longitudinal growth • Working with the National Center for the Improvement of Educational Evaluation (NCIEA) and the state’s Technical Advisory Panel, CDE helped create the quantile regression-based Colorado growth model • In 2009, additional legislation revised the state accountability system to use student growth as a central measure of school and district effectiveness 5

Overview of Colorado Accountability and Incorporation of Growth 6

Colorado’s system of school and district accountability is primarily designed to provide valid and actionable information regarding the progress of all students toward meeting the state academic standards and prioritize support for schools and districts identified for academic improvements. EVALUATE Local State CDE creates School & District Performance Frameworks . CDE recognizes areas of success and Schools and districts analyze state and local data. identifies schools and districts for additional support based on student academic outcomes. ASSESS NEEDS AND PLAN Local State Schools and districts work with local communities CDE supports the Unified Improvement Planning to assess needs and select strategies to support process for all school and districts informed by state and continuous improvement. local data. IMPLEMENT Local State CDE allocates resources and supports in alignment Schools and districts apply for additional resources with school and district needs identified in Unified and implement selected strategies for improvement. Improvement Plans. Local INTERVENE State The State Board of Education works with CDE and Low performing schools and districts take more districts to determine more rigorous action steps for rigorous action if student performance remains persistently low performing systems through consistently low . Performance Watch .

Evaluating School and District Performance Colorado has built a system of School and District Performance Frameworks (SPFs and DPFs) around the following indicators: student academic achievement, • student longitudinal academic growth, • student academic growth to standard*, and • postsecondary and workforce readiness (PWR) • The scoring system is compensatory and the most weight is given to growth (currently 60% for elementary and middle schools and 40% for high schools and districts) * Note that Colorado is still in the process of re-defining and re-introducing the growth to standard metric following the transition to a new set of state assessments 8

9

Stakeholder Praise for Growth Unlike achievement and other cumulative outcome-based measures, growth: • Measures the impact of the school or district on student performance in comparison to academic peers • Reflects student learning made within the past year while enrolled in the current district or school • Is sensitive to year-to-year changes in school practices • Reduces the focus on “bubble kids”, as students at all levels of achievement are capable of showing success • Is largely unrelated to student demographic composition 10

Stakeholder Criticisms of Growth 1. Normative model is recalculated every year, meaning system as a whole cannot show progress 2. Normative model measures student growth compared to academic peers, not against external performance criterion 3. Students can make high growth over time and still fail to reach grade level proficiency 11

Addressing Stakeholder Criticisms 1. Normative model can be baselined to ensure cross- year results are comparable Colorado has investigated this option in the past and will be doing • so again in the upcoming year 2-3. A growth to standard measure can track whether students previously scoring below proficient are on track to attain higher levels of performance within a given timeframe Colorado has been working on a new “On Track” metric for the • grade 3-8 state content assessments since spring of 2018 and plans to operationalize for 2020 12

Addressing Stakeholder Criticisms 3. Schools and districts with higher growth percentiles are likely to see increased mean scale scores between successive years 13

Addressing Stakeholder Criticisms 3. Sustained higher growth percentiles also result in increased mean scale scores across multiple successive years 14

Evolving Growth and Accountability Over Time 15

Transitioning Assessments • Revised Colorado Academic Standards were implemented between 2010 and 2014 • Colorado joined the Partnership for Assessment of Readiness for College and Careers (PARCC) consortium, and in 2015 administered a new grade 3-11 state assessment called Colorado Measures of Academic Standards (CMAS) • Given the significant shifts in ELA and Math constructs from the previous state assessments, Colorado did not calculate growth or publish state accountability results in 2015 16

Transitioning Assessments • In 2016, growth and performance framework calculations were re-established based on the 2015 and 2016 CMAS results and continue to be used • To try and stem a local parent test refusal movement, legislation was passed in 2016 transitioning the 11 th grade assessment to a nationally recognized college entrance exam with 10 th graders taking the precursor exam • This resulted in Colorado transitioning to the grade 10 PSAT in 2016 and grade 11 SAT in 2017 17

Transitioning Assessments • Additional legislation was passed in 2017 shifting the 9 th grade assessment to align with the college entrance exam, resulting in the 2018 transition to grade 9 PSAT • This staggered implementation of high school assessments has posed significant challenges for calculating growth and incorporating it into recent accountability determinations • CDE has tried to minimize the impact on school and district stakeholders and continue to report growth information wherever possible and appropriate in any given year 18

Transitioning Assessments • Also in 2017, Colorado’s State Board of Education directed CDE to withdraw from the PARCC consortium, shorten the CMAS assessment, and undertake writing new items with Colorado educators • 2018 was the first year of this revised CMAS assessment, and consistency within the content area constructs enabled a seamless continuation of growth calculations from 2015 • Additional assessment transitions are inevitable in future years and CDE plans to continue evolving and adapting growth and accountability calculations as necessary to meet legislative and stakeholder demands 19

The Changing Face of Federal Accountability • During the first two years of the school and district performance frameworks, Colorado was still running separate Adequate Yearly Progress (AYP) calculations to meet federal NCLB requirements • In 2012, CDE was granted a waiver from AYP requirements and was able to use the state system to fulfill both state and federal accountability expectations • After the passage of the Every Student Succeeds Act (ESSA), CDE submitted an ESSA state plan that kept the alignment between the state and federal accountability systems 20

The Changing Face of Federal Accountability • Colorado’s state plan was sent to the US Department of Education in May 2017 as part of the first round of state submissions • Colorado’s state plan was approved in May 2018, the last state to be approved from the initial submission • Extensive follow-up conversations and negotiations took place, but in the end different approaches to calculating achievement and PWR metrics pushed Colorado to once again separate identification calculations for state and federal purposes • In both systems, growth serves as the foundation for the identification calculations 21