Utah School Accountability Standard Setting ANN-MICHELLE NEAL, ED.S ACCOUNTABILITY SPECIALIST UTAH STATE BOARD OF EDUCATION
History of Accountability in Utah School Grading School Grading Accountability Accountability Accountability reporting Reporting System System requirement expanded to established implemented enacted school level 1990 2000 2004 2011 2013 2014 Adequate Governor’s Yearly Progress PACE report reports cards published published
Accountability System - NCLB The previous Achievement accountability system was primarily focused on statewide assessment. ESSA provided an opportunity to change that by adding Graduation Growth additional indicators.
Accountability Design
What’s New? Increased Focus on Equity ◦ English Learner Progress ◦ Access to advanced and career coursework ◦ ACT performance ◦ opportunity for ALL students to take the ACT in 11 th grade. Increased Focus on our Lowest Performing Students ◦ Growth of the lowest performing 25% of students in a school ◦ A portion of points awarded for Graduation will be allotted to students who graduate in 5 years.
Legislative Requirements UTAH SENATE BILL 220 ESSA The board shall engage in a criteria setting process to establish performance thresholds for overall ratings: ◦ A system for assigning a school an overall rating based on evaluating the school's performance against specific criteria. (Section 53A-1-1113.5)
From Normative to Criterion Normative • 2011-2016 • Limited the % of schools earning an A or B; cut scores for school grades could adjust in any given year • Created unpredictability for schools Criterion • 2018 and on • Performance thresholds established through a standard setting process • Cut scores for school grades are fixed; do not shift based on schools’ performance
School Letter Grades: No More Moving Targets Letter grades are still in place (legislative requirement), but score thresholds are fixed
Legislative Requirements Labels for the performance levels are defined in statute: ◦ “A” represents an exemplary school ◦ “B” represents a commendable school ◦ “C” represents a typical school ◦ “D” represents a developing school ◦ “F” represents a critical needs school (Section 53A-1-1105) However, the statute does not specify what the threshold scores are for each performance level. In other words, what does it mean to be an “A” or “B” school?
Utah School Letter Grades
Legislative Requirements Standard setting process to arrive at overall determinations. The board shall engage in a criteria setting process to establish performance thresholds for overall ratings: ◦ A system for assigning a school an overall rating based on evaluating the school's performance against specific criteria. ◦ In establishing the performance thresholds the board shall solicit and consider input from: ◦ legislators; ◦ the governor; ◦ representatives from local school boards; ◦ other representatives from school districts, including superintendents; ◦ other representatives from charter school governing boards; ◦ teachers; and, ◦ parents.
Accountability Standard Setting School School Performance Profiles and Level Empirical Descriptors Data Legislative Stakeholder Requirements Expertise Standard Setting Meeting Threshold Scores That Assign Schools to Performance Levels
Procedure Develop policy descriptors for each level Develop school performance level descriptors Convene panel or other group (e.g. task force, advisory group) to: ◦ Evaluate school profiles with respect to descriptors ◦ Review data on achievement patterns ◦ Set Indicator thresholds and overall thresholds Evaluate Document Produce material to support interpretation and use
Setting Policy Descriptors On May 16, a committee of key legislators and state board members and school leaders convened to review and revise the Policy Descriptors (PDs) for Utah’s school accountability system. PDs are high-level statements about each performance level. They should be linked to Utah’s goals and policy priorities and identify the most critical outcomes that are valued and considered non- negotiable. PDs were passed on to School-Level Performance Descriptor Development Group
Setting School Level Performance Desc. On May 17, members of the Assessment and Accountability Policy Advisory Committee (AAPAC) convened to work on the school performance level descriptors (SPLDs) for Utah’s school accountability system. SPLDs are more detailed descriptions of the characteristics of schools in each performance level. PDs and SPLDs were passed on to Standard Setting Committee
Policy and School Performance Desc.
Setting Performance Levels On May 18, a committee of AAPAC members, policymakers, parents, educators, association representatives, and technical experts met to recommend performance level threshold scores for school ratings in Utah’s school accountability system. The threshold scores were developed to assign overall letter grades to schools based on their performance on various indicators specified in SB 220.
Role of Threshold Scores Threshold Scores F D C B A School School School School School School Performance Expectations “Top” “Just barely” C School B School
Method Two key tools used in performance level setting ◦ Ordered school profiles (OSP): list of all schools in Utah, ordered by the % of total points earned on the legislatively-mandated accountability indicators in 2016. ◦ Detailed school profiles: a report that includes additional empirical data about a given school, including test participation rates, detailed breakdowns of each accountability indicator by subject area, and historical demographic and performance data, for all students and by subgroups.
Ordered School Profile Lists
Detailed School Profiles
Rounds to Establish Consensus • Locate probable ranges 1. Range • Independent review of Finding the data • Pinpoint threshold scores 2. within each probable Judgements range • Discussion groups
Summary of Outcomes GRADES 3-8 HIGH SCHOOL
Summary of Outcomes 100% 7% 13% 90% 24% 80% 70% 34% A Schools 60% B Schools 50% C Schools 49% 40% D Schools 36% 30% F Schools 20% 15% 13% 10% 5% 4% 0% Grades 3-8 High School
Thank You Ann-Michelle Neal ann-michelle.neal@schools.utah.gov
Recommend
More recommend
