Fo Formative Eva valuation of New Ham Hampshir hire' e's Per erformanc ance e Asses essmen ent t of of C Com ompetency E Education on ( (PACE) Gene Wilhoit, Center for Innovation in Education Arthur Thacker, HumRRO Paul Leather, New Hampshire Dept. of Education Susan Lyons, Center for Assessment (Access the Evaluation Here)
Formative Evaluation of New Hampshire’s Performance Assessment of Competency Education (PACE) Paul Leather, Deputy Commissioner New Hampshire Department of Education 2
NH PACE -- Performance Assessment of Competency Education • PACE is a learning system designed to Current Status: capitalize on the latest advances in • 3 years of waiver implementation, and understanding how people learn. The • ESSA planning to result in Waiver request, goal is to structure learning opportunities subject to Sec. 1204 implementation by that allow students to grapple with Secretary DeVos gaining meaningful knowledge and skills at a depth of understanding that can they can transfer to new real-world situations. • Our goal is to scale PACE across our state of New Hampshire and to inspire other states to accept a similar challenge. 6/28/17 3
PACE – What are the basics? • First-in-the-nation USED approved alternative assessment and accountability system pilot • Reduced level of standardized testing along with locally developed common performance assessments • Smarter Balanced/SAT given once in elementary school, once in middle school and once in high school (three grades instead of seven) • In all other years, administration of common and local performance assessments developed by the districts and validated at the state level
Engaging the US Department of Education March 27, 2012 Sept. 23, 2014 1 st Meeting March 3, 2015 2 nd Meeting with Arne Approval! with Arne Duncan Theory PD Design Implementation of Action NH Process 5
What has changed with PACE? NCLB/Waiver PACE • Districts must meet guard rails before • All districts held accountable at once to joining PACE state system • PACE Assessments developed locally – • State level assessment – last two in NH System is State AND Local are nationally developed • Annual determinations based on multiple • Annual Determinations based on State measures Assessments solely • Local communities and educators OWN • There often are two systems – state and the single system of accountability local • Intensive Educator Development occurs PRIOR to joining PACE, and continues as • Educator development tied to school PACE involvement evolves improvement after assessment results are released 6
Assessment Continuum Student- Designed CCSS Performance Extended Examples Projects Assessments Based Items Performance Traditional (Envision, NY (SBAC & & Tasks Tasks Tests Performance PARCC) (MARS, (SCALE, Standards Consortium, BAM) EPIC, ILN) Singapore, IB) Narrow Assessment Assessments of Deeper Learning Systems of Descriptions Performance Standardized Longer, deeper Standardized, standardized tasks that investigations,(2-3 tests with m-c multiple- performance require months) & & open-ended choice tests of items and students to items + short exhibitions, routine skills tasks (1 day to formulate and (1-2 day) including 1 week) that carry out their performance graduation measure key own inquiries, portfolios, tasks of some concepts in analyze & requiring students applied skills thought- present to initiate, design, provoking items findings, and conduct, analyze, that require (sometimes) revise, and extended revise in present their work problem solving response to in multiple Linda Darling-Hammond feedback modalities
What is PACE? – Water Tower Proposal! Geometry PACE Common Task • The Problem: Your town’s population is predicted to increase over the next 3 years. As one of the town planners, you are asked to address this issue in terms of the town’s water supply. In order to meet the future needs of the town, you need to make a proposal to add a water tower somewhere on town property that will be capable of holding 45,000 ± 2,000 cubic feet of water. The town is looking for a water tower to contain the most amount of water while using the least amount of construction material. • Student Task: Your job is to prepare a proposal that can be submitted to the town planning committee. Using your calculations of surface area and volume for the two designs, describe and analyze the characteristics that lead you to a final recommendation. 6/28/17 8
PACE: Local, State and Common Assessments Grade ELA MATH SCIENCE K-2 Local PBA Local PBA Local PBA 3 Smarter Balanced Common PACE PBA Local PBA 4 Common PACE PBA Smarter Balanced Common PACE PBA 5 Common PACE PBA Common PACE PBA Local PBA 6 Common PACE PBA Common PACE PBA Local PBA 7 Common PACE PBA Common PACE PBA Local PBA 8 Smarter Balanced Smarter Balanced Common PACE PBA 9 Common PACE PBA Common PACE PBA Common PACE PBA 10 Common PACE PBA Common PACE PBA Common PACE PBA 11 SAT SAT Common PACE PBA 12 Local PBA/Capstone Local PBA/Capstone Local PBA/Capstone Exhibitions Exhibitions Exhibitions
Who is PACE -- Which Districts/Schools? Tiers 2 and 3 – Tier 1 -- Additional Tier 1 -- Additional Tier 1 -- Year 3 (2016-17) Year 3 (2016-17) Year 1 (2014-15) Year 2 (2015-16) • Rochester • Concord • SAU 35 White Allenstown • • Fall Mountain Mountains • Sanborn • Monroe • Plymouth Regional (as negotiated • Pittsfield • SAU 23 No with USED) • Epping Haverhill • Seacoast SAU 58 Groveton • • Souhegan HS Charter Manchester • Rollinsford • Ashland • • SAU 39 Amherst and Mont Vernon VLACS • 6/28/17 10
Organization to PACE Scaling Tier1 Tier 2 Tier 3 Implementing Districts Preparing Districts Planning Districts • Data Collection • Performance • Develop CBE Assessment • Calibrating and Scoring • Competencies, Development Instruction, Assessment • Body of Work Protocols • Implementation Process and Grading Adopted from NH DOE graphic
PACE System Design is based on High Performing Professional Learning Communities • Where educators work together to continuously: • Review competency expectations, aligned with college and career outcomes • Evolve instructional practice to support student learning of competencies • Develop strategies to enhance personalized learning where student agency is a primary student outcome • Improving the Assessment system to measure student achievement and growth related to competencies • Educators review student work on performance tasks regularly 6/28/17 12
Formative Evaluation of New Hampshire’s Performance Assessment of Competency Education (PACE) Presentation for NCSA Presenters : Arthur Thacker, HumRRO Sunny Becker, HumRRO June 29, 2017 Headquarters: 66 Canal Center Plaza, Suite 700, Alexandria, VA 22314-1578 | Phone: 703.549.3611 | www.humrro.org
Presentation Overview • What is HumRRO? • Brief Reminder of Select PACE Characteristics • Evaluation Design • Key Evaluation Findings • Evaluation Recommendations • The Story of PACE 14
Human Resources Research Organization (HumRRO) • Independent, non-profit U.S. based organization • Approximately 100 employees • Professional staff mostly education researchers, I-O psychologists, statisticians, and measurement experts • Expertise: • Program evaluations at local, state, and national levels Evaluations of school improvement initiatives Large scale, long range evaluations of state and national education programs Quality assurance for National Assessment of Educational Progress (NAEP) Independent, real-time replication of assessment results as a quality control check for several states 15
Select PACE Characteristics • Competency education • Performance assessments integrated into students’ day-to-day work • Tier 1, 2, and 3 districts • Common and local tasks • Task bank 16
Evaluation Design ( 1 of 2) • April 2016 – February 2017 • Formative evaluation • Theory of Action • Data sources • observations of major PACE events (e.g., task development and scoring sessions); • interviews with students, parents, teachers, principals, and district leaders; • classroom observations; • surveys of teachers; • review of materials provided to and by Tier 1 districts; and • analyses of score data. • Accumulated evidence regarding each claim in the Theory of Action over the course of the evaluation. 17
PACE Theory of Action PA 18
Key Evaluation Findings (1 of 3) • Report includes detailed findings on each claim within the Theory of Action • Buy-in • Educators determine what is assessed, how it is assessed, and how the tasks are scored • Assessments integrated into instruction • Replaces Smarter Balanced assessments and requires deep knowledge • Collaboration • Cross-district activities (task development sessions, professional development, scoring sessions, standard-setting, etc.) • LibGuide (online repository) 19
Key Evaluation Findings (2 of 3) • Teaching & Learning • Increased depth of knowledge • “Testing to what is taught” vs. “teaching to the test” • Learning while testing • Context • Previous experience with competency-based education • District size • Substantial work for teachers 20
Recommend
More recommend
