ESSA Innovative Assessment Designs: Examples from Science Brian - PowerPoint PPT Presentation

ESSA Innovative Assessment Designs: Examples from Science Brian Gong, Center for Assessment April McCrae, Delaware Department of Educa;on Michelle Center, California Department of Educa;on Karen Kidwell, Kentucky Department of Educa;on CCSSO Na;onal Conference on Student Assessment June 28, 2017 Aus;n, TX

Assessment Challenges and Solution Approaches for Next Generation Science Standards: Domain de<inition, claims, evidence speci<ications Brian Gong Center for Assessment Presenta;on in the session on “ESSA Innova;ve Assessment Designs: Examples from Science” CCSSO Na;onal Conference on Student Assessment June 28, 2017 Aus;n, TX

Overview • Descrip;on of NGSS and assessment context • Three issues, some possible solu;on approaches • Summary NGSS assessment issues & approaches - 6/28/17 3

Next Generation Science Standards • Designed to replace previous na;onal/widely adopted science standards (Na;onal Science Educa;on Standards, 1996; Science for All Americans, 1989) • Framework for K-12 Science Educa:on (NRC, 2012) • NGSS Standards , including Performance Expecta;ons (NGSS, 2013 [Achieve and 26 states]) • Other materials, e.g., Developing Assessments for the Next Genera:on Science Standards (NRC, 2014); Assessment Framework /aligned test item cluster (WestEd for CCSSO Science Item Assessment Collabora;ve, 2015) NGSS assessment issues & approaches - 6/28/17 4

Summary of NGSS structure and content • Grades K-high school • Three dimensions – Science and Engineering Prac;ces (SEP) • 1) Asking ques;ons (for science) and defining problems (for engineering); 2) Developing and using models; 3) Planning and carrying out inves;ga;ons; 4) Analyzing and interpre;ng data; 5) Using mathema;cs and computa;onal thinking; 6) Construc;ng explana;ons (for science) and designing solu;ons (for engineering); 7) Engaging in argument from evidence; 8) Obtaining, evalua;ng, and communica;ng informa;on – Disciplinary Content Ideas (DCI) • 11 Core ideas (4 in life sciences; 4 in physical sciences; 3 in earth and space sciences) • 39 sub-ideas • Each sub-idea is elaborated in a list of what students should understand about that sub- idea at the end of grades 2, 5, 8, and 12. – Cross-Cugng Concepts (CCC) • 1) Paherns; 2) Cause and effect: Mechanism and explana;on; 3) Scale, propor;on, and quan;ty; 4) Systems and system models; 5) Energy and maher: Flows, cycles, and conserva;on; 6) Structure and func;on; 7) Stability and change NGSS assessment issues & approaches - 6/28/17 5

NGSS structure and content - PEs • Performance Expecta;ons developed as part of NGSS to define assessment targets in terms of what students should know and be able to do • Each PE combines selected DCI, SEP, and CCC • PE developed within life science, physical science, and earth & space science content areas; oken mul;ple PE for each discipline area by grade NGSS assessment issues & approaches - 6/28/17 6

Sample Performance Expectation Source: hhps://www.nextgenscience.org/resources/how-read-next-genera;on-science-standards NGSS assessment issues & approaches - 6/28/17 7

Assessment context of NGSS • Adopted by 17 states and DC thus far (2017). Several other states using related content frameworks. • Four states have launched opera;onal NGSS-aligned tests (IL, KA, NV, DC). A few NGSS states are field- tes;ng new assessments in 2016-17 and/or 2017-18; most states are in assessment design stage. • Federal Every Student Succeeds Act (ESSA, 2015) requires states to assess science at least once in grade spans 3-6, 6-9, and 10-12. Assessment must be aligned to state’s content standards—no federal men;on of NGSS. NGSS assessment issues & approaches - 6/28/17 8

ESSA <lexibility supports possible NGSS assessment designs ESSA gives states authority for several flexibili;es that are relevant to possible NGSS • assessment designs – ESSA allows states to choose their content standards; USED may allow states to assess a subset of the “instruc;onal learning content standards” as PE are a subset of the possible combina;ons of the NGSS SEP/DCI/CCC – ESSA explicitly states that “porqolios, performance events … ” may be used in state assessments; while not prohibited before, this may be read as some encouragement – ESSA requires states to have a valida;on argument; not clear how recep;ve USED/Peer Review might be to valida;on arguments around more limited (less generalizable) claims. These may be necessary to address the sparse sampling in the NGSS PE – ESSA allows “banking” of math scores of grade 8 students who take a state’s high school math test – ESSA allows states to use interim assessments to produce a summa;ve score, and to use a “locally selected, na;onally recognized high school assessment” in lieu of the state assessment. These may provide models for how to handle innova;ve assessments that have similar innova;ve aspects (e.g., through-course; mul;ple, non-strictly comparable assessments) – ESSA allows a Demonstra;on Authority around competency-based assessments, which may provide guidance for how to handle similar innova;ve NGSS assessments, such as ones based on claims of criterion mastery and individualized administra;on condi;ons (e.g., ;ming: “move on when ready”) NGSS assessment issues & approaches - 6/28/17 9

Rest of presentation • Present an evidence-centered design approach to test development to define terms and show rela;onships of parts to each other and to valida;on argument • Discuss three domain defini;on issues posed by the NGSS, and some possible solu;on approaches in terms of domain defini;ons, claims, and evidence • For selected possible solu;on approaches, discuss a few measurement challenges and possible solu;ons NGSS assessment issues & approaches - 6/28/17 10

Evidence-Centered Design summary • Assessment is intended, by design , to gather evidence to support making a claim about student performance in rela;on to something – Define the “something”—the domain/”construct” – Define the claim(s) – Define how the claims will be reported – Define what evidence is necessary to support the claims – Define what informa;on will be gathered and how it will be processed to become evidence – Design how the informa;on-gathering is comprehensive, prac;cal, fair, etc. • In prac;ce, usually itera;ve NGSS assessment issues & approaches - 6/28/17 11

Issue 1: Very large, complex domain • NGSS defines – 8 Science and Engineering Prac;ces, 39+ Disciplinary Core Idea sub-parts, 7 Cross-Cugng Concepts = 2,184 possible combina;ons [616 if restricted to 11 DCI] • Challenge: The possible domain defined by the complete crossings of SEP x DCI x CCC is too large to assess conven;onally (and probably too large to learn in current condi;ons) • Solu;on approach 1: Reduce domain for learning – 1A: Eliminate elements or combina;ons (many states: use PEs) – 1B: Combine elements (e.g., “Inquiry” from several SEP) NGSS assessment issues & approaches - 6/28/17 12

Large, complex domain – 2 • Solu;on approach 2: Reduce domain for assessment – 2A: Eliminate elements or combina;ons (NGSS = PEs) – 2B: Combine elements, e.g., combine 8 SEP into 2 categories for repor;ng and test design • Solu;on approach 3: Reduce assessment – 3A: Assess a limited por;on of the domain (e.g., assess DCI, not SEP) • Solu;on approach 4: Spread out assessment – 4A: Rotate coverage over years/forms – 4B: Distribute coverage over forms, matrix sample over students – 4C: Assess over mul;ple years NGSS assessment issues & approaches - 6/28/17 13

Issue 2: NGSS PEs weak for domain de<inition • Performance Expecta;ons developed as part of NGSS to define what should be assessed and specify what evidence (i.e., what students should be able to do) • Challenge 1: The set of NGSS PEs are so sparsely sampled it is difficult to have enough evidence to generalize about individual PEs or their componen;al DCI/SEP/CCC within or across grades • Challenge 2: The set of NGSS PEs do not have an obvious or documented structure of how the combina;on of them relate to a claim about “science learning” in a single grade or across grades NGSS assessment issues & approaches - 6/28/17 14

PE coverage of SEP x DCI content area, grades 3-5 NGSS Scientific &Engineering Practices (SEP) Addressed in the Performance Expectations Grade AQDP DUM PCOI AID UMCT CEDS EAE OECI Grade 3 PS X X LS X X X X ESS X X X Grade 4 PS X X X X LS X X ESS X X X X Grade 5 PS X X X X LS X X ESS X X X X 3-5 ETS X X X Disciplinary Core Idea Areas Scien;fic and Engineering Prac;ces 1. PS = Physical Sciences 1. Asking ques;ons (for science) and defining problems (for engineering); 2. LS = Life Sciences 2. Developing and using models; 3. ESS = Earth and Space Sciences 3. Planning and carrying out inves;ga;ons; 4. ETS = Engineering, Technology, and 4. Analyzing and interpre;ng data; Society 5. Using mathema;cs and computa;onal thinking; 6. Construc;ng explana;ons (for science) and designing solu;ons (for engineering); 7. Engaging in argument from evidence; 8. Obtaining, evalua;ng, and communica;ng informa;on NGSS assessment issues & approaches - 6/28/17 15