Considerations for Transitioning Students from Operational 2% Assessments to General Assessments: Challenges for States Moderator: Martha Thurlow (National Center on Educational Outcomes) Presenters: Sheryl Lazarus (National Center on Educational Outcomes) Wendy Stoica (Ohio Department of Education) Trinell Bowman (Maryland Department of Education) Susan Weigert (U.S. Department of Education, Office of Special Education Programs) Discussant: Marianne Perie (CETE, University of Kansas) National Conference on Student Assessment National Harbor, MD June 20, 2013 National Center on Educational Outcomes (NCEO)
Lessons Learned Chapter Authors • Parker, Gorin, and Bechard • Perie, Fincher, Payne, and Swaffield • Elliott, Kettler, Zigmond, & Kurz • Lazarus & Thurlow** • Elliott, Rodriguez, Roach, Beddow, Kettler, and Kurz • Cohen, Danielson, Stoica, Wothke, and Zhang** • Bechard • Bowman** • Dean and Roberts • Loving-Ryder and Siler • Zigmond, Kloo, Lemons & Lupp • Nagle and Cameto National Center on Educational Outcomes (NCEO)
Lessons Learned in Federally Funded Projects That Can Improve the Instruction and Assessment of Low Performing Students with . Disabilities (Thurlow, M., Lazarus, S., & Bechard, S. (Eds.)) Available at: www.nceo.info • Projects funded by the U.S. Department of Education in 2006-2007 under three funding sources (General Supervision Enhancement Grants, Enhanced Assessment Grants, and Supplemental Funding) compiled their findings in chapters that discussed the lessons learned from their studies on alternate assessments based on modified achievement standards (AA-MAS). • This presentation highlights lessons from three chapters, plus perspectives from OSEP
Sheryl Lazarus National Center on Educational Outcomes (NCEO)
The Characteristics of Low Performing Students with Disabilities Sheryl Lazarus National Center on Educational Outcomes (NCEO) Email: laza0019@umn.edu .
Demographic Characteristics of Low Performing Students with Disabilities • 4 projects looked at demographic characteristics of low performing students. More likely to be: – Male – From a racial/ethnic minority – From a low socio-economic background – Have ELL status • Both students with and without disabilities were low performing. National Center on Educational Outcomes (NCEO)
Disability Categories • 4 projects looked at the disabilities categories of students who might be candidates for an AA-MAS. • All found that a majority of the students had specific learning disabilities. • Other categories: o Intellectual disabilities o Speech and language impairments o Other health impairments o Emotional/behavioral disabilities o Autism o Other disability categories National Center on Educational Outcomes (NCEO)
Opportunity to Learn 4 projects looked at whether students who were candidates for the AA-MAS had the opportunity to learn the content. All found that some students may not have had the opportunity to learn. National Center on Educational Outcomes (NCEO)
Learning Characteristics/Barriers 10 projects found one or more of these learning characteristics/barriers: • Difficulty interacting with print • Difficulty solving problems that require multi-step solutions • Easily distracted • High vocabulary load • Lack of availability of needed accommodations • Limited meta cognition • Need clarification of instructions • Poor organizational skills • Self-monitoring skills • Slower work pace • Text structure (passage length and formatting) • Working memory capacity National Center on Educational Outcomes (NCEO)
Wendy Stoica National Center on Educational Outcomes (NCEO)
AA-MAS Test Development: Item Modifications Wendy Stoica June 20, 2013
Research Questions 1. Do Target and GenEd group students differ on the cognitive variables? 2. Are the cognitive variables accessible via teacher judgment?
Item Modifications Fall 2008 Modifications of Reading Items Bold/Underlined Important elements of the reading passage are bolded or underlined. This will facilitate structured recall of the content passage by AA-MAS students. Questions are interspersed within passages and offset with a box. Boxed Simplified Language Language is simplified beyond that which is typical for universal design. Thought Questions Additional questions designed to focus students’ attention on particular aspects of a passage are interspersed with the passage. These questions are not answered but serve to help students focus attention. Modifications of Mathematics Items Bold/Underlined (1 level) Important elements of the reading passage are bolded or underlined. This will facilitate structured recall of the content passage by AA-MAS students. Modified Graphics Graphics may have been enlarged or simplified in order to increase readability. Relevant Pictures Information in the stem of the item is given as a picture or table to help students organize and understand the information necessary to answer the question. Scaffolding Complex items are decomposed into simpler parts. Simplified Language Language is simplified beyond that which is typical for universal design.
Spring 2009 Modifications of Reading Items Bold or Underlined Important elements of the reading passage are bolded or underlined. This will facilitate structured recall of the content passage by AA-MAS students. Primed Items A specially designed priming item is presented immediately before a test item. The priming item assists the students’ memory of the cognitive process so that they will more readily see the solution to the test item. Cued Items Additional items break up complex questions into a series of simpler steps to reduce the planning load. Graphic Organizers Graphic organizers provide a graphical structure to help students organize their thoughts. Passage Primed Thought questions are introduced before a reading passage to help the students engage in the content of the passage. Modifications of Mathematics Items Bold or Underlined Important elements of the problem are bolded or underlined. This will facilitate structured recall of the content passage by AA-MAS students. Primed Items A specially designed priming item is presented immediately before a test item. The priming item assists the students’ memory of the cognitive process so that they will more readily see the solution to the test item. Cued Items Additional items break up complex questions into a series of simpler steps to reduce the planning load. Relevant Pictures Added icons help students visualize the problem at hand.
Results of interaction of item modifications with cognitive traits for reading Working Memory Executive Function/ Focused Attention Sustained Attention Planning Increases Influence Decreases Influence Bold Increases Influence Cue Graphic Organizers Decreases Influence Prime Passage Prime
Results of interaction of item modifications with cognitive traits for mathematics Working Memory Executive Function/ Focused Attention Sustained Attention Planning Increases Influence Increases Influence Bold Decreases Decreases Influence Increases Influence Increases Influence Cue Influence Decreases Increases Influence Graphic Organizers Influence Decreases Prime Influence
More Research Questions 1. Do the revised items improve access? 2. Does performance exceed chance? 3. Do the revisions improve performance and is the discrepancy reduced between the Target and GenEd group? 4. How do the revisions relate to the student’s cognitive traits?
Conclusions/Recommendations 1. 17% of general education students not identified as having disabilities actually had significant deficits in sustained attention and 11% of the general education students had significant deficits in executive functioning. 2. Students often had deficits only on one, or perhaps two, cognitive traits. 3. Question the ability of classroom teachers to identify those students with disabilities who have specific cognitive processing deficits which has implications for how appropriate data might be collected to inform IEP team decision making. 4. IEP teams require additional training to help them understand that the indiscriminate use of accommodations may actually make tests less accessible for some students with disabilities.
education.ohio.gov
Social Media Ohio Families and Education Ohio Teachers’ Homeroom ohio-department-of-education storify.com/ohioEdDept @OHEducation OhioEdDept
Trinell Bowman National Center on Educational Outcomes (NCEO)
Maryland’s Alternate Assessment Based on Modified Achievement Standards (AA-MAS) What did we learn? 2013 National Conference on Student Assessment National Harbor, Maryland June 2013 Trinell Bowman Maryland State Department of Education 22
Lessons Learned from the Development of the AA-MAS • Cognitive Labs were critical to the development process and for the first time, students were asked to describe the “why” and “how” as they answered test questions. • Online testing offers more standardization of certain accommodations such as the use of the audio test and calculator. • Online testing may offer additional support to students, which they normally may not receive when taking a paper and pencil test. Examples include: eliminating a choice, highlighting critical information or marking a test question for review. 23
Recommend
More recommend
