Creating Personalized Education Systems: Growth Modeling and - PowerPoint PPT Presentation

Creating Personalized Education Systems: Growth Modeling and Students with Disabilities Xin Wei & Renée Cameto SRI International June 19, 2011

Background  Poor academic performance of students with disabilities  Lack of understanding of reading and math growth trajectories for students with disabilities limit our ability to build sound accountability and assessment systems  This study using SEELS data aims at building personalized growth targets to meet high expectations held for students with different types of disabilities 2

Reading Achievement Level of Students with Disabilities  Students with LD had lower levels of reading foundational skills (Jenkins & O’Connor, 2002) and about 80% of them experience serious problems learning to read (Kavale & Reece, 1992).  Students with speech/language impairments have lower average reading scores than students without disabilities (Catts, Bridges, Little, & Tomblin, 2008).  Students with hearing impairments also have significant challenges with reading and are reported to have below average performance than typically developing students (Wolk & Allen, 1984).  Students in every other disability category (e.g., mental retardation, visual impairments, traumatic brain injury) also have documented reading scores well below their peers without disabilities (Blackorby et al., 2004). 3

Math Achievement Level of Students with Disabilities  Large gaps exist between students with disabilities and their peers in the general population. 30% of students with disabilities score above the 50th percentile in mathematics calculation, whereas 40% score below the 25th percentile (Blackorby et al., 2002).  Students with learning disabilities performed below students in the general population but typically outperformed students with mild intellectual disability in math (Caffrey & Fuchs, 2007; Gresham, MacMillan, & Bocian, 1996).  Students with emotional disturbance had lower academic scores than students with learning disabilities (Sabornie, Cullinan, Osborne, & Brock, 2005).  Students with mental retardation or multiple disabilities have the lowest scores in calculation, with about three-fourths of them receiving scores in the lowest quartile (Blackorby et al., 2002). 4

SEELS Data Two-stage sampling procedure  265 LEAs and special schools (size, region, wealth)  11,500 students (800-950 per disability category)  4,000 schools  7,000 teachers 5

Timing of Data Collection Wave 1: Wave 2: Wave 3: Data Collection 2000-01 2001-02 2003-04 Parent interview Summer of 2000 Winter-Spring Winter-Spring Direct assessment/ Winter-Spring Winter-Spring Winter-Spring student interview Language arts Winter-Spring Winter-Spring Winter-Spring teacher survey School program Winter-Spring Winter-Spring Winter-Spring survey School Winter-Spring Winter-Spring Winter-Spring characteristics survey Transcript Winter-Spring Winter-Spring Winter-Spring 6

Measures  Woodcock-Johnson III letter word identification, passage comprehension, calculation, and math applied problems.  W score is appropriate for measuring growth. The W score for each subtest is centered on a mean of 500 to approximate the average performance of a 10-year-old child (Woodcock et al., 2001). A student’s disability categorization was obtained from school district rosters  when the sample was drawn in 1999 and included the 12 disability categories under IDEA.  Age is measured as years and months when a student took the WJ III test.  Gender, race, and socioeconomic status (mother’s education, father’s education, and family income) are covariates.  General education students growth curves were extracted from WJ III examiner’s manual (Mather, Shrank, & Woodcock, 2007). 7

Analysis  A quadratic growth curve model  A two-level HLM model 1. Level-1 HLM model is the within-person model, which included repeated measures of student scores in reading or math across three waves predicted by student’s age and age squared at each wave. 2. Level-2 model is the between-person model, which estimated the differences in scores between students from different disability categories, gender, SES, and race-ethnicity. 3. Intercept, age, and age squared are random effects, and disability category and other covariates are fixed effects. 4. Restricted maximum likelihood estimation with an unstructured covariance structure was specified. 5. Age was centered by mean. 8

Reading Growth Trajectories by Disability

Math Growth Trajectories by Disability 540 540 Learning Disabilities 530 530 Speech Impairment 520 520 Intellectual 510 510 Disability Emotional 500 500 Disturbances 490 490 Applied Problems Hearing Impairment Calculation 480 480 Visual Impairment 470 470 Orthopedic Impairment 460 460 Other Health Impairment 450 450 Autism 440 440 Traumatic 430 430 Brain Injury 420 420 Multiple Disabilities 410 410 General Population 400 400 7 8 9 10 11 12 13 14 15 16 17 7 8 9 10 11 12 13 14 15 16 17 Age Age

Results  Reading and math growth rates decrease with age.  Large variations in achievement levels across different disability categories. Students with speech/language impairments or visual impairments exhibited performance about 1 standard deviation higher than their peers classified with intellectual disabilities or multiple disabilities.  The slope and acceleration of growth were similar between students with learning disabilities and students in most other disabilities categories. When examining the slope of reading achievement across disability categories at age 12.67, we found students with speech/language impairments, hearing impairments, and autism grew significantly more slowly than in students with learning disabilities on both reading measures. Autism grew significantly slower than students with learning disabilities and students with speech/language impairments decelerated significantly faster than students with learning disabilities on both math outcomes. 11

Personalized Growth Target  Normative growth trajectories were plotted on the previous graphs.  Personalized growth targets take into account students age, the complexity of the skills tested, and student background characteristics (e.g., disability type, gender, race, socioeconomic status, and ethnicity). The user can input student background characteristics into the system and get an estimated score a child should achievement if she or he is on track.  This system plots and compares individual trajectories to personalized growth targets, which provides states, schools and teachers information on whether or not a student is on track and can help students with disabilities to achieve the high expectations set for next generation learners. 12

An example 540 530 520 510 500 490 Applied Problems 480 470 460 450 440 430 420 410 400 7 8 9 10 11 12 13 14 15 16 17 Age Estimated Growth Trajectories for a White Female Child with LD from an average SES family The Actural Scores This Child Got General Population Trajectories

Additional Information  Corresponding author: Xin Wei, PhD, Research Analyst SRI International, 333 Ravenswood Avenue, Menlo Park, CA 94025; E-mail : xin.wei@sri.com  Presenter: Renée Cameto, PhD, Principal Scientist, SRI International, renee.cameto@sri.com  Wei, X., Blackorby, J., & Schiller, E. (2011). Growth in reading achievement of students with disabilities, ages 7 to 17 . Exceptional Children.  More information about the SEELS data can be found at www.SEELS.net 14

Headquarters: Silicon Valley SRI International 333 Ravenswood Avenue Menlo Park, CA 94025-3493 650.859.2000 Washington, D.C. SRI International 1100 Wilson Blvd., Suite 2800 Arlington, VA 22209-3915 703.524.2053 Thank you! Princeton, New Jersey SRI International Sarnoff 201 Washington Road Princeton, NJ 08540 609.734.2553 Additional U.S. and international locations www.sri.com 15