USING COMPUTERIZED ASSESSMENT TO IDENTIFY PROFILES OF READING & LANGUAGE SKILLS IN ELEMENTARY AND SECONDARY STUDENTS Barbara Foorman and Yaacov Petscher - Florida Center for Reading Research at Florida State University Liz Brooke and Alison Mitchell - Lexia Learning
AGENDA ¡ The Structure of Reading ¡ Identifying Profiles ¡ Connections to Instruction
THE STRUCTURE OF READING How does reading relate to language? How do we reconcile the structure of reading with profiles of strengths and weaknesses?
FL FLORID ORIDA CENT A CENTER F ER FOR READING OR READING RES RESEAR EARCH (F (FCR CRR) ) READING G ASSESSMENT ASSESSMENT ( (FRA) FRA) As part of federal grants, FCRR developed the FRA • 2010-2014: Developed computer-adaptive K-12 component skills battery (item tryouts, IRT analyses, & linking studies. Called the FCRR Reading Assessment (FRA) • 2010-2015: Conducted cross-sectional and longitudinal study of reading & language development.
STR STRUCTU CTURE O OF R READ ADING K NG K - - 3 3 Phonological awareness Decoding Reading fluency Comprehension Syntax Oral Vocabulary language Listening Comp Foorman et al. (2015) Reading & Writing
STR STRUCTU CTURE O OF R READ ADING 4 - NG 4 - 1 10 0 Decoding fluency Reading 72% - 99% Comprehension variance Syntax Oral language Vocabulary Foorman, et al. (2015) Journal of Educational Psychology
BA BACK CKGR GROUND O ON PR PROFILES S • Targeting instruction to profiles of students’ strengths & weaknesses is central to teaching • Profiles often based on descriptive data of reading errors, text reading levels, or learning profiles. • Regression-based techniques used to quantify profiles of good & poor readers and profiles within poor readers. • Regression-based approaches use arbitrary achievement cut points (e.g., below 40 th percentile on standardized test). • A latent class approach (LCA) utilizes multiple measures to reduce measurement error and improve reliability and stability of classification. • When the latent variable is continuous, the approach is often called latent profile analysis (LPA). LPA used here.
RESEAR RES EARCH QUESTIONS QUESTIONS ¡ What are the latent profiles of reading and language skills in a large, representative sample of Florida students in grades K-10? ¡ What are the relations among the latent profiles and a norm-referenced reading test in K-2 and a latent variable of reading comprehension in grades 3-10?
MET METHOD HOD • Participants: 7,752 students in K-10; 2295 in K-2 and 5,457 in 3-10. Representative of Florida demographics. • Procedure: - K-2 FRA individually administered in two 45-min sessions in mid-year; 3-10 FRA administered in computer lab in two 45- min sessions in mid-year. - SESAT Word Reading administered in small groups in K; teachers administered SAT-10 and FCAT as usual. • Design and analysis: FRA raw scores converted to Z scores. Latent profile and general linear modeling were conducted at each grade (with linear step-up correction to correct against false discovery rate).
CONSTRUCTS/FRA SCREENING TASKS IN GRADES K-2 Constr Con struct ct Task/Ab /Abbreviation Grade Gr e Pho honological A Awareness Pho honological a awareness ( (PA) A) K K Al Alpha habetics LS L Letter S Sounds ( (AP AP1/2 /2; L LS) K K D Decoding Word R Reading ( (WR) G1 + + G G2 Encoding E Spelling ( (Spell) G2 G2 Oral L Language V Vocabulary Vocabulary P Pairs ( (VOC) K- K-2 Syntax S Sentence C Comprehe hension ( (SC) K K L Listening C Comprehe hension Following D Directions ( (FD) K- K-2
CONSTRUCTS/FRA SCREENING TASKS IN GRADES 3-10 Constr Con struct ct Tasks/Ab /Abbreviation Word recognition Word Recognition (WRT) Academic Language Vocabulary (morphological awareness) Vocabulary Knowledge (VKT) Discourse (verb tense, anaphora, connectives) Syntactic Knowledge (SKT) Reading Comprehension Reading Comprehension (RCT)
STANDARDIZED READING OUTCOMES IN GRADES K-10 Gr Grade(s) e(s) Test/S /Subtest Kindergarten Stanford Early Scholastic Achievement Test (SESAT) Word Reading 1-10 Stanford Achievement Test (10 th ed; SAT-10) Reading comprehension 3-10 Florida Comprehensive Assessment Test (FCAT 2.0) Reading Note . A latent factor score for Reading Comprehension was created from the developmental scale scores from the FRA’s RCT, SAT-10, and FCAT 2.0
CORRELATIONS ¡ Print-related measures were moderately correlated in K (PA with LS, .48, and with SESAT WR, .58; and SESAT WR with LS, .51). OL measures were moderately correlated with each other. ¡ In G1-G2, print-related measures were more strongly correlated: SAT-10 with WR (.75 in first and .58 in G2); WR & Spell in G2 (.77). Oral language measures were moderately correlated with each other in all three grades and VOC was moderately correlated with SAT-10 in G1 (.58) & G2 (.62). ¡ The three RC measures were strongly correlated, with the RCT bivariate correlations ranging from .67 in G8 to .81 in G5. FCAT and SAT-10 correlations ranged from a low of .71 in G8 to a high of .81 in G3 & G5. VKT and SKT were moderately correlated in these grades (.31 to .46) as were the bivariate correlations of WRT (.29 to .51).
IDENTIFYING PROFILES What are the challenges associated with identifying profiles of student strengths and weaknesses? What are ways to do so that are meaningful, but remain reliable and valid and can be assessed efficiently?
LATENT PROFILE ANALYSIS – WHAT? Hybrid Con Conti tinuou ous s Ca Categor egorica cal Mode Models ls Me Measure asures s Me Measure asures s Continuous Latent Factor Analysis Item Response Theory Categorical Latent Latent Profile Latent Class Analysis Analysis
LATENT PROFILE ANALYSIS – WHAT? Vocabulary Class PPVT EVT SYN PPVT EVT SYN
LATENT PROFILE ANALYSIS – WHY?
WHY ADAPTIVE MEASURES? 3-6 hours!!!
Latent profile model fit for kindergarten through grade 5 and grade 8 Grade Profiles Parameters LL AIC aBIC -2LL K 2 19 -3255.01 6548.01 6624.87 3 22 -2681.62 5407.25 5496.24 1146.77* 4 28 -2653.41 5362.82 5476.08 56.42* 5 34 -2629.75 5357.51 5465.04 47.32* 6 40 -2618.44 5316.88 5458.68 22.63 * 1 2 10 -2818.26 5656.52 5705.49 3 14 -2785.14 5598.28 5666.84 66.24* 4 18 -2768.46 5572.92 5661.01 33.36* 30.94 * 5 22 -2752.99 5549.98 5657.71 6 26 -2743.43 5546.86 5674.17 19.12* 2 2 13 -3768.29 7562.59 7624.79 3 18 -3697.13 7430.26 7516.38 142.33* 4 23 -3669.02 7384.03 7494.08 56.22* 5 28 -3655.54 7367.07 7501.04 26.96* 6 33 -3642.95 7355.89 7513.78 25.17 * 3 2 10 -2202.90 4425.81 4438.14 3 14 -2173.78 4375.56 4392.83 58.24* 4 18 -2154.42 4344.83 4367.04 38.73* 5 22 -2129.87 4303.74 4330.88 49.10 * 6 26 -2104.72 4261.43 4293.51 50.30* 4 2 10 -2166.75 4353.49 4365.49 3 14 -2140.35 4308.69 4325.50 52.80* 4 18 -2112.79 4261.58 4283.18 55.12* 5 22 -2097.77 4239.54 4265.95 30.04* 6 26 -2087.22 4226.44 4257.65 21.10 * 5 2 10 -2451.39 4922.79 4935.95 3 14 -2405.92 4839.83 4858.25 90.96* 4 18 -2383.61 4803.22 4826.91 44.61* 40.97 * 5 22 -2363.13 4770.25 4799.20 Note. LL=log likelihood, AIC=Akaike Information Criteria, aBIC=sample adjusted Bayes Information Criteria, -2LL=log likelihood ratio test. Values in bold represent 19 Final selected class. *p<.001.
Latent Profile Analysis of FRA Measures in Kindergarten (N=422) VOC FD PA LS SC 2 1 c6; 19% Z-Score c3; 42% 0 c4; 23% Kindergarten Z -1 c5; 7% c1; 7% -2 c2; 2% -3 c1 c2 c3 c4 c5 c6 Note . VOC=Vocabulary Pairs; FD=Following Directions; PA=Phonological Awareness; LS=Letter Sounds; SC=Sentence Comprehension 20
K SES K S ESAT W WR B R BY LA LATENT ENT CLAS CLASSES ES Distribution of SESAT Distribution of SESAT F 37.11 F 37.11 Prob > F <.0001 Prob > F <.0001 550 550 500 500 SESAT SESAT 450 450 400 400 350 350 1 1 2 2 3 3 4 4 5 5 6 6 group group Note. The average absolute value of the standardized difference in SESAT WR performance across all classes was Hedge’s g = 1.10, indicating the magnitude of differences in FRA skill profile performance on standardized outcome
Latent Profile Analysis of FRA Measures in Grade 1 (N=989) VOC FD WR 2 1 c2; 35% Z-Score c5; 43% 0 c2; 35% -1 1 Z c1; 1% Grade 1 c3; 3% -2 -3 -4 c1 c2 c3 c4 c5 Note . VOC=Vocabulary Pairs; FD=Following Directions; WR=Word Reading 22
G1 S G1 SAT-1 -10 R 0 RC B C BY L Y LATENT CL TENT CLASSES ASSES Note. The average absolute value of the standardized difference in SAT-10 RC performance across all classes was Hedge’s g = 1.43, indicating the magnitude of differences in FRA skill profile performance on standardized outcome
Latent Profile Analysis of FRA Measures in Grade 2 (N=884) VOC FD Spell WR 2 c5; 32% 1 Z-Scores c2; 10% 0 2 Z c4; 32% Grade 2 c3; 15% -1 c6; 5% c1; 52% -2 c1 c2 c3 c4 c5 c6 Note . VOC=Vocabulary Pairs; FD=Following Directions; Spell=Spelling; WR=Word Reading 24
G2 S G2 SAT-1 -10 R 0 RC B C BY L Y LATE TENT CL NT CLASSE ASSES Note. The average absolute value of the standardized difference in SAT-10 RC performance across all classes was Hedge’s g = 1.48, indicating the magnitude of differences in FRA skill profile performance on standardized outcome
Recommend
More recommend
