Model Equation Comparing Missing Values Handling Algorithms Observations: v = 1... n in the Context of the Rasch Model Items: i = 1... k Rainer W. Alexandrowicz ( ) ( ) θ − β ξ ε x x ( ) e vi vi v i θ β = = p x , v i θ ξ = vi v i e θ − β + ξ ε + v i 1 v 1 e v v i − β ε = e i i Alpen-Adria-Universität Klagenfurt, Institut für Psychologie, Abteilung für Angewandte Psychologie und Methodenforschung Universitätsstraße 65-67 9020 Klagenfurt Österreich rainer.alexandrowicz@uni-klu.ac.at Conditional ML Estimation (CML) I CML II ( ) ∏∏ n k = ε γ − x b 1 k L ε r ( ) ( ) ∑∏ Assumption: vi vi with γ = ε x ε, B b vi r i vi C i r v = v x r i 1 The design matrix B is assumed to be known before answers v v = = v 1 i 1 are obtained (cf. Molenaar, 1995, p. 40), e.g. when using testlets . Data matrix X Design matrix B To establish a common scale for all item parameters, link items must exist (well vs. ill-conditioned data). This can be 1 0 1 a a 1 1 1 0 0 warranted by adequately assembling the testlets. 0 1 0 a a 1 1 1 0 0 1 1 0 1 1 1 1 1 1 1 a a 1 1 0 0 0 1 1 1 a a 1 0 1 0 0 1 1 1 Molenaar, I. W. (1995). Estimation of Item Parameters. In: G.H. Fischer & I.W. Molenaar (Eds.). Rasch Models. Foundations, Recent Developments, and Applications. (pp. 39-51). NY: Springer.
Problem Method (i) Simulation Study: Data sets conforming to the Rasch Model were generated and missingness was induced according to the taxonomy of Rubin (1976) [MCAR, MAR, NMAR]. Special focus: MCAR vs. NMAR: Problem: MCAR: a given percentage of values were deleted Missing values appear in the randomly across the data matrix course of testing, hence the NMAR: the probability of missingness was determined assumption does not hold. according to a 4PL-like model: ( ) − − ( ) 1 c d Question: θ β = + p m , , a , b , c , d c ( ) vi v i θ − β − + a b 1 e Is the design matrix B a valid v i means for handling missing + Intermediate step: values not known prior to data Use person parameter estimate as propensity to produce a missing value. acquisition? Rubin, D. B. (1976). Inference and missing data. Biometrika, 63 , 581–592. ( ) − − ( ) 1 c d θ β = + p m , , a , b , c , d c ( ) vi v i θ − β − + a b 1 e Method (ii) Method (iii) v i Missing Values Handling Methods p(missing|NMAR) 1.0 1) Treat as structural missings, i.e. pretend, they were never presented to the d testee (involving B with zeros inserted where missing values occured). 0.8 2) Assume no answer given = no answer known; testee preferres to omit a question to taking the risk of a wrong answer. Missings are replaced by zeros. 0.6 3) Opposite to 2: Missing values are replaced by ones (e.g. because testee did β i + b a not want to admit support of nuklear power plants or a right wing party in a survey; social desirability; ...). 0.4 4) Assume testee was, say, distracted, but would have been able to sometimes respond correctly and sometimes not; however, we do not know which. 1 ∑ [ ] = obs . 0.2 p x Missing values are replaced by 0 or 1 drawn randomly from a Bernoulli with i vi n v p=.5 c ( ) ˆ ˆ 5) „Mean imputation“: Replace missings by draws from a Bernoulli with = + θ β p p , 0.0 vi v i 6) „Model based imputation“: Replace missings by draws from a Bernoulli with -2 -1 0 1 2 (two step method).
Results (i) Results (ii) Item Bias: k=20, n=1000 MCAR – Item Bias MCAR – LR Test 0.3 0.2 m i k=20; Original Values k=20; Zero Fill i k=20; Missing Values m x 0.14 0.07 o m i 0.06 x o m 0.1 0.12 o x 0.06 m i i 0.05 r m i x x 0.10 r x o 0.05 o o 0.04 i x r o m i m r r Bias x r r o 0.08 0.04 x 0.0 r o Density m x r r Density Density m o 0.03 o r i m i r r r i m r r r o r 0.06 o 0.03 x x x x i i i o m r o r m m 0.02 o x i x x o 0.04 o 0.02 i x m o -0.1 x o x 0.01 0.02 m i i 0.01 i m m i m 0.00 0.00 0.00 0 10 20 30 40 14 16 18 20 22 24 26 28 -0.2 10 20 30 40 x x x -0.3 5 10 15 20 Item No. increasing difficulty Results (iii) Results (iv) MCAR – LR Test NMAR (1) – Item Bias NMAR - Item Bias: k=10, n=1000 0.4 k=20; Random Fill k=20; Item Mean Imputation k=20; Model Based Imputation 0.08 0.06 i 0.2 m i 0.05 0.06 m 0.06 x o x i m 0.04 o x r m Density Density 0.04 Density r o o 0.04 0.03 x Bias i x i 0.0 r r r m r o r o r x 0.02 x x r o r 0.02 o m m i 0.02 o i o i 0.01 x m m i m i 0.00 0.00 x -0.2 0.00 0 10 20 30 40 10 15 20 25 30 35 40 5 10 15 20 25 30 35 40 x x x -0.4 2 4 6 8 10 Item No.
Results (v) Results (vi) k=5 NMAR (1) – LR-Test NMAR (1) – LR-Test Missing Values Zero Fill Item Mean Imputation Original Values Model Based Imputation 0.30 Random Fill 0.30 0.30 0.30 0.30 0.30 0.25 0.25 0.25 0.25 0.25 0.25 0.20 0.20 0.20 0.20 0.20 0.20 Density Density 0.15 0.15 Density Density 0.15 0.15 Density Density 0.15 0.15 0.10 0.10 0.10 0.10 0.10 0.10 0.05 0.05 0.05 0.05 0.05 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0 5 10 15 0 5 10 15 0 5 10 15 x x 0 5 10 15 0 5 10 15 0 5 10 15 x x x x in fact: This principle is not NMAR! Results (vii) Results (viii) NMAR (2) – Item Bias NMAR (2) – Item Bias Design: Only even items affected by missing n=8000; k=10 NB: green = item mean; values
Thank You! Conclusio (so far) MCAR: Largely unproblematic, all missing values handling methods performed equally well with respect to item bias. Random imputation and mean imputation outperformed the other principles (but structural missing/CML) in many instances. LR-Test statistic appears to perform well, density misfit probably due to small number of replications (however, this requires affirmation). NMAR: If you have eRm (and perhaps someone who knows how to operate it) at hand, use CML and treat missing values as structurally missing. If not, or if you deliberately want to impute, do not use fixed value imputation (e.g. setting missings to wrong answer). Rather, use rainer.alexandrowicz@aau.at item mean or just draw zeros and ones at random.
Recommend
More recommend
