A variational method for the Rasch model Frank Rijmen and Ji r - PowerPoint PPT Presentation

A variational method for the Rasch model Frank Rijmen and Jiˇ r´ ı Vomlel Catholic University Leuven, Belgium Academy of Sciences of the Czech Republic Salzburg, December, 3, 2005 F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 1 / 12

Variables and parameters Model variables binary response variable - its values indicates whether Y n , i the answer of person n to question i was correct n = 1 , . . . , N person index i = 1 , . . . , I question index Model parameters δ i difficulty of question i - fixed effects β n ability (knowledge level) of person n - a random effect F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 2 / 12

Models for the response variable Y � 1 if β n ≥ δ i = Y n , i 0 otherwise. exp( β n − δ i ) P ( Y n , i = 1) = 1 + exp( β n − δ i ) P ( Y n , i = 1 | β n ) for δ i = − 2 1 0.8 0.6 0.4 0.2 -4 -6 -2 0 2 F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 3 / 12

Probability distribution for random effect β n N (0 , σ 2 ) P ( β n ) = a normal (Gaussian) distribution with the mean equal zero, and variance σ 2 . 0.4 0.3 0.2 0.1 0 -3 -2 -1 0 1 2 3 F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 4 / 12

Computations with the Rasch model prior N β (0 , 1) 0.4 0.3 0.2 0.1 0 -3 -2 -1 0 1 2 3 N β (0 , 1) F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 5 / 12

Computations with the Rasch model P ( Y = 1 | β, δ 1 = − 2) posterior 1 0.35 0.3 0.8 0.25 0.6 0.2 0.15 0.4 0.1 0.2 0.05 0 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 N β (0 , 1) · P ( Y = 1 | β, δ 1 = − 2) F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 5 / 12

Computations with the Rasch model P ( Y = 0 | β, δ 2 = 0) posterior 1 0.175 0.15 0.8 0.125 0.6 0.1 0.075 0.4 0.05 0.2 0.025 0 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 N β (0 , 1) · P ( Y = 1 | β, δ 1 = − 2) · P ( Y = 0 | β, δ 2 = 0) F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 5 / 12

Computations with the Rasch model P ( Y = 0 | β, δ 3 = +1) posterior 1 0.14 0.12 0.8 0.1 0.6 0.08 0.06 0.4 0.04 0.2 0.02 0 -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 N β (0 , 1) · P ( Y = 1 | β, δ 1 = − 2) · P ( Y = 0 | β, δ 2 = 0) · P ( Y = 0 | β, δ 3 = +1) F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 5 / 12

Computations with the Rasch model P ( Y = 0 | β, δ 4 = +2) posterior 1 0.12 0.8 0.1 0.08 0.6 0.06 0.4 0.04 0.2 0.02 0 -3 -2 -1 0 1 2 3 -1 1 -3 -2 0 2 3 N β (0 , 1) · P ( Y = 1 | β, δ 1 = − 2) · P ( Y = 0 | β, δ 2 = 0) · P ( Y = 0 | β, δ 3 = +1) · P ( Y = 0 | β, δ 4 = +2) F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 5 / 12

Likelihood of the model given data Assume we have observed answers to I questions from N persons, i.e., we have data y . . . . y 1 , 1 y 1 , 2 y 1 , I y 2 , 1 y 2 , 2 . . . y 2 , I . . . y N , 1 y N , 2 . . . y N , I The task is to find model parameters σ, δ 1 , . . . , δ I that maximize likelihood. N I � � N β n (0 , σ 2 ) · � = P ( y ni | β n , δ i ) d β n L n =1 i =1 ... but this integral does not have a closed-form solution! F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 6 / 12

Likelihood of the model given data Assume we have observed answers to I questions from N persons, i.e., we have data y . . . . y 1 , 1 y 1 , 2 y 1 , I y 2 , 1 y 2 , 2 . . . y 2 , I . . . y N , 1 y N , 2 . . . y N , I The task is to find model parameters σ, δ 1 , . . . , δ I that maximize likelihood. N I � � N β n (0 , σ 2 ) · � = P ( y ni | β n , δ i ) d β n L n =1 i =1 ... but this integral does not have a closed-form solution! F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 6 / 12

Likelihood of the model given data Assume we have observed answers to I questions from N persons, i.e., we have data y . . . . y 1 , 1 y 1 , 2 y 1 , I y 2 , 1 y 2 , 2 . . . y 2 , I . . . y N , 1 y N , 2 . . . y N , I The task is to find model parameters σ, δ 1 , . . . , δ I that maximize likelihood. N I � � N β n (0 , σ 2 ) · � = P ( y ni | β n , δ i ) d β n L n =1 i =1 ... but this integral does not have a closed-form solution! F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 6 / 12

Likelihood of the model given data Assume we have observed answers to I questions from N persons, i.e., we have data y . . . . y 1 , 1 y 1 , 2 y 1 , I y 2 , 1 y 2 , 2 . . . y 2 , I . . . y N , 1 y N , 2 . . . y N , I The task is to find model parameters σ, δ 1 , . . . , δ I that maximize likelihood. N I � � N β n (0 , σ 2 ) · � = P ( y ni | β n , δ i ) d β n L n =1 i =1 ... but this integral does not have a closed-form solution! F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 6 / 12

Likelihood of the model given data Assume we have observed answers to I questions from N persons, i.e., we have data y . . . . y 1 , 1 y 1 , 2 y 1 , I y 2 , 1 y 2 , 2 . . . y 2 , I . . . y N , 1 y N , 2 . . . y N , I The task is to find model parameters σ, δ 1 , . . . , δ I that maximize likelihood. N I � � N β n (0 , σ 2 ) · � = P ( y ni | β n , δ i ) d β n L n =1 i =1 ... but this integral does not have a closed-form solution! F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 6 / 12

Likelihood of the model given data Assume we have observed answers to I questions from N persons, i.e., we have data y . . . . y 1 , 1 y 1 , 2 y 1 , I y 2 , 1 y 2 , 2 . . . y 2 , I . . . y N , 1 y N , 2 . . . y N , I The task is to find model parameters σ, δ 1 , . . . , δ I that maximize likelihood. N I � � N β n (0 , σ 2 ) · � = P ( y ni | β n , δ i ) d β n L n =1 i =1 ... but this integral does not have a closed-form solution! F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 6 / 12

Likelihood of the model given data Assume we have observed answers to I questions from N persons, i.e., we have data y . . . . y 1 , 1 y 1 , 2 y 1 , I y 2 , 1 y 2 , 2 . . . y 2 , I . . . y N , 1 y N , 2 . . . y N , I The task is to find model parameters σ, δ 1 , . . . , δ I that maximize likelihood. N I � � N β n (0 , σ 2 ) · � = P ( y ni | β n , δ i ) d β n L n =1 i =1 ... but this integral does not have a closed-form solution! F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 6 / 12

Approximations to the integral Gaussian quadrature Laplace approximation Variational approximation F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 7 / 12

Approximations to the integral Gaussian quadrature Laplace approximation Variational approximation F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 7 / 12

Approximations to the integral Gaussian quadrature Laplace approximation Variational approximation F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 7 / 12

Variational approximation Let h = β n − δ i . exp( h ) P ( Y n , i = 1 | h ) = 1 + exp( h ) exp( h / 2) = exp( − h / 2) + exp( h / 2) = exp { h / 2 − log [exp( h / 2) + exp( − h / 2)] } exp { h / 2 + f ( h ) } = f ( h ) is approximated by the first order Taylor expansion ˜ f ( h , ξ ) in variable h 2 around point ξ . � � �� f ( ξ ) + ∂ f ( h ) � h = ξ · ( h 2 − ξ 2 ) P ( Y n , i = 1 | h ) ≈ exp h / 2 + � ∂ ( h 2 ) � � � h / 2 + ˜ = exp f ( h , ξ ) ˜ = P ( Y n , i = 1 | h , ξ ) F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 8 / 12

Variational approximation Let h = β n − δ i . exp( h ) P ( Y n , i = 1 | h ) = 1 + exp( h ) exp( h / 2) = exp( − h / 2) + exp( h / 2) = exp { h / 2 − log [exp( h / 2) + exp( − h / 2)] } exp { h / 2 + f ( h ) } = f ( h ) is approximated by the first order Taylor expansion ˜ f ( h , ξ ) in variable h 2 around point ξ . � � �� f ( ξ ) + ∂ f ( h ) � h = ξ · ( h 2 − ξ 2 ) P ( Y n , i = 1 | h ) ≈ exp h / 2 + � ∂ ( h 2 ) � � � h / 2 + ˜ = exp f ( h , ξ ) ˜ = P ( Y n , i = 1 | h , ξ ) F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 8 / 12

Variational approximation Let h = β n − δ i . exp( h ) P ( Y n , i = 1 | h ) = 1 + exp( h ) exp( h / 2) = exp( − h / 2) + exp( h / 2) = exp { h / 2 − log [exp( h / 2) + exp( − h / 2)] } exp { h / 2 + f ( h ) } = f ( h ) is approximated by the first order Taylor expansion ˜ f ( h , ξ ) in variable h 2 around point ξ . � � �� f ( ξ ) + ∂ f ( h ) � h = ξ · ( h 2 − ξ 2 ) P ( Y n , i = 1 | h ) ≈ exp h / 2 + � ∂ ( h 2 ) � � � h / 2 + ˜ = exp f ( h , ξ ) ˜ = P ( Y n , i = 1 | h , ξ ) F. Rijmen and J. Vomlel () Rasch model Salzburg, December, 3, 2005 8 / 12