Item Response Theory: Inference about the model (36)

Chair: Makato Sano, Friday 24th July, 9.30 - 10.50, Lowercroft, School of Pythagoras.

Xinhui Xiong, Lewis Charles and Wang Mingmei Psychometrics program, Fordham University, New York, USA. Accuracy of estimating item parameters for the Rasch model based on small sample size. (233)

Haruhiko Mitsunaga and Shin-ichi Mayekawa, Tokyo Institute of Technology, Japan. Item parameter calibration with non normal ability distribution. (209)

Makoto Sano, Prometric Japan Co. Ltd., Tokyo, Japan. Detecting overestimation of discrimination parameter under surface local dependence: Real data study and effect of item deletion. (113)

David Hessen, University of Utrecht, The Netherlands. A likelihood ratio test for special Rasch models. (186)

ABSTRACTS

Accuracy of estimating item parameters for the Rasch model based on small sample size. (233)
Xinhui Xiong, Lewis Charles and Wang Mingmei
A variety of methods have been developed to estimate the parameters of the Rasch model (also known as the one parameter logistic model) for binary items. Different software packages have implemented different methods and provide results with different degrees of accuracy, depending on sample size, number of items, and model appropriateness. This study focuses on parameter estimation for the case of small samples of persons and few items that often occurs in psychological research. Two software packages were selected for evaluation: Parscale4.1 (implement MML(Marginal Maximum Likelihood method)) and LogXact7 (implement CML(Conditional Maximum Likelihood method)). Simulated data were generated with SAS. The results show that Parscale4.1 produces slightly better item parameter estimates than LogXact7 for simulated data with a small sample of 50 persons and 5 items, although both programs provide satisfactory estimates. For the two conditional methods (asymptotic and exact) implemented in LogXact7, the results indicate that the exact method produces slightly better item parameter estimates than the asymptotic method.

Item parameter calibration with non normal ability distribution. (209)
Haruhiko Mitsunaga and Shin-ichi Mayekawa
In this presentation, we review a general method to estimate the probabilities of a scored multinomial distribution under the assumption that the probabilities are proportional to the PDF of some continuous distribution. For example, a mixture of the normal distributions can be fitted to a test score distribution, enabling us to describe the discrete distribution in a parametric way. Then we propose an application of this method in an IRT setting where the ability distribution is approximated as the scored multinomial distribution. While the existing item calibration methods such as BILOG-MG assume that the latent trait distribution is a normal distribution, it is possible to assume that the ability distribution is a mixture distribution. The method can be implemented in the M-step of the usual item parameter calibration programs. The method is convergent and the resulting ability distribution is easy to interpret. Simulation study using artificial data shows that the proposed method can recover the distribution correctly.

Detecting overestimation of discrimination parameter under surface local dependence: Real data study and effect of item deletion. (113)
Makoto Sano
Chen and Thissen (1997) proposed two models of local item dependence. One of the two is surface local dependence (SLD) which typically affects overestimations of discrimination parameters in Item Response Theory. This study evaluates the performance of some local item dependence indices focusing on the SLD. Real data study was performed with two pilot testing results of an IT certification exam and a business aptitude test in Japan. Two-parameter logistic model was fitted as suggested by Tuerlinckx and De Boeck (2001) and local dependence indices were computed with jIRTNew (Tsai & Hsu, 2005). This approach of using information entropy (Tsai & Hsu, 2005) is promising for detecting SLD and overestimation of discrimination parameter rather than applying X2 or G2 indices with real data. These results support the conclusion of a previous simulation study by Sano (2009). Also the effect of overestimated item deletion on discrimination parameter estimation was investigated. This initial study suggests that the deletion of one of the overestimated items (preventing interaction between SLD item pairs) affect good recovery of the true value of the discrimination parameters if the difficulty parameter of the item has not been affected by another interacting item within the original form.  

A likelihood ratio test for special Rasch models. (186)
David Hessen
In this paper, a general class of special Rasch models for dichotomous item scores is considered. Although Andersen's likelihood ratio test can be used to test whether a Rasch model fits to the data, the test does not differentiate between special Rasch models. Therefore, in this paper a new likelihood ratio test is proposed for testing special Rasch models. The test proposed does not require individual response pattern frequencies and is useful in practice when the observed total score frequencies are sufficiently large.