Nonparametric Item Response Theory (37)

Chair: Timo Bechger, Friday 24th July, 11.30 - 12.50, Castlereagh Room, Fisher Building.

Hendrick Straat, L. Andries van der Ark and Klaas Sijtsma, Department of Methodology and Statistics, Tilburg University, The Netherlands. Comparing optimization algorithms for item selection in Mokken scale analysis. (085)

Jesper Tijmstra, David J. Hessen, Peter G.M. van der Heijden and Klaas Sijtsma, Utrecht University, The Netherlands. Testing weak item independence. (091)

Rudy Ligtvoet, L. Andries van der Ark, Wicher P. Bergsma and Klaas Sijtsma, Department of Methodology and Statistics, Tilburg University, The Netherlands. Polytomous latent scales for the investigation of the ordering of items. (126)

Wobbe P. Zijlstra, L. Andries van der Ark and Klaas Sijtsma, Department of Methodology and Statistics, Tilburg University, The Netherlands. Forward search applied to nonparametric IRT. (175)

ABSTRACTS

Comparing optimization algorithms for item selection in Mokken scale analysis. (085)
Hendrick Straat, L. Andries van der Ark and Klaas Sijtsma
Mokken scale analysis uses an automated bottom-up item selection procedure, which selects items that satisfy particular scaling criteria into a scale. This procedure su ers from two problems. First, an item, which satis ed the scaling criteria when it was selected, may not satisfy these criteria anymore after other items have been selected later on to complete the scale. The rst study used simulated data to demonstrate that indeed under particular conditions it is likely that items do not satisfy the scaling conditions in the nal scale. Second, the bottom-up procedure does not evaluate all possible selections of items and, consequently, the optimal scale may not be found. The second study investigated whether a genetic algorithm and two adaptations of the automated item selection procedure improve upon the automated item selection procedure from Mokken scale analysis. It was found that the genetic algorithm much improved upon all three alternatives, and we conclude that it is a good alternative for item selection in the context of Mokken scale analyses.

Testing weak item independence. (091)
Jesper Tijmstra, David J. Hessen, Peter G.M. van der Heijden and Klaas Sijtsma
Within Item Response Theory (IRT), both parametric and non-parametric models have been proposed that have the property of weak item independence over the total score. This property states that the ordering of the item passing probabilities is the same at every level of the total score. When this property is violated, comparing the performance of persons becomes more difficult. While weak item independence over the total score holds for a selection of IRT models (Rasch model, double monotonicity Mokken model), it is violated under less restrictive IRT models, such as the Birnbaum model or the monotone homogeneity Mokken model. By testing for weak item independence, these two groups of models can be distinguished The current research proposes and compares four different tests for weak item independence. The tests make use of Kendall’s W, a weighted version of Kendall’s W, Scheiblechner’s weak item index sigma, and a Bayesian approach developed by Karabatsos and Sheu. A comparison in terms of test sensitivity, obtained through simulation study, indicates that Kendall’s W should be preferred for small tests (e.g. k = 5), while the Bayesian approach should be preferred for larger tests. Additionally, the four tests are applied to two sets of empirical data.

Polytomous latent scales for the investigation of the ordering of items. (126)
Rudy Ligtvoet, L. Andries van der Ark, Wicher P. Bergsma and Klaas Sijtsma
Latent scales are defined within the framework of nonparametric IRT for polytomously scored items. Latent scales imply an invariant item ordering, without imposing parametric restrictions on the shape of the item response functions. A hierarchical relationship between three latent scales is derived. The observable properties of manifest-scale cumulative probability model, manifest invariant item ordering, and increasingness in transposition are derived. A real-data example illustrates the investigation of latent scales by means of these manifest properties.

Forward search applied to nonparametric IRT. (175)
Wobbe P. Zijlstra, L. Andries van der Ark and Klaas Sijtsma
Mokken scale analysis (MSA) is a popular method for analyzing test and questionnaire data. MSA does not provide methods to identify or accommodate outliers. In general, it is well known that the presence of outliers may result in biased statistical results and wrong conclusions. The forward search is a robust diagnostic analysis in which outlier detection is central. We concentrate on the identi cation of outliers in Mokken scale analysis. We adapt the forward search algorithm for MSA. Possible choices made for the adapted forward search algorithm are investigated. These choices pertain to the de nition of residuals, the inclusion or exclusion of model- t statistics in the forward search algorithm, and the inclusion or exclusion of tests for item selection. The application of the adapted forward search algorithm for MSA is demonstrated using real data.