Differential Item Functioning: New models and applications (28)

Chair: Richard Swartz, Thursday 23rd July, 14.55 - 16.15, Castlereagh Room, Fisher Building. 

Rianne Janssen and Mussie Kebede Akalu, Faculty of Psychology and Educational Sciences, K U Leuven, Belgium. Modeling item order effects within a DIF framework. (248)

Sofie Frederickx, Francis Tuerlinckx, Paul De Boeck and David Magis, Department of Psychology, University of Leuven, Belgium. An item mixture model to detect Differential Item Functioning. (179) ♥

Karl Schweizer, Psychology Department, Goethe University Frankfurt, Germany. Latent variable models representing the position effect and their application to reasoning items. (225)

ABSTRACTS

Modeling item order effects within a DIF framework. (248)
Rianne Janssen and Mussie Kebede Akalu
The position of an item within a test can influence test performance (e.g., Bejar, 1985; Kingston & Dorans, 1982). When examinees improve towards the end of the test, the effect is known as a practice effect. The opposite is the case for a fatigue effect. Test speededness refers to testing situations where some test takers do not have sufficient time to attempt every item in the test within the allocated time. Item position effects are not without consequences for the estimation of models from Item Response Theory (IRT). They may lead to distorted item parameter estimates (see, e.g., Oschima, 1994; Wise, 1986), or they may be a possible threat to the local indepence assumption. Alternative IRT models have been proposed to model the response strategy switches of examinees towards the end of the test (Bolt, Cohen & Wollack, 2002; Goegebeur, De Boeck, Molenbergs & del Pino, 2006; Yamamoto & Everson, 1997). In the current paper, the framework of differential item functioning (DIF) is used to model the effect of item position. The DIF framework can be applied whenever a test is adminstered according to a design that allows investigating item position effects as is the case, for instance, when the items of the test are adminstered in one of two orders, with one order being the reverse of the other order. When the item order differences refer to a clear practice or fatigue effect the DIF parameters may follow a clear increasing or decreasing pattern. This can be modeled with the linear logistic test model (LLTM; Fischer, 1977). The proposed framework to investigate item order effects is applied to a test on listening ability in French for foreign language learners. There were two tests of 57 items, which had 28 items in common. For each test, the audio fragments were administered in one of two orders, with one order being the reverse of the other order. This resulted in four test booklets. All models were estimated using the procedure NLMIXED in SAS. Although the results were somewhat divergent over the different fit criteria, it will be argued that the two-parameter model with DIF only on item difficulty is to be preferred. Items later in the test became more difficulty, but they did not discriminate better among students. The correlation between the item order effect and item position was .77 and .69 for Test 1 and 2 respectively, resulting in a reasonable fit for the LLTM.  

An item mixture model to detect Differential Item Functioning. (179)
Sofie Frederickx, Francis Tuerlinckx, Paul De Boeck and David Magis
In this presentation we present a new methodology for detecting Differential Item Functioning (DIF). We introduce a DIF model that is based on a Rasch model with random item difficulties (besides the common random person abilities). In addition, a mixture model is assumed for the item difficulties such that the items may belong to one of two classes: a DIF or a non-DIF class. The crucial difference between the DIF class and the non-DIF class is that the item difficulties in the DIF class may differ according to the observed person groups while they are equal across the person groups for the items from the non-DIF class. Statistical inference for the item mixture DIF model is carried out in a Bayesian framework. The performance of the item mixture DIF model is evaluated using a simulation study in which it is compared with traditional procedures, like the Likelihood Ratio test, the Mantel-Haenszel procedure and the standardized p-DIF procedure. In this comparison, the item mixture DIF model performs better than the other methods. Finally, the usefulness of the model is also demonstrated on a real life dataset.  

Latent variable models representing the position effect and their application to reasoning items. (225)
Karl Schweizer
The position effect describes the dependency of the responses to the items of a measure on the positions of the items within the sequence of items. Such a position effect has repeatedly been reported for measures of ability and personality. The occurrence of such a position effect is especially likely in reasoning measures because of the specific ordering of items. However, despite the manifestness of this effect the standard latent variable models neglect the item position as source that influences the responses to the items of a measure, as for example the congeneric and essentially tau-equivalent models. These models normally include one latent true random variable for representing the attribute of interest. For being able to consider the position effect appropriately another weighted latent true random variable is integrated into the true parts of these models. This additional latent true random variable represents the position effect by either a linear or quadratic function. The application of the original models and the models adjusted for representing the position effect to data of a reasoning measure revealed that considering the position effect can improve the model fit considerable and prevent models from being erroneously dismissed as inappropriate.