Cognitive diagnosis, mixtures and mixed effects (16)

Chair: Ming Lei, Tuesday 21st July, 10.50 - 12.10, Dirac Room, Fisher Building. 

Hye-Jeong Choi, Jonathan L. Templin, Allan S. Cohen, University of Georgia, Athens GA, USA and Robert A. Henson, University of North Carolina at Greensboro, USAA Diagnostic classification mixture IRT model (DCMixRM). (055) ♥

Ariel Alonso, Saskia Litière and Geert Molenberghs, Hasselt University, Belgium. Misspecified random-effects distribution in generalized linear mixed models: perception and problems. (258)

Jimmy de la Torre and Chia-Yi Chiu, Department of Educational Psychology, Rutgers, The State University of New Jersey, N ew Brunswick, NJ, USAQ-Matrix Validation under the Generalized DINA Model Framework. (20)

ABSTRACTS

A Diagnostic classification mixture IRT model (DCMixRM). (055)
Hye-Jeong Choi, Jonathan L. Templin, Allan S. Cohen and Robert A. Henson
Mixture IRT models (MixIRT) have been found to be useful in identifying heterogeneity in a population. An important concern with MixIRT, however, is how to explain the secondary dimension(s) along which the different latent classes form. Most of current methods rely on post-hoc analyses to identify characteristics of examinees in different latent classes. Previous research by Smit et al. (1999) suggested the use of covariates in MixIRT both for improving detection and for describing examinee differences among latent classes. In the current study a new model is developed that simultaneously provides latent class membership in a mixture Rasch model (MixRM) and mastery state in a diagnostic classification model. The new model, a diagnostic classification mixture Rasch model (DCMixRM), is formed by combining the MixRM with the Logistic Cognitive Diagnostic Model (Henson et al. in press). The DCMixRM has several appealing features. First, inclusion of mastery states as covariates to predict latent class membership directly provides information leading to plausible substantive interpretations of the secondary dimension(s) that along which the latent classes to form. Second, it improves the power to detect latent classes by using latent collateral information extracted from the item responses.

Misspecified random-effects distribution in generalized linear mixed models: perception and problems. (258)
Ariel Alonso, Saskia Litière and Geert Molenberghs
Generalized linear mixed models have become a frequently used tool for the analysis of non- Gaussian longitudinal data. They also play a prominent role in Item Response Theory where many models, like the Rash model, can be seen as special case of this wide family. Estimation of the parameters and the inferential procedures are often based on maximum likelihood theory, which assumes that the underlying probability model is correctly specified. One of the assumptions at the core of generalized linear mixed model is the normality of the random effects. Since random effects are unobserved, this assumption can rarely be verified accurately. In the present work we show that the maximum likelihood estimators are inconsistent in the presence of this misspecification. The bias induced in the mean structure parameters is usually small but the estimates of the variance components are always severely biased. Further, we illustrate that the power and the Type I error rate of the commonly used inferential procedures are also severely affected. The situation is aggravated if more than one random effect is included in the model. Finally, we propose to deal with possible misspecification by way of sensitivity analysis.

Q-Matrix Validation under the Generalized DINA Model Framework. (20)
Jimmy de la Torre and Chia-Yi Chiu
The Q-matrix is an integral part of a cognitive diagnosis model (CDM) specification. However, it is not unusual in most CDM applications to assume that the Q-matrix is correct after it has been constructed. As a result, any model misfit due to the misspecification of the Q-matrix cannot be detected and remedied. An empirical method of Q-matrix validation in the specific context of the deterministic input, noisy “and” gate (DINA) model has been developed to address this concern. To generalize this idea, this paper proposes a method of Q-matrix validation that can be used in conjunction with the generalized DINA (G-DINA) model. The G-DINA model is equivalent to other general CDMs based on alternative link functions in its saturated form, and reduces to several commonly encountered CDMs in its restricted forms. A discrimination index to identify the appropriateness of a q-vector specification is proposed and is mathematically shown to be optimal under the correct specification. In addition, an algorithm based on this index is developed to demonstrate via a simulation study that the index can identify and correct under- and over-specified q-vectors. Finally, the paper discusses relevant issues regarding the implementation of the method in practice.