Poster Session Two: Foyer, Wednesday July 22nd, 18.00 - 19.00
Anita Rawls, Department of Educational Studies, Columbia,
Ana R. Delgado, Facultad de Psicología, Universidad de Salamanca, Spain, Gerardo Prieto, María V. Perea and Valentina Ladera. Probing the Benton Visual Retention Test with the Rasch Model. (058)
Kou Murayama,
Naoko Kuga, Tokyo Institute of Technology, Japan, Shin-ichi Mayekawa. New analytic solution to weighted metric unfolding. (115)
Jenn-Yun Tein,
Shu-Ping Chen, Department of Psychology, National Chengchi University, Taipei City, Taiwan, and Chung-Ping Cheng. Model specification for latent endogenous interaction and quadratic effects. (180)
Takamitsu Hashimoto, National Center for University Entrance Examinations, Tokio, Japan, and Maomi Ueno, The University of Electro-Communications, Tokyo, Japan. Kullback-Libler Divergence and item parameter estimates when one item affects another item. (210)
José Antonio López-López, Faculty of Psychology, University of Murcia
Fulgencio Marín-Martínez, Faculty of Psychology, University of Mercia, Spain, and Julio Sánchez-Meca. Statistical factors affecting an average effect size in meta-analysis. (222)
Julio Sánchez-Meca, Faculty of Psychology, University of Murcia, Spain, and José A. López-Pina. The Reliability Generalization Approach: A comparison of different procedures to estimate an average coefficient alpha* (223)
Hongyun Liu,
Ionel Dorofte, Technical University of Iasi, Iasi, Romania, and Tatiana Dorofte. To a revolution of measure in psychometrics: from IQ to PQ. (244)
Silia Vitoratou, Department of Statistics,
Renske Kuijpers, Department of Methodology and Statistics, Tilburg University, The Netherlands, and M. A. L. M. van Assen. A scaled effect size measure taking into account the distributions of the variables. (131)
Jenn-Yun Tein,
Youn Y. Choi,
Nisha Gottfredson and Daniel J. Bauer , Department of Psychology, University of North Carolina at Chapel Hill, USA. Exploring the efficacy of latent class pattern mixture models for handling non-randomly missing data. (059)
Milena Falcaro and Andrew Pickles, Biostatistics Group, The University of Manchester, UK. Analysing censored longitudinal data with non-ignorable missing values. (134)
Jeannette Brodbeck and Tim J. Croudace, Department of Psychiatry, University of Cambridge, UK, Anna Brown, Department of Psychology, University of Barcelona, Spain. Using a Cohort-Sequential Latent Growth Model to examine alcohol use from ages 16 to 29. (263)
ABSTRACTS
Anita Rawls
Test validity is an important concept that should be established when decision making decisions that will affect a student’s future. Policymakers and educators want to be certain they are making decisions that will not adversely affect any particular gender, ethnicity, level of English proficiency, socioeconomic status, and disability. Providing this validity evidence can be done by demonstrating the measure is invariant across groups. The study reviews methods that are used to detect measurement invariance and applies three of these methods to a reading test for gender, ethnic, and administrative subgroups. The purpose of this study is to extend the body of knowledge to understand the relationship between confirmatory factor analysis (CFA), Rasch item analysis, and hierarchical linear modeling (HLM) in the detection of measurement invariance across subgroups. The results of the study will either support or refute the efforts of researchers to interchange the use of CFA, Rasch, and HLM to detect measurement invariance.
Probing the Benton Visual Retention Test with the Rasch Model. (058)
Ana R. Delgado, Gerardo Prieto, María V. Perea and Valentina Ladera
Metric considerations are at the root of some important methodological problems in current clinical neuropsychological practice. Parametric statistical methods are typically used for analyzing neuropsychological test scores, even though the quantitative assumption has not been tested. In this context, the Rasch Model (RM) is useful to construct an interval scale on which both items and participants are scored. This is desirable both scientifically and from the point of view of the interpretation and communication of results. Thus, our objective is to probe the Benton Visual Retention Test (BVRT) with the RM. This study consists of a secondary analysis of neuropsychological test data. A total of 273 participants from the original studies were included. Some 219 had been selected from the general adult Spanish population and 54 were patients that had suffered Traumatic Brain Injury. Results show that the BVRT shows essential unidimensionality and good fit to the RM, but that score reliability is very low. (This contribution is funded by Research Grant Ref: JCyL/SA057A08.) (058)
Distinguishing controlled from automatic cognitive processes using a mixture modeling framework for a Gaussian signal-detection model. (083)
Kou Murayama
This research proposed a mathematical model to distinguish controlled from automatic cognitive processes in the affect misattribution procedure (AMP; Payne et al., 2005), which is a widely used computerized tool in social psychology. Previous studies have shown that the AMP is a reliable and valid measure of implicit attitudes. However, it is easy for participants to correct their responses intentionally. The present model addressed this issue by explicitly incorporating the parameters of controlled, as well as automatic, processes. Specifically, I assumed a Gaussian signal-detection process for the automatic component in which people set a threshold to determine if they like the target. In addition, the model assumed that people may intentionally correct their threshold to some extent (denoted by B) with a probability of R. Using a mixture modeling framework, it is possible to estimate the parameters by minimizing the least-square function. A series of three psychological experiments were conducted to investigate the validity of the proposed model. All of the experiments showed that the estimated parameters were meaningful and sensitive to the experimental manipulations in a predicted way, suggesting the validity of the proposed model.
New analytic solution to weighted metric unfolding. (115)
Naoko Kuga and Shin-ichi Mayekawa
Kuga and Mayekawa (2006, 2007) developped an analytic solution (KMMDU) to the metric multidimensional unfolding model in which the observations are assumed to be linearly transformed squared distances between two sets of points. The method can recover the exact configuration from the column standardized error-free squared distance matrix. In this research we extended our analytic method to the weighted Euclidean model, where each variable is allowed to have unique set of weights for each dimension. Similar to KMMDU, we first calculated the set of eigen vectors of the column centered observation matrix, and using it as an orthonormal basis of the row configuration, PREFMAP algorithm was used to derive the column configuration and wieghts. In order to cope with the weighted Euclidean model, simultaneous diagonalization algorithm such as INDSCAL/SUMSCAL was incorporated prior to the use of PREFMAP algorithm.
Jenn-Yun Tein
Model specification for latent endogenous interaction and quadratic effects. (180)
Shu-Ping Chen and Chung-Ping Cheng
Estimating the interaction between theoretical constructs is an important concern in the social sciences. Kenny and Judd (1984) formulated the first nonlinear structural equation model by using product variables as indicators of latent product variables and imposing nonlinear constraints on parameters. Jöreskog and Yang (1996) refined the Kenny-Judd model by analyzing the mean vector and covariance matrix of the observed variables in the specification of nonlinear relations. However, the vast majority of previous studies of nonlinear structural equation modeling involved interaction between two latent exogenous variables. This study generalizes the Jöreskog and Yang approach to include nonlinear relations between two latent endogenous variables. The procedure is demonstrated using three models, including the latent interaction effect model (i.e. moderated mediation), the quadratic effect model, and a more elaborate model involving all second order latent effects. As endogenous variables are fed into these models, nonlinear constraints of greater complexity must be specified. The accuracy of the nonlinear constraints is confirmed via implementation in LISREL8.7 with artificial data of known parameter values. Future applied researchers, in estimating nonlinear effect of latent endogenous variables, can use this research as reference.
Kullback-Libler Divergence and item parameter estimates when one item affects another item. (210)
Takamitsu Hashimoto and Maomi Ueno
Most commonly employed models of IRT involve an assumption of local independence, which means that, for a given person, the probability of answering an item correctly is independent of the probability for other items. When a response to an item changes the difficulty of another item, this assumption is violated. This study revealed that KL divergence inflated when it was calculated with assumption of local independence and actually the assumption was violated. When the "parent" item and the "child" item were specified and they were treated correctly, parameter estimates were close to the true values and KL divergence was close to that calculated by substituting true parameter values. The estimation method is as follows: Firstly, divide the "child" item into 2 items. Secondly, let a response to one "child" item be unobserved when a response to the "parent" item is correct, and let a response to the other "child" item be unobserved when a response to the "parent" item is incorrect. At last, estimate parameters of all items including the two "child" items. Then the different 2 estimates of difficulty parameters and a not-inflated estimate of slope parameter of the "child" are obtained.
Methodological and statistical factors in reliability generalization: An application to the Maudsley Obsessive-Compulsive Inventory. (221)
José Antonio López-López,
Statistical factors affecting an average effect size in meta-analysis. (222)
Fulgencio Marín-Martínez and Julio Sánchez-Meca
The optimal weight for averaging a set of independent effect sizes is the inverse variance of each effect size, but in practice these weights have to be estimated, being affected by sampling error. In random-effects meta-analyses there are different proposals for averaging independent effect sizes: weighting by sample size as an approximation to the optimal weights and weighting by an estimation of the inverse-variance of each effect size. Furthermore, the latter proposal also includes different alternatives, depending on the procedure to estimate the heterogeneity variance in the meta-analysis. In this poster, the bias and mean squared error (MSE) of ten estimators of an average effect size were assessed via Monte Carlo simulation of random-effects meta-analyses with the standardized mean difference as the effect-size index. The population effect size, the between-studies variance, the number of studies and the sample size were manipulated in the simulations. The use of different heterogeneity variance estimators scarcely affected to the performance of the Hedges & Vevea’s average, that showed a slight negative bias and the best results in terms of MSE. The Hunter & Schmidt’s average, although systematically unbiased, showed the worst results in terms of MSE. The practical consequences of the results are discussed. *This research has been supported by Fondo de Investigación Sanitaria (Project nº PI07/90384).
The Reliability Generalization Approach: A comparison of different procedures to estimate an average coefficient alpha* (223)
Julio Sánchez-Meca and José A. López-Pina
One of the most recent extensions of the meta-analytic technique consists into integrate the reliability coefficients obtained in different applications of a same psychological test. The reliability generalization (RG) approach is based on the idea that reliability is not a characteristic of the test, but of the test scores obtained in a given application of it. From this perspective, the RG approach aims to empirically examine when reliability estimates obtained in different applications of the same test are stable and which study characteristics can affect the scores reliability. Because RG is a new technique, there is not yet a consensus in the scientific community about the statistical model to be used to accomplish their objectives. The purpose of this paper is to carry out a Monte Carlo study to compare the bias and efficiency of several procedures to obtain an overall coefficient alpha from a set of applications of the same test. In the simulations there will be manipulated such factors as the number of applications (studies), the average sample size of the studies, and the parametric coefficient alpha. Finally, the results of the simulation and its implications for RG studies are discussed. *This research has been supported by Fondo de Investigación Sanitaria (Project nº PI07/90384).
Unidimensional and multidimensional models for dichotomous variables: A comparison of the CFA and IRT approaches. (236)
Hongyun Liu and
In the present paper, dichotomous (right and wrong) item response factor analysis model is analyzed and compared in two approaches, which are the item response theory (IRT) and the structural equation model approach. Two Monte-Carlo simulation studies are conducted to examine the effect factors of parameter estimations in unidimensional and multidimensional situations, respectively. The results show that for the unidimensional model, the convergence ratio is affected by the sample size in the SEM framework, but not affected in the IRT framework. The parameters results of IRT model are not stable when the number of the items is small. For large sample size (above 1000) and for moderate and large number of items, the parameters estimations are similar between IRT and SEM framework. For the simulation study in multidimensional model, the convergence ratio is affected by the sample size both in the SEM framework and IRT framework. For large sample size (above 1000), the parameters estimations are similar between IRT and SEM framework, but the speed of IRT analysis is much slower than SEM analysis, which is because the different estimation method are used between two frameworks. Based on the results above, advantages and disadvantages of the two approaches are discussed.
To a revolution of measure in psychometrics: from IQ to PQ. (244)
Ionel Dorofte and Tatiana Dorofte
We strongly believe that the permanent and ostentatious joining of these scoring methodologies of «normal curve» and coefficient "IQ" is artificial or even abusive. We propose a new «metrical» system of performance based on the statistics of a reference group, either a real or ideal one (Gauss curve) the share or assigned scores need to be in an inverse ratio to the probability of occurrence of the respective value.. Within this study, we shall take into account an easier modelling calculating PQ depending on 1/p (we shall mark the Performance Coefficient by PQ and the probability of occurrence of the value by p). At the very heart of normal distribution where probability is of 0.4 the information shall be minimal and asymptotically increase towards the ends . We propose a new curve (boiler performance) and a new coefficient (PQ) which allows a significant leap in accuracy and understanding in what is measurement in science psychometrics. At this stage, we are interested in the qualitative presentation as an aspectual idea of the curve. We notice that in median area the value differentiation between individuals is minor at the ends; any increase of the performance is highlighted by the calculation formula.
Marginal likelihood approximation and Bayes Factor in Generalized Linear Latent Variable Models. (184)
Silia Vitoratou, Ioannis Ntzoufras and Irini Moustaki
Within the Generalized Linear Latent Variable Models context (GLVM; Moustaki and Knott 2000) we discuss the implementation of Bayesian measures of model complexity such as the Bayes Factor (BF; Kass and Raftery, 1995). Patz and Juncker (1999) initially proposed a Bayesian approach regarding the estimation of the parameters of a latent variable model with categorical responses. A-priori distributions are assigned to the model parameters as well as to the latent vector. A Markov chain, whose stationary distribution is the required posterior distribution P(α,β,z|x), is simulated via a Metropolis-Hastings within Gibbs algorithm (Chib and Greenberg, 1995). After a sufficiently log run of the chain, inference can be made about each parameter. We expand this work addressing the problem of approximating the marginal likelihood, over all parameters and for each competing model, involved in the calculation of the BF. Five methods proposed in the Bayesian literature are applied in the GLVM, namely:Harmonic mean estimator (Raftery et al, 2007), Importance sampling estimator (Newton and Raftery, 1994), Laplace estimator (Lewis and Raftery, 1997), Chib and Jeliazkov estimator (Chib and Jeliazkov, 2001), Power posterior estimator (Friel and Pettit, 2008).A comparison with respect to the accuracy and computational complexity of estimators is illustrated. References: Chib, S and Greenberg, E. (1995). Understanding the Metropolis–Hastings algorithm, American Statistician, 49, 327–335. Chib, S. and Jeliazkov, I. (2001). Marginal Likelihood from the Metropolis-Hastings Output. Journal of the American Statistical Association, 96, 270-281. Friel, N. and Pettit, A.N., (2008). Marginal likelihood estimation via power posteriors. Journal of Royal Statistical Society, 70, 589–607 Kass, R.E. and Raftery, A.E. (1995). Bayes Factor. Journal of the American Statistical Association, 90, 773–795, Lewis, S.M. and Raftery, A.E. (1997). Estimating Bayes Factors via Posterior Simulation with the Laplace-Metropolis Estimator. Journal of the American Statistical Association, 92, 648-655. Moustaki, I. and Knott, M. (2000). Generalised Latent Trait Models. Psychometrika, 65, 391-411. Newton, M.A. and Raftery, A.E. (1994). Approximate Bayesian Inference with the Weighted Likelihood Bootstrap, Journal of the Royal Statistical Society, 56, 3-48. Patz, R., and Junker, B. (1999). Applications and extensions ofMCMC in IRT: Multiple item types, missing data, and rated responses. Journal of Educational and Behavioral Statistics, 24, 342–366. Raftery, A., Newton, M., Satagopan, J. and Krivitsky, P. (2007). Estimating the integrated likelihood via posterior simulation using the harmonic mean identity. Bayesian Statistics, 8, 1–45.
A scaled effect size measure taking into account the distributions of tthevariables. (131)
Renske Kuijpers and M. A. L. M. van Assen
Many journals require authors to report effect sizes. Cohen (1988) constructed often used rules of thumb to interpret these effect size measures. However, these measures and rules do not take into account the distributions of the variables, which can lead to misleading interpretations. For instance, the maximum proportion of explained variance of a normally distributed dependent variable by a dichotomous independent variable is much lower (about 0.64) than 1 if the dependent variable is normally distributed. Furthermore, the larger the number of levels of an independent variable, the larger the maximum proportion of explained variance that can be obtained. We propose an effect size measure of explained variance that takes into account the distribution of the dependent variable and the number of levels of one independent variable. In addition, a program is constructed that calculates this new effect size measure for a given data set.
Statistical power to detect the correct number of classes in Latent Class Analysis. (160)
Jenn-Yun Tein, Heining Cham
Latent class analysis is useful to discern underlying groups based on observed variables. However, the application of such analyses to studies with only modest sample sizes is questionable. The purpose of this simulation study is to investigate Type I error rate and statistical power for detecting latent class structures with continuous or categorical indicator variables. To date, only a handful of studies have examined the effect of sample size in selecting the number of latent classes; however the simulated models are simplistic and limited for practice. In particular, the degree of separation (or distance) between variables, which we found to be an important factor for detecting the correct model is generally ignored. This simulation will be a fully factorial design with sample size, number of classes, number of variables, distance between variables, and probability of each class as factors of interests. For each condition considered, we will provide an estimate of the power and Type I error rate for favoring the correct model against an incorrect model which has one less or one more class. Model selection will be determined by the Bayesian information criteria (BIC) and the bootstrap likelihood ration test (BLRT).
Assessing dimensionality using different methodologies in complex assessment with dichotomous data. (241)
Youn Y. Choi and Dong Gi Seo
Recently, a complex assessment with dichotomously scored items has been utilizing in educational testing. Since most complex assessments usually contain more than one latent domain, identifying an internal test structure and relationship among dimensions can provide empirical evidence for hypothesized multidimensionality, validity of interpretation in a complex assessment, feedback to instructors and assessment developers. The purpose of this paper is to evaluate the capability of three approaches when detecting dimensions: linear factor analysis, non-linear factor analysis, and non-parametric IRT approach. This study focused initially on identifying limitations of traditional factor analysis method for utilizing Item Response Theory due to measurement scales and nonlinear relation between factors and item performance. In addition, an interpretation of test scores might not be valid when exist of guessing parameter, extreme item difficulties and slopes. Since previous researches have studied only limited conditions of either small number of dimensions (at least 2 dimensions) with simple structure, absence of guessing parameter, or large number of items and samples in this area, different kinds of conditions are investigated: (a) more than two dimensions, (b) present of guessing parameter, and (c) small number of samples and items (Knol and Berger, 1991; Ayala and Hertzog, 1991; Gosz and Walker, 2002; Tate, 2003; Stone and Yeh, 2006; Mroch and Bolt, 2006). In order to implement three methodologies, R software program for a linear factor analysis with tetrachoric correlation, NOHARM program (Fraser & McDonald,1988) for nonlinear factor analysis, TESTFACT program (Wood et al., 2002) based on full information item factor analysis, and DETECT (Kim, 1994; Zhang & Stout, 1999a) for non-parametric IRT model are utilized. Overall, linear factor analysis is the most sensitive to all conditions among three methods. In the absence of guessing assuming two dimensions, non-linear factor analysis-based programs (NOHARM and TESTFACT) showed the chi-square statistic having a very low Type I error rate, which had a similar power with DETECT program. In the presence of guessing assuming four dimensions, non-linear factor analysis-based programs showed the chi-square statistic with unacceptably high Type I error rate, while DETECT showed a consistent power of detecting dimensionalities with the absence of guessing condition assuming two dimensions.
Exploring the efficacy of latent class pattern mixture models for handling non-randomly missing data. (059)
Nisha Gottfredson and Daniel J. Bauer
Pattern mixture models can used to obtain unbiased parameter estimates when longitudinal data are missing due to a non-random process (Little, 1995). The advantage of pattern mixture models over selection models is that they require users to make few (often unrealistic) assumptions about the missingness mechanism. However, pattern mixture models become infeasible to use as the number of repeated measures increases or when there a large number of missing data patterns are present in the data. Latent class pattern mixture models have been touted as a way to extend the pattern mixture approach to these situations (Roy, 2003); yet the ability of these models to recover unbiased parameter estimates under a variety of missingness mechanisms has not been empirically evaluated. We present results of a simulation study in which we varied the missingness mechanism (random coefficient dependent missingness or outcome dependent missingness), number of repeated measures, and informativeness of the missingness (the least severe case being random missingness). We consider the bias and efficiency of the parameter estimates obtained from three variants of the the latent class pattern mixture model (Lin, McCulloch, and Rosenheck, 2004; Roy, 2006; Morgan-Lopez & Fals-Stewart, 2007).
Analysing censored longitudinal data with non-ignorable missing values. (134)
Milena Falcaro and Andrew Pickles
Longitudinal studies are often complicated by the presence of missing values and poorly measured outcomes. We here present a model for the analysis of censored longitudinal data with non-ignorable missing values. The repeated measures and the missing data mechanism are jointly modelled by assuming they depend on a common underlying process. Although the method is general, we consider a quadratic growth curve model for characterizing the correlation structure between the repeated measures and we allow latent components of the model to also influence the probability of dropout. The method makes allowance for complex but uninformative censoring by treating the recorded continuous outcomes as possibly censored ordinal variables and by using subject-specific thresholds. The underlying components of variance are analytically combined to form the covariance matrix of the joint distribution. Numerical integration is then undertaken over this multivariate distribution. The methodology is illustrated for the analysis of the Steel-Dyne Cognitive Ageing cohort of elderly men and women. We examine the longitudinal depression scores where participants were observed at individually-varying time points and statistical issues such as non-ignorable dropout and left censoring/floor effects needed to be taken into account.
Using a Cohort-Sequential Latent Growth Model to examine alcohol use from ages 16 to 29. (263)
Jeannette Brodbeck, Anna Brown and Tim J. Croudace
Research questions: Development of alcohol use from age 16 to 29 and viability of a Cohort-Seqential Latent Growth Model for repeated ordinal measures using mplus. Methods: A random sample of 2,844 Swiss urban adolescents and young adults aged 16 to 24 was interviewed in 2003, 2005 and 2008. Frequency of alcohol use was assessed from 0= never to 4=3-7 times a week. Using a Cohort-Sequential technique, longitudinal curves spanning a 13 year period are estimated in using only 5 years of data. Results: The linear model yielded a significant mean slope, suggesting a small but steady increase in the frequency of alcohol use from 17 -29. Sixteen year-olds increase their alcohol consumption faster than the other age groups. The variance for the intercept indicated considerable variation across individuals in initial frequency alcohol use. The variance of the slope yielded a small, but still significant variation in the increase of the frequency of use. Conclusion: The results are in line with other developmental studies of alcohol use in adolescents and young adults. The Latent Growth Model Cohort-Sequential multisample analysis proved to be an efficient method for examining time trends across longitudinal data.