Education research applications (01)

Chair: Kei Miazaki, Tuesday 21st July, 10.50 - 12.10. Boys Smith Room, Fisher Building 

Yung-Mei TsengInstitute of Education, National Kaohsiung Normal University, Taiwan. The use of generalizability theory in the application of elementary writing. (224)

Chi-Ling Lin and Shu-Ching Yang, Graduate Institute of Education, National Sun Yat-Sen University, Taiwan. The meta-analysis of effect research of the life meaning sense in Taiwan. (117)

Tim GaffneyCalifornia Department of Education, Standards and Assessment Division, Sacramento, CA, USA, Cassandra Perryman, Department of Psychology, California State University, Sacramento CA, USA.  A longitudinal look at the factor structure of educational achievement tests. (226)

ABSTRACTS

The use of generalizability theory in the application of elementary writing. (224)
Yung-Mei Tseng
In this study, we use generalizability theory to understand the students who score in the writing and the associated rater consistency. A total of 22 fourth graders in Taiwan were administered the subjects. These students wrote three different styles of writings, and three elementary school teachers were requested as rater. Three main findings were identified: (1) there was interaction between students and the style of writing. (2) raters were not influenced by the style of writing. (3) there was difference among the raters. Implications of these findings for study and future research directions are discussed.

The meta-analysis of effect research of the life meaning sense in Taiwan. (117)
Chi-Ling Lin and Shu-Ching Yang
Using meta-analysis, this study reviews the results of the quantitative studies in Taiwan between 1999 September and 2008 July dealing with quai-experiments on life education and death education. Meta-analysis and Hierarchical Linear Model were employed to explore the overall effect of teaching and search for possible moderating variables based on 20 master/doctoral thesis and 4 journal articles. Results indicate that: (a) with respect to overall effect, the overall teaching effect regarding life meaning sense is medium(=0.468); (b) with teaching intervention, educational stages characterized by senior high and vocational school giving much higher effect, while gender and sample size are moderating variables. Whether the researcher and instructor in pre-test and post-test research design of the study were the same person or not were found to have little impact on teaching effect.

A longitudinal look at the factor structure of educational achievement tests. (226)
Tim Gaffney and Cassandra Perryman
This research analyzed the subtest-level factor structure of California’s criterion referenced educational achievement tests used for high-stakes educational accountability purposes. It is one of several follow-up studies on the unidimensionality exhibited by these assessments across content domains as presented to the Psychometric Society at Tilburg in 2005. In the current study, factor analysis (including exploratory, confirmatory, and hierarchical models) of longitudinal subtest scores (an elementary and a high school cohort, N = 400,000 and 216,000 respectively) over a three year period assessed (a) the convergent and discriminant validity of test content domains, and (b) performance when high-stakes outcomes shift from the school to the test-taker. It will be shown that (a) weak but identifiable verbal and quantitative factors were present, (b) factors emerge that appear to represent the shift of accountability from school to test-taker, and (c) constructs within contend domain (e.g., reading and writing) again failed to emerge even in the longitudinal analysis of subtests as opposed to items. The meaning of these findings and their implications for generating diagnostic and remedial strategies for educators from educational (multiple-choice) achievement tests will be discussed.