Psychometrics and technology-enhanced education research
Keynote address by Ken Koedinger, Human-Computer Interaction Institute, Carnegie Mellon University, USA
Chair: Brian Junker, Thursday 23rd July, 8.30 - 9.30, Palmeston Lecture Theatre, Fisher Building.
Educational technologies are being increasingly used in schools and colleges. As just one example, the intelligent tutoring system component of our Cognitive Tutor Algebra course is used about two days a week by a half million students a year. On-line tutoring technologies provide students with individualized instructional support beyond that provided by teachers and textbooks alone. Well-designed systems can assess students as they work, adapt instruction to their individual needs, and provide stakeholders with detailed reports on students' strengths and weaknesses. Further, these systems provide a powerful research platform for data collection and experimentation to advance theories of learning, assessment, and instruction.
In this talk, I will focus on opportunities for transforming assessment. Can the continuous embedded assessment provided by on-line tutoring systems not only replace the need for one-shot standardized tests, but also provide richer, more useful, and timely information? I will discuss work from a team of psychologists, computer scientists, and statisticians exploring whether an on-line math tutoring system can accurately predict state test scores while at the same time enhancing student learning and helping teachers improve their teaching. I will also discuss the use of artificial intelligence and machine learning to go beyond cognitive assessment to create assessment models of student learning skills, like help-seeking or self-explanation, and student motivational states, like disengagement or flow. A key theme is how the timing data available from on-line student interactions provides research opportunities to develop new models of psychological states and processes, individual differences, and longitudinal change. All the data sets I will discuss (and many more) are available in an open data repository (pslcdatashop.web.cmu.edu). I hope this talk will encourage Psychometric researchers to dive in and make use of such data!