Predicting students performance with SimStudent that learns cognitive skills from observation

Matsuda, N., Cohen, W. W., Sewall, J., Lacerda, G., & Koedinger, K. R. (2007). Predicting students performance with SimStudent that learns cognitive skills from observation. In R. Luckin, K. R. Koedinger & J. Greer (Eds.), Proceedings of the international conference on Artificial Intelligence in Education (pp. 467-476). Amsterdam, Netherlands: IOS

Abstract: SimStudent is a machine-learning agent that learns cognitive skills by demonstration. SimStudent was originally built as a building block for Cognitive Tutor Authoring Tools to help an author build a cognitive model without heavy programming. In this paper, we evaluate a second use of SimStudent for student modeling for Intelligent Tutoring Systems. The basic idea is to have SimStudent observe human students solving problems with an intelligent tutoring system. It then induces a cognitive model that replicates their performance on these problems. If the model is accurate, it predicts the human students' performance on novel problems. An evaluation study with log data from genuine student-tutor interactions showed that when trained on 15 problems, SimStudent accurately predicted the human students' correct behavior more than 80% of the time (i.e., it can tell when the human students make correct steps). However, the current implementation of SimStudent does not accurately predict when the human students make errors. This paper gives a detailed description of the evaluation study and discusses the lessons learned.

(234KB PDF file)