Evaluating a simulated student using real students data for training and testing

Matsuda, N., Cohen, W. W., Sewall, J., Lacerda, G., & Koedinger, K. R. (2007). Evaluating a simulated student using real students data for training and testing. In C. Conati, K. McCoy & G. Paliouras (Eds.), Proceedings of the international conference on User Modeling (LNAI 4511) (pp. 107-116). Berlin, Heidelberg: Springer

Abstract: The Simulated Students are machine-learning agents that learn cognitive skills by demonstration. They were originally developed as a building block for the Cognitive Tutor Authoring Tools (CTAT) so that the authors do not have to build a cognitive model by hand, but instead simply demonstrate solutions for the Simulated Students to automatically generate a cognitive model. The Simulated-Student technology could then be used to model human students' performance as well. To evaluate applicability of the Simulated Students as a tool for modeling real students, we applied the Simulated Students to a genuine learning log gathered from classroom instructions using the Algebra I Cognitive Tutor. Such data can be seen as the human students' "demonstrations" on how to solve problems. The results from the empirical study show that the Simulated Students can indeed model human students' performances. After training on 20 problems solved by a set of human students, a cognitive model generated by Simulated Students explained 82% of the problem-solving steps performed correctly by another set of human students.

PDF file (228KB)