Seamless evaluation integration into IDS educational games

Serious games have recently enjoyed unprecedented interest among researchers. Despite the fact that the main focus of this domain remains on player entertainment and engagement, serious games have established themselves as an alternative educational paradigm. However, little interest has been given to investigating methods for the assessment of their effectiveness. Game evaluation focus remains on investigating usability and technical aspects. Those attempting to evaluate the educational impact typically rely on traditional questionnaires that typically negatively affecting user immersion and the overall game enjoyment. Among game genres, Interactive Digital Storytelling (IDS) is a fast growing genre that merges computer games, multimedia and cinematic storytelling with entertaining education. Like any other teaching intervention, educational IDS’s have to be evaluated against a defined set of Learning Objectives (LOs). In this paper we present a User-Centred Seamless Evaluation Framework for IDS games and describe the algorithm for the integration of knowledge assessment to measure knowledge improvement against given LOs without affecting player enjoyment. We implement this algorithm in an educational game and show that the framework has been perceived by the majority of the players as a positive enhancement to the game, 94% of the participants reported preferring this form of assessment as opposed to the more “traditional” methods. Statistically significant knowledge improvements were obtained after the game play.