Algorithmic Exam Generation

Omer Geiger, M.Sc. Thesis Seminar
Wednesday, 3.6.2015, 14:00
Taub 601
Prof. Shaul Markovitch

Given a class of students, and a pool of questions in the domain of study, what subset will constitute a ``good'' exam? Millions of educators are dealing with this difficult problem worldwide, yet the task of composing exams is still performed manually. In this work we present a novel algorithmic framework for exam composition. Our framework requires two input components: a student population represented by a distribution over a set of overlay models, each consisting of a set of mastered abilities, or actions; and a that, given any two student models, defines which should be graded higher. To determine the performance of a student model on a potential question, we test whether it satisfies a disjunctive action landmark, i.e., whether its abilities are sufficient to follow at least one solution path. Based on these, we present a novel utility function for evaluating exams. An exam is highly evaluated if it is expected to order the student population with high correlation to the target order. The merit of our algorithmic framework is exemplified with real auto-generated questions in the domain of middle-school algebra.

Back to the index of events