Technical Report MSC-2016-06

Title: Algorithmic Exam Generation
Authors: Omer Geiger
Supervisors: Shaul Markovitch
PDFCurrently accessibly only within the Technion network
Abstract: Given a class of students, and a pool of questions in the domain of study, what subset will constitute a ``good'' exam? Millions of educators are dealing with this difficult problem worldwide, yet the task of composing exams is still performed manually. In this work we present a novel algorithmic framework for exam composition. Our main formulation requires two input components: a student population represented by a distribution over a set of overlay models, each consisting of a set of mastered abilities, or actions; and a target-model-ordering that, given any two student models, defines which should be graded higher. To determine the performance of a student model on a potential question, we test whether it satisfies a disjunctive action landmark, i.e., whether its abilities are sufficient to follow at least one solution path. Based on these, we present a novel utility function for evaluating exams. An exam is highly evaluated if it is expected to order the student population with high correlation to the target order. In an alternative formulation we devised, the target ordering is replaced with a target grade mapping indicating the desired grade for each student model. In this case, good exams are those for which the expected grades are close to those specified by the target mapping. The merit of our algorithmic framework is exemplified with real auto-generated questions in two domains: middle-school algebra and trigonometric equations.
CopyrightThe above paper is copyright by the Technion, Author(s), or others. Please contact the author(s) for more information

Remark: Any link to this technical report should be to this page (, rather than to the URL of the PDF files directly. The latter URLs may change without notice.

To the list of the MSC technical reports of 2016
To the main CS technical reports page

Computer science department, Technion