Skip navigation.

Writing Multiple Choice Test Questions

Writing Multiple Choice Test Questions

This is a tutorial on creating multiple choice questions, framed by Haladyna's heuristics for test design and Anderson & Krathwohl's update to Bloom's taxonomy. My interest in computer-gradable test questions is to support teaching and learning rather than high-stakes examination. Some of the design heuristics are probably different for this case. For example, which is the more desirable attribute for a test question: 1. defensibility (you can defend its fairness and appropriateness to a critic) or 2. potential to help a student gain insight? In high-stakes exams, (a) [defensibility] is clearly more important, but as a support for learning, I'd rather have (b) [support for insight]. This tutorial's examples are from software engineering, but from my perspective as someone who has also taught psychology and law, I think the ideas are applicable across many disciplines. The tutorial's advice and examples specifically target three projects: * In the Black Box Software Testing Course [some course materials here], students take the multiple choice tests while they watch the video lectures or work through the assigned readings [research description here]. * We are following the same structure for learning units for graduate student instruction in software engineering ethics. * In the Open Certification Project for Software Testing we are creating a public database of questions, with peer commentary/criticism. Anyone can review the questions, including people preparing for the exam. For the rationale behind this approach, see this paper by Kaner and Tim Coulter.