Writing Multiple Choice Test Questions
Writing Multiple Choice Test Questions
Submitted by kaner on Wed, 24/10/2007 - 21:25.This is a tutorial on creating multiple choice questions, framed by Haladyna's heuristics for test design and Anderson & Krathwohl's update to Bloom's taxonomy. My interest in computer-gradable test questions is to support teaching and learning rather than high-stakes examination. Some of the design heuristics are probably different for this case. For example, which is the more desirable attribute for a test question:
1. defensibility (you can defend its fairness and appropriateness to a critic) or
2. potential to help a student gain insight?
In high-stakes exams, (a) [defensibility] is clearly more important, but as a support for learning, I'd rather have (b) [support for insight].
This tutorial's examples are from software engineering, but from my perspective as someone who has also taught psychology and law, I think the ideas are applicable across many disciplines.
The tutorial's advice and examples specifically target three projects:
* In the Black Box Software Testing Course [some course materials here], students take the multiple choice tests while they watch the video lectures or work through the assigned readings [research description here].
* We are following the same structure for learning units for graduate student instruction in software engineering ethics.
* In the Open Certification Project for Software Testing we are creating a public database of questions, with peer commentary/criticism. Anyone can review the questions, including people preparing for the exam. For the rationale behind this approach, see this paper by Kaner and Tim Coulter.
