About 20 years ago, I participated in my very first item-writing workshop with the American Council on Exercise as part of a three-day exam validation exercise. This was one of those paradigm-shifting experiences for me as an educator because I had never been that immersed in exam accreditation before, but also because I never knew that there was an actual art and process to writing valid multiple choice questions. The key here is the word “valid.” Who knew that carefully constructed multiple choice questions could actually utilize all levels of Bloom’s taxonomy? I thought multiple choice questions were only for the two lower levels of the pyramid: remembering and comprehending. The experience of exam question item writing for accreditation completely changed the way I assessed my students.
Here are some of the basics, but if you search the internet for “writing better test questions” or “writing better multiple choice questions,” you will find some very good resources to help you write really good stems for all types of questions. As with everything we write, nothing can substitute for good planning before you start.
- Start with a test blueprint. By that, I mean start with your course or unit learning outcomes and decide what percentage of each outcome you will assign to your test. Divide up the number of questions accordingly. If you’re addressing five unit learning outcomes and there are 50 questions, then perhaps you would write 10 questions for each outcome, or any combination that represents what you want to emphasize. Then, divide the number of questions per outcome by the level of Bloom’s taxonomy (below) you wish to use. If you’ve got 10 questions to work with, then the breakdown might look like: 1 remember, 2 understand, 2 apply, 3 analyze, 1 evaluate, and 1 create.
- Use verbs in the stem (the first part of the question) that match the Bloom’s level you’re writing for. There are a million great resources like this one that will help you with question starters. Pay close attention to the “Key Words” for each level in the taxonomy.
- Write the stem as a complete question, even if it’s written as an incomplete sentence. The student should be able to answer the question without having to read all of the answer options.
- Always use plausible incorrect answers (the distractors). This also means making sure the tense matches so that students can’t automatically eliminate an answer based on grammar alone. The best distractors help diagnose where the student went wrong in their thinking.
- Avoid negative items such as NOT or EXCEPT, as well as “none of the above” or “all of the above.” Only use the negative options when it is important that students know what not to do. The same applies to “never” or “always.”
- Make the answer options about the same length so that the correct answer is not obvious just because it’s longer or more detailed.
These are just a few pointers but understand that good test questions are inherently difficult to write! For this reason, it’s also a good idea to take measures to avoid having your test questions out there so that you don’t have to write new ones each time to you teach the class. For a list of 30 tips for writing good multiple choice questions, click here!