How to Write Exceptional Multiple Choice Questions for Your Anatomy Courses
by Robert Tallitsch, PhD | December 2, 2019
As we’ve discussed in previous blogs, student engagement facilitates learning! But we must also assess what our students know and understand. Because many of our students will take board exams in the future, it is crucial to teach them how to answer those tricky, in depth multiple-choice questions (MCQs). But…as many of you already know, it can be just as difficult to write MCQs that truly assess what our students are learning in class. In addition, justifying MCQs to those students that insist your tests are “tricky” is even harder.
But what if those students are correct? Maybe your questions ARE tricky, and maybe your multiple-choice questions AREN'T actually testing what they know and understand. So, let’s talk about how you can write fair multiple-choice questions that truly assess what your students are learning and comprehending. To begin, let's take a look at the makeup of the example question below:
Which muscle(s) pass through the axilla (arm pit) and inserts onto the intertubercular sulcus (groove) of the humerus?
- infraspinatus
- supraspinatus
- subscapularis
- teres major
- teres minor
The prompt, or stem of the question reads “Which of the following muscle(s) pass through the axilla (arm pit) and inserts onto the intertubercular sulcus (groove) of the humerus?” The answer set is comprised of choices “1” through “5”. The distractors (incorrect answers) are choices “1” through “3” and choice "5”; while choice “4” is the correct answer. Also note that the definition of “axilla” and the alternative term for intertubercular sulcus are in the prompt. This reduces the possibility of the student getting the question wrong if they don’t know some of the key terms in the prompt.
3 Rules to Follow When Writing MCQs
1. The prompt is the toughest part of the question to write. Each prompt should contain sufficient information such that a student that knows the subject matter can give the correct answer without even looking at the answer set. In addition, SOTL (Study of Teaching and Learning) literature states that the prompt:
- must be less than, or equal to five lines in length,
- contains only relevant information,
- shouldn’t utilize grammar and syntax that eliminates potential answers,
- avoids abbreviations, and
- is longer than the choices in the answer set.
2. SOTL literature also states that all of the choices should be
- plausible answers,
- homogeneous in content,
- similar in terms of construction and length,
- arranged in alphabetical or numerical order, and
- avoids abbreviations.
3. Finally, SOTL literature advises that writers of MCQs
- do not use “all of the above” or “none of the above” choices, and
- should avoid the use of distractors that are opposites.
High quality MCQs also require the students to perform higher-order thinking skills (according to Bloom)! To do so, try not to focus on making the questions harder. Try to focus on making the student use more thought processes while answering your questions. For an example of this, note the differences in these two examples below:
Which muscle or muscle group is the major extensor of the thigh?
- gluteus maximus
- gluteus minimus
- hamstrings
- iliopsoas
- obturator internus
A 65-year-old male has difficulty rising from a seated position and straightening his trunk. He has no difficulty flexing his leg. Which of the following muscles is most likely injured?
- gluteus maximus
- gluteus minimus
- hamstrings
- iliopsoas
- obturator internus
As shown, the two prompts are essentially asking the same question. However, the second prompt includes a real-life scenario similar to what students will see in their board exams and in their professional careers. Structuring your MCQs in this manner requires a student to utilize higher-order thinking skills and additional thought processes prior to answering the questions at stake. It is also important to write an exam that is composed of a significant proportion of questions that test each thinking skill. A test with a disproportionate number of questions in any of those categories will not allow you to distinguish the thinking skill(s) your students cannot achieve.
Assessing Multiple-Choice Exam Results
Now…your exam is written, the students have taken it, and you have graded it. How do you tell if all the questions were written properly and were fair? Here is a quick and dirty way that satisfied me and my students.
- After grading the exams, separate out the top 5 or 7 and bottom 5 or 7 exam grades, depending on the size of your class.
- Using a different color ink for each group go through each exam and, on your answer key, mark questions that were missed by each of these students.
- Any question missed by a majority of both groups should be omitted, as there was something in that question that the students in each group did not understand.
- If no questions were missed by a majority of each group congratulations; all questions were fair and understandable. But, if any question was missed by a majority of both groups you have to compensate for the bad question(s). You must do one of the following:
- Omit any question that was missed by a majority of both groups and give every student in the class credit for that question, whether they answered it correctly or not.
- When you write your test, you work on the assumption that a multiple-choice test of 50 questions will have 1 to 3 “bogus” questions in it, so you construct the test with 53 questions. If you do this your grading scale is based on only 100 points, and not 106 (assuming each question is worth 2 points), even though one or more students may score 106 on the exam. This automatically eliminates any bias from a “bogus question.” If you find that more than 3 questions are missed by a majority of both groups you will need to give all students credit for the additional bogus question(s), whether or not they answered them correctly.
Why do I suggest giving all students credit for a bogus question whether or not they answered it correctly? You do not know whether or not students spent more time on the poorly worded question(s) and, as a result, had to rush to finish the exam, thereby increasing the possibility of missing one or more questions than they would have if all of the questions had been worded correctly. In other words, when in doubt the student gets the break.
As you begin preparing and/or updating your anatomy assessment material for the upcoming semester, I hope you find value in some of the methods I've utilized over the past 43 years. Yes, getting your students engaged facilitates learning. However, as anatomy educators it is just as important to assess our students fairly and properly.
Additional Articles: