Some Things Every Instructor Should Know About Constructing Test Questions... And More

by Robert Tallitsch, PhD | August 25, 2026

During my career as a college professor I was frequently told that I was “an anomaly” because I enjoyed writing and grading tests. I was intrigued and challenged by the process of writing good tests. Each time I wrote an exam I wanted to answer at least the three questions listed below.

Was the test fair?
Did my students truly learn the material?
Were my students able to utilize the information to solve complex problems?

Then, once the tests were written and administered, I couldn’t wait to see how my students had done and whether or not I had succeeded in the above goals. Another goal, which I almost always met, was to have the graded exams back in the students’ hands during the next class period, regardless of the class size. This provided timely feedback for the students, thereby enabling them to change their study habits prior to the next exam, if change was indeed needed.

As I look back on the tests I wrote early in my career I literally cringe, in that I now know those tests really didn’t tell me much about what my students had learned. Rather, they told me that I wrote tests that were ambiguous at best, tricky at worst. They also were written in such a way that those students who could figure out what I was asking in the test questions did well, and those that couldn’t didn’t. Simply put, I wasn’t achieving any of the three goals listed above. As I matured as a teacher I attended more workshops and read more of the Study of Teaching and Learning (SoTL) literature about better test-writing practices. As a result, I was able to regularly achieve the three goals I listed above. So....let me help you write better test questions that utilize higher-order thinking skills, be they multiple-choice, true-false, or essay questions, and that help you answer the same three questions I always asked myself.

Let’s start with multiple-choice (MC) questions. One of the best presentations on the construction of multiple-choice questions that I ever heard was by Dr. Brian Hill, who was, at that time, the Vice Dean for Post-Baccalaureate Education at the Virginia College of Osteopathic Medicine (Virginia Campus). His presentation was at the 2012 Human Anatomy and Physiology Society (HAPS) Annual Meeting in Tulsa, OK. In this presentation, as well as subsequent ones of his that I attended, Dr. Hill made very important points regarding the construction of excellent multiple-choice questions. The major points made by Dr. Hill were:

All high-stakes and licensure exams utilize MC questions.
All of these exams have a strict writing/format criteria, which are based on best practices within the SoTL literature.
The stem of a MC question is the most difficult part of the question to write.
- Each stem should contain sufficient information such that knowledgeable students should be able to give the correct answer without looking at the answer set.
- The stem should be positively worded versus negative worded, should be no more than five lines in length, and the grammar and syntax of the stem should not eliminate any potential answers.
- The stem and answer set should avoid abbreviations and the use of imprecise phrases, such as ‘associated with’, ‘often’, ‘usually’, ‘frequently’, etc..
- The stem should be longer than the choices within the answer set.
The answer set should always
- be composed of plausible answers,
- be arranged in alphabetical or numerical order,
- be similar in terms of construction and length, and
- avoid the use of distractors that are opposites.

When worded correctly, multiple-choice questions can be constructed in such a way as to involve higher-order thinking skills. To do this don’t focus on making the question harder; do focus on making the learner use more thought processes, as demonstrated in the three examples below.

Here is an example of a multiple-choice question that uses lower-order thinking skills:

Which muscle or muscle group is the major extensor of the thigh?

a. Gluteus maximus

b. Gluteus minimus

c. Hamstrings

d. Iliopsoas

e. Obturator internus

Here is an example of a multiple-choice question that uses higher-order thinking skills:

A 65-year-old individual has difficulty rising from a seated position and straightening his trunk, but has no difficulty flexing the leg. Which of the following muscles is most like to have been injured?

a. Gluteus maximus

b. Gluteus minimus

c. Hamstrings

d. Iliopsoas

e. Obturator internus

And here is an example of a multiple-choice question that uses even higher-order thinking skills:

A 65-year-old individual with a peripheral nerve injury has difficulty rising from a seated position and straightening the trunk, but has no difficulty flexing the leg. Which of the following nerves is most likely to have been injured?

a. Femoral nerve

b. Inferior gluteal nerve

c. Obturator nerve

d. Sciatic nerve

e. Superior Gluteal nerve

In the three questions above the first question only requires the student to know which muscle is the major extensor of the thigh. However, the second question requires the student to understand (a) the position that the joints of the hip and knee are in when seated, (b) the definition of straightening the trunk, (c) the anatomical definitions of the terms trunk and leg; (d) the definition of flexing the leg, and (e) which muscles have been injured such that the patient has difficulties rising from a seated position and straightening the trunk. The third question is the most difficult, and involves the most thought processes (i.e. highest-order thinking skills). In the third question the student needs to know not only all the information required to answer the second question but also needs to know what nerves innervate all the muscles listed in the question as well as the muscles involved in rising from a seated position and straightening the trunk.

Most instructors are able to construct multiple-choice questions that test knowledge and comprehension. Fewer instructors, however, construct tests that test application, analysis, synthesis and evaluation. In addition, as a test writer instructors must work to ensure that a multiple-choice test is composed with a defined proportion of questions from all levels of Bloom’s taxonomy.

Now, let’s turn to true-false questions. I am always surprised by comments that generally frown on the use of true-false questions on examinations. Many of the comments center around the belief that true-false questions are either ambiguous, trivial, unable to test higher-order thinking skills, and/or susceptible to guessing. That said, several SoTL investigators point out that these opinions may be unfounded. The SoTL literature argues that the problem with true-false questions is not with the form; rather the problem is with the writer of the questions. In addition, SoTL literature has shown that the validity of true-false questions is not essentially different from multiple-choice questions if the true-false questions are written correctly.

The SoTL literature lists four essentials to writing good true-false test questions:

Choose a significant idea.
Devise a problem which will require the understanding or application of the idea.
Word the statement of the problem so that those who lack understanding of the element being tested will be attracted to a wrong answer.
Review the statement critically to make sure that any who do understand the point being tested should be able to answer it correctly.

A process that I have used that almost always resulted in a good true-false is as follows:

Start with a well-written multiple-choice question (such as the last two examples given above).
Rewrite the stem of the multiple-choice question, utilizing one of the choices of the question, in such a way that it is now a true-false question.

For example:

True or false?

A 65-year-old individual has difficulty rising from a seated position and straightening his trunk, but has no difficulty flexing his leg. This musculoskeletal difficulty arises from damage to the femoral nerve.

This question contains the same information found in the well-written multiple-choice question found above. Like the multiple-choice question, this true-false question requires the student to understand and utilize the same anatomical knowledge to correctly arrive at the answer. However, it has been rewritten in the format of a true-false question rather than a multiple-choice question. This process can be followed in any discipline that is suitable for testing by multiple-choice questions.

At least to me, proofreading is one of the most difficult tasks associated with writing. This is especially true when writing an exam. Because of that, ambiguous or awkward syntax would often go undetected in my examinations—no matter how hard I tried to proofread them. To eliminate this problem I developed the following practice whenever I wrote and graded an objective test:

The total possible number of points on the exam would be one hundred and four for a fifty-question test (with two points per question).
Grades would be calculated out of a possible one hundred points.
Prior to recording the grades for the test I would separate out the seven tests with the highest grades and the seven tests with the lowest grades. (My average course enrollment was 60 or more students.) Then I would determine which test questions were missed by each of these students.
Any question missed by a majority of both groups was deemed aberrant and, therefore, discarded.
If only one or two aberrant questions were found then grades stood as given, as the number was within the average of one or two aberrant questions for a fifty question objective test. However, if more than two questions were deemed aberrant those questions were discarded and the appropriate number of points were added to every student’s grade. (For instance, if three questions were found to be aberrant in the item analysis two additional points were added to each student’s grade.)

Utilizing this procedure for any test that contained objective questions eliminated almost all conversations with students about poorly worded test questions.

Now let’s move on to essay questions. Too many faculty members avoid writing essay questions (especially for anatomy courses), for a variety of reasons. Three of the most common reasons, based on my discussions with colleagues, are:

Writing an excellent essay question is no easier than writing an excellent true-false question or an excellent multiple-choice question.

The logistics of including one or more essay questions on an exam may not be feasible given the class size, making the grading of an essay exam longer.

Essay exams/questions open the instructor to the possibility of numerous student questions or disagreements regarding how their question/exam was graded.

SoTL literature demonstrates that essay tests demonstrate higher-order thinking processes and may be a more authentic form of assessment, as compared to objective examinations. Numerous SoTL studies have demonstrated that an essay question, regardless of discipline, is the best way to emphasize effective communication skills, thought processes, and abilities to organize information.

Regardless of what the SoTL literature says are the benefits of the essay exam, a poorly written essay question or essay test does none of the above—just as with a poorly written true-false or multiple-choice exam. Therefore one must consider the following SoTL recommendations when writing an essay question or essay test:

Provide clear instructions and wording in your question prompt. Too often students think that unless they have completely filled the blue book or the space provided on the exam page their question is inadequate and will not result in a high grade. Prompts like “discuss” and “describe” typically elicit poor responses. More appropriate prompts would be the use of verbs, such as “analyze,” “compare”, “summarize”, or “justify”. Prompts such as these need to be “contextualized” for students. For example, asking a student to ‘State the impact of the Dust Bowl’ provides few cues to students regarding instructor expectations. What exactly does ‘discuss’ mean to the instructor? What kind of impact? And on whom? A clearer version of this question might read ‘Identify and explain the long-term impact of the Dust Bowl on the American economy.’

Specify your expectations and grading procedures. Prior to the administration of an essay exam, students should know the total time allowed for the exam, the point value of each question, how each response will be graded, the priorities of each component contained within the students’ answers, and your expectations with respect to the overall grading of writing mechanics (grammar, punctuation, etc.).

Use a grading rubric! The use of a well-devised grading rubric fosters student understanding and improves student confidence in the grading process. SoTL literature also emphasizes that students should receive a copy of your grading rubric prior to the day of the examination, and that such a practice facilitates student preparation for the examination.

Whenever I administered a test that contained one or more essay questions I always posted this statement as a header for the essay question section of the exam:

Bob’s red-pen rule: “If, for some reason, you do not know how to start answering any question in this section simply come up to me at the front of the room and say so. I will provide you with a clue to the answer. However, I will also mark in the margin of the test, in red pen, the number of points deducted for the help I provided to you to start answering the question.”

This red-pen rule eliminated the possibility of a student receiving zero credit for an entire question simply because they didn’t understand one or more concepts within the question prompt. In addition, most of the time students would be able to go back to their seat, think about what I wrote in the margin, and finish the question, earning significantly more points than they would have without the red-pen rule simply because they drew a blank on how to start their answers.

As I stated at the beginning of this piece, learning how to write good exam questions, be they objective or essay, is one of, if not the hardest process for an instructor. Dig into the SoTL literature; talk to colleagues; work as hard at this aspect of your teaching as you do in organizing your class and presentations. Then and only then will you be able to know whether or not your students are truly able to know the information you are covering and utilize it in order to solve complex clinical problems.