Test Construction

Test Construction

Most tests are a form of summative assessment; that is, they measure students’ performance on a given task. (For more information on summative assessment, see the CITL resource on formative and summative assessment.) McKeachie (2010) only half-jokes that “Unfortunately, it appears to be generally true that the examinations that are the easiest to construct are the most difficult to grade.” The inverse is also true: time spent constructing a clear exam will save time in the grading of it.

Closed-answer or “objective” tests

By “objective” this handbook refers to tests made up of multiple choice (or “multi-op”), matching, fill-in, true/false, fill-in-the-blank, or short-answer items as objective tests. Objective tests have the advantages of allowing an instructor to assess a large and potentially representative sample of course material and allow for reliable and efficient scoring. The disadvantages of objective tests include a tendency to emphasize only “recognition” skills, the ease with which correct answers can be guessed on many item types, and the inability to measure students’ organization and synthesis of material

Since the practical arguments for giving objective exams are compelling, we offer a few suggestions for writing multiple-choice items. The first is to find and adapt existing test items. Teachers’ manuals containing collections of items accompany many textbooks. However, the general rule is “adapt rather than adopt.” Existing items will rarely fit your specific needs; you should tailor them to more adequately reflect your objectives.

Objective-answer tests can be constructed to require students to apply concepts, or synthesize and analyze data and text. Consider using small “cases studies,” problems or situations. Provide a small collection of data, such as a description of a situation, a series of graphs, quotes, a paragraph, or any cluster of the kinds of raw information that might be appropriate material for the activities of your discipline. Then develop a series of questions based on that material, the answers to which require students to process and think through the material and question significantly before answering.

Here are a few additional guidelines to keep in mind when writing multiple-choice tests:

  • As much of the question as possible should be included in the stem.
  • Make sure there is only one clearly correct answer (unless you are instructing students to select more than one).
  • Make sure the correct answer is not given away by its being noticeably shorter, longer, or more complex than the distractors.
  • Make the wording in the response choices consistent with the item stem.
  • Beware of using answers such as “none of these” or “all of the above.”
  • Use negatives sparingly in the question or stem; do not use double negatives.
  • Beware of using sets of opposite answers unless more than one pair is presented (e.g., go to work, not go to work).

Essay exams

Conventional wisdom accurately portrays short-answer and essay examinations as the easiest to write and the most difficult to grade, particularly if they are graded well. You should give students an exam question for each crucial concept that they must understand.

If you want students to study in both depth and breadth, don't give them a choice among topics. This allows them to choose not to answer questions about those things they didn’t study. Instructors generally expect a great deal from students, but remember that their mastery of a subject depends as much on prior preparation and experience as it does on diligence and intelligence; even at the end of the semester some students will be struggling to understand the material. Design your questions so that all students can answer at their own levels.

The following are some suggestions that may enhance the quality of the essay tests that you produce

  • Have in mind the processes that you want measured (e.g., analysis, synthesis).
  • Start questions with words such as “compare,” “contrast,” “explain why.” Don’t use “what,” “when,” or “list.” (These latter types are better measured with objective-type items).
  • Write items that define the parameters of expected answers as clearly as possible.
  • Make sure that the essay question is specific enough to invite the level of detail you expect in the answer. A question such as “Discuss the causes of the American Civil War,” might get a wide range of answers, and therefore be impossible to grade reliably. A more controlled question would be, “Explain how the differing economic systems of the North and South contributed to the conflicts that led to the Civil War.
  • Design the question to prompt students’ organization of the answer. For example, a question like “Which three economic factors were most influential in the formation of the League of Nations?”
  • Don’t have too many questions for the time available.
  • For take-home exams, indicate whether or not students may collaborate and whether the help of a Writing Tutorial Services tutor is permissible.

Grading essay exams

A more detailed discussion of grading student work is offered in evaluating student written work and applies to handling essay exams as well.

However, unlike formal essays, essay exams are usually written in class under a time limit; they often fall at particularly busy times of the year like mid-term and finals week. Consequently, they are differently stressful for students, and as a result you may encounter errors and oversights that do not appear in formal essays. Similarly, it is not unusual to find essays that do not provide responses we have anticipated.

Your grading changes in response. Adjustments to the grading scale may be necessary in light of exam essays that provide answers you had not anticipated. Comments may be briefer, and focused primarily on the product students have produced; that is, exams do not require suggestions for revision.