Designing Test Questions
Descriptions follow with uses, advantages, disadvantages, and tips for writing test questions in the following formats.
- Assessing knowledge-level content
- Evaluating student understanding of popular misconceptions
- Concepts with two logical responses
- Can test large amounts of content
- Students can answer 3-4 questions per minute
- They are usually easy (from the student's perspective)
- It is difficult to discriminate between students that know the material and students who don't
- Students have a 50-50 chance of getting the right answer by guessing
- Need a large number of items for high reliability
Tips for Writing Good True/False items:
- Avoid double negatives.
- Avoid long/complex sentences.
- Use specific determinants with caution: never, only, all, none, always, could, might, can, may, sometimes, generally, some, few.
- Use only one central idea in each item.
- Don't emphasize the trivial.
- Use exact quantitative language.
- Don't lift items straight from the book.
- Make more false than true (60/40). (Students are more likely to answer true.)
- Assessing knowledge level information
- Some comprehension level, if appropriately constructed
- Terms with definitions
- Phrases with other phrases
- Causes with effects
- Parts with larger units
- Problems with solutions
- Maximum coverage at knowledge level in a minimum amount of space/prep time
- Valuable in content areas that have a lot of facts
- Time consuming for students to complete.
- Not good for higher levels of learning.
Tips for Writing Good Matching items:
- Need 15 items or less.
- Give good directions on basis for matching.
- Use items in response column more than once (reduces the effects of guessing).
- Use homogenous material in each exercise.
- Make all responses plausible.
- Put all items on a single page.
- Put response in some logical order (chronological, alphabetical, etc.).
- Responses should be short.
- Application, synthesis, analysis, and evaluation levels
- Question/Right answer
- Incomplete statement
- Best answer
- Very effective
- Versatile at all levels
- Minimum of writing for student
- Guessing reduced
- Can cover broad range of content
- Difficult to construct good test items
- Difficult to come up with plausible distractors/alternative responses
Tips for Writing Good Multiple Choice items:
- Stem should present single, clearly formulated problem.
- Stem should be in simple, understood language; delete extraneous words.
- Avoid "all of the above"--can answer based on partial knowledge (if one is incorrect or two are correct, but unsure of the third...).
- Avoid "none of the above."
- Make all distractors plausible/homogeneous.
- Don't overlap response alternatives (decreases discrimination between students who know the material and those who don't).
- Don't use double negatives.
- Present alternatives in logical or numerical order.
- Place correct answer at random (A answer is most often).
- Make each item independent of others on test.
- Way to judge a good stem: student's who know the content should be able to answer before reading the alternatives
- List alternatives on separate lines, indent, separate by blank line, use letters vs. numbers for alternative answers.
- Need more than 3 alternatives, 4 is best.
- Application, synthesis, analysis, and evaluation levels
- Easy to construct.
- Good for "who," what," where," "when," content.
- Minimizes guessing.
- Encourages more intensive study—student must know the answer vs. recognizing the answer.
- May overemphasize memorization of facts
- Take care—questions may have more than one correct answer
- Scoring can be laborious
Tips for Writing Good Short Answer Items:
- When using with definitions: supply term, not the definition—for a better judge of student knowledge.
- For numbers, indicate the degree of precision/units expected.
- Use direct questions, not an incomplete statement.
- If you do use incomplete statements, don't use more than two blanks within an item.
- Arrange blanks to make scoring easy.
- Try to phrase question so there is only one answer possible.
- Application, synthesis and evaluation levels
- Extended response: synthesis and evaluation levels; a lot of freedom in answers
- Restricted response: more consistent scoring, outlines parameters of responses
- Students less likely to guess
- Easy to construct
- Stimulates more study
- Allows students to demonstrate ability to organize knowledge, express opinions, show originality
- Can limit amount of material tested, therefore has decreased validity
- Subjective, potentially unreliable scoring
- Time consuming to score unless good rubric is designed
Tips for Writing Good Essay Items:
- Provide reasonable time limits for thinking and writing.
- Avoid letting students answer a choice of questions. (You won't get a good idea of the broadness of student achievement when they only answer a set of questions.)
- Give definitive task to student--ask them to compare, analyze, evaluate, etc. (and consider defining what those terms mean to you.)
- Use checklist point system to score with a model answer: write outline, determine how many points to assign to each part
- Score one question at a time, all at the same time.
- Knowledge, synthesis, evaluation levels
- Useful as an instructional tool—allows students to learn at the same time as testing
- Allows teacher to give clues to facilitate learning
- Useful to test speech and foreign language competencies
- Time consuming to give and take.
- Could have poor student performance because they haven't had much practice with presentation.
- Provides no written record without checklists or video/audiotaping.
- Knowledge, application, synthesis, evaluation levels
- Can assess compatible skills: writing, documentation, critical thinking, problem solving
- Can allow student to present totality of learning
- Students become active participants in the evaluation process
- Can be difficult and time consuming to grade
- Application of knowledge, skills, abilities
- Measures some skills and abilities not possible to measure in other ways.
- Can not be used in some fields of study.
- Difficult to construct.
- Difficult to grade unless well-defined rubrics are used.
- Time-consuming to give and take.