Chapter 5 Summary

Brief Overview

This chapter deals with four major topics
  1. Errors in measurement, especially the distinction between random error and bias.
  2. Reliability and steps one can take to determine whether random error is leaking into the measure, where it is leaking in, and how to stop the leaks.
  3. Evaluating/making a case for a measure's construct validaty.
  4. Choosing/devising a valid treatment manipulation.

More Detailed Summary

Page 98 points out
  1. the value of operational definitions for allowing psychological science to be and
  2. the value of understanding operational definitions for
That is,

Pages 99-111 discuss error in measurement. We stress that

  • bias is different than random error. The referee example we use seems to help students make this distinction.
  • observers introduce both random error and bias.
  • there are ways to reduce observer errors (see Table 5-1, p. 105), including one of the most popular in all of psychology - virtually eliminating the observer.
  • standardizing procedures can reduce both random error and bias
  • blind procedures can reduce bias (The example of the professor grading exams seems to help students understand the value of standardization, blind techniques, and reducing the role of the observer).
  • psychological tests often use many items in an attempt to balance out the random error
  • participants may bias results by trying to make a good impression (social desirability bias) and by obeying demand characteristics.
  • there are ways (see Table 5-2, p. 108) of reducing the impact of these subject biases

    The concept map below summarizes first part of the section on measurement errors (the part in which we distinguish between random error and bias).

    The concept map below summarizes the next part (in which we discuss how to reduce observer errors).

    The concept map below summarizes the key points of the last part (in which we try to reduce measurement errors due to the participant).

    The next part of the chapter  (pp. 111-122) deals with reliability. The "key points table" (Table 5-4, p. 122) and the flow chart ( Figure 5-6, p. 123) summarize the main points in this section.

    Pages 124-131 deal with construct validity. Figure 5-7 (page 130) summarizes the main ideas quite well, as does Table 5-5 on p. 131. (If you want to go into more depth about construct validity, you could explore/assign the classic paper that introduces the term construct validity.)

    One interesting way to talk about the value of validity is to discuss some "witch tests," such as the one that if a person floats, rather than drowns, they are a witch. The basic logic of this test is lampooned in the movie "Monty Python and the Holy Grail" (the 4 min scene that starts 17:25 into the movie). For a shorter and more serious critique, you can use Virginia Governor Tim Kaine's statement (quoted on p. 21 of the July 24, 2006 Newsweek) about why he decided to pardon Grace Sherwood ( In 1706, Sherwood had been convcted of being a witch) : "With 300 years of hindsight, we all certainly can agree that trial by water is an injustice."

    Pages 131-136 provide tips for manipulating variables. As Table 5-6 (p. 132) illustrates, most of the principles behind measuring variables also can be applied to manipulating variables. Table 5-7 (p. 136) summarizes the rest of the chapter, pointing out that there are three basic kinds of manipulations (instructional, environmental, stooges) and there are pros and cons to each type.

    Back to Chapter 5 Menu

    Back to Research Design Explained Home Page