Chapter 2: Professor-Oriented Summary

Understanding Types of Validity

If you stress that internal validity involves the ability to make cause-effect statements, many students will grasp the concept immediately. However, at least two obstacles cause some students have trouble understanding internal validity. First, many students do not make distinctions between descriptive statements and causal statements. To give students practice on that distinction, you might go over this chart and quiz. Second, some students get confused about the difference between external validity and internal validity because individual differences can contribute to both internal and external validity problems. For example, because of individual differences, 

    a. the treatment group may differ from the no-treatment group before the treatment is introduced (leading to internal validity problems), and

    b. even if the treatment causes differences in the particular group tested, these effects may not generalize to individuals with different characteristics (leading to external validity problems).

To help students distinguish between external and internal validity--while interjecting some humor-- go to the Twinkies project. Stress that

    1. internal validity is "established" by assuming that, before the test,  the control and experimental Twinkies are identical (otherwise, there would be no need for the control Twinkie),

    2. external validity is "established" by asserting that both Twinkies "appeared within the range of what would be described as a 'normal' Twinkie."

(You could also mention that, although the researchers are confident of the internal validity of their results--the water they used destroys Twinkies--the researchers express some doubts about the external validity of their results--because there may be something strange about the tap water on their campus. So, like good scientists, they encourage replication--a great method for establishing external validity.)

Students  tend to have the most difficulty understanding construct validity.

One reason students have trouble with construct validity is that they don't really understand what constructs are. To help them understand constructs (and thus construct validity), emphasize that:

• "Constructs are constructed" (Levy,1997)
• Constructs can't be directly observed. They require us to "look inside the black box." Unlike cartoon characters, humans don't produce thought bubbles that allow us to see what they are thinking.
  
  
• Because people may mask their feelings, what we see may not reflect what people are really thinking.
  
  

Students may confuse construct validity with external validity. To help them, tell them that external validity often refers to P(eople) and P(laces), whereas construct validity tends to focus on M(anipulations) and M(easures).

Critiquing Study's Validity

Students are pretty good at finding flaws with the external validity of a study (although they are fooled by large, non-random samples). In fact, they may tend to be overly critical of the external validity of research. Just because each individual is unique, that doesn't mean we can't study individuals scientifically and find general rules. Science forms general laws about things that are unique all the time. Consider the following two examples:

Similarly, lab studies often do generalize to the real world.

Whereas students are overly skeptical of the external validity of research, they are too trusting of cause-effect claims. To motivate them to question these claims, you may want to stress (1) the practical importance of knowing the cause and (2) the embarrassment that can result from being fooled about causes.

To stress the practical importance of knowing the cause, you might

1. Talk about people wrongly accusing parents of neglecting their autistic child because of the finding that parents of an autistic children held that child less than the normal child was held.
2. Talk about the millions of dollars and tons of frustration that were caused by jumping to the conclusion that the reason poor readers eyes moved differently across the page than good readers was because something was wrong with their eyes. As a result, schools purchased machines to train poor readers' eyes.
3. Discuss facilitated communication.
4. Talk about the problems resulting from jumping to the conclusion that high self-esteem is the cause of all kinds of good outcomes and that low self-esteem is the cause of all kinds of bad outcomes.
5. Talk about all the unfortunate students who used to pay to have their papers typed, thinking that typing it would result in a better grade when a controlled study showed that teachers graded typed papers harder than hand-written ones.
6. Recent research suggests that children's delinquent behavior is not a an effect of divorce. Divorced parents of delinquents may be feeling guilt ("I should have stuck it out for the sake of Johnny") that they shouldn't.
7. Our neighbors came to us in tears. They had returned from their child's 8-month check-up. The physician noted that the child seemed to have allergic symptoms. He then asked them whether they had a dog. When they said they did, the physician ordered them to get rid of their dog. They were quite upset because they considered the dog to be "a member of the family."
8. Just talk about the fact that if you don't know what is causing a problem, it's hard to fix it.

1. A person reportedly asked a reference librarian the following question: "Why were so many Civil War battles fought on national parks?"
2. The funny case against bread

Back to Research Design Explained Home Page