| FIRST TIME THROUGH | SECOND TIME THROUGH | |
|---|---|---|
| INTRODUCTION | Why do we care about this area of research? What references do I need to read? What are hypotheses? Why do the authors expect their hypotheses to be supported? What is the basis for their predictions? How does the study fit in with existing work? If it cures a weakness in previous research, what was wrong with previous research? If it extends or fills a gap in previous research, what is that gap? What variables are they looking at? |
Do I agree with their arguments? Does the hypothesis really follow from theory or previous research? |
| METHOD | Who were the subjects? How were they obtained? What was done to the subjects? What was the independent variable? How were the subjects treated? What did subjects do? What was the dependent variable? What was the design? |
Are there reasons to expect that results from these subjects are atypical? Might they have gotten different results with a different subject population? Were groups equivalent before the study began? Were there enough subjects? Was there a mortality problem? Were their responses independent? Are there any other variables that should have been controlled? If you were a subject, would you have guessed the hypothesis? Would you have taken the task seriously? Do they have adequate control groups? Is there any confounding? Would it have been better to use different operational definitions? Did the measure have adequate reliability? Should they have used more or different levels of the treatment variable? Was this the best design for the problem or would you have used a different one? Did they consider the design's sensitivity and the potential for order effects? In real life, does the variable occur within subjects or between subjects? Was it between or within in this study? Could there have been any order effects? |
| RESULTS | How are they deriving the scores that they put into the analysis? That is, how are they scoring subjects' responses? How do they make numbers out of the participants' behavior? What are the average scores for the different groups? |
Were there manipulation checks? Do the statistics directly test the predictions made in the introduction? Do the statistical tests match up with the verbal descriptions? That is, if the authors say that Group 1 scored better than Group 2, do they have an analysis that directly compares Group 1 against Group 2? Are the statistics appropriate for the dependent variable's scale of measurement? Did they do the appropriate post hoc tests? Did they properly interpret any null results? Did they cautiously interpret any ordinal interactions? |
| DISCUSSION | Do they think the results matched their predictions? How do they explain any discrepancies? What additional studies do they recommend? |
What questions do I have? Are there other explanations for the results, such as hypothesis guessing or competing theories? Are there additional studies I would recommend? Did the authors make cause-effect statements on the basis of correlational evidence? Did the authors state something that was not supported by the results? For example, do they treat a nonsignificant or talk about a comparison that was not made? Do they misinterpret null results? |
Note: For an even more comprehensive table, see the Instructor's Manual.
Back to Chapter Main Menu