Skip to content
AP Psychology · Mental and Physical Health

Publication bias

Publication bias appears in 3 reviewed AP Psychology questions and sits in Mental and Physical Health. Below are worked examples showing exactly how the exam tests it, with the answer marked and every distractor explained.

Publication bias is the tendency of journals to publish studies with statistically significant results more often than studies that find no effect. Studies with null results often go unpublished, which is called the file drawer problem.

Why it matters

If only the positive studies get printed, the published research makes an effect look stronger and more reliable than it really is. This is one reason many findings fail when other labs test them again.

How the exam tests it

Questions often describe a study with p < .05 that fails when an independent lab repeats it. The stem may then ask you to name the replication crisis, which publication bias helps cause.

Don’t confuse it with

Students confuse publication bias with the replication crisis, but publication bias is a cause, the selective printing of positive results, while the replication crisis is the result, the discovery that many findings do not hold up.

Publication bias in real questions

Example 1Harder
A psychology study concludes that a new therapy is effective (p < .05). A replication attempt by an independent lab finds p = .48 using the same protocol. Which contemporary issue in psychology does this best illustrate?
  1. AThe replication crisis, reflecting widespread concerns that many psychological findings do not hold up when independently testedCorrect
  2. BPublication bias, because journals prefer publishing negative results that challenge prior findings
  3. CDemand characteristics in the original study, because participants guessed the purpose and inflated results
  4. DType II error in the replication study, because the independent lab failed to reject a false null hypothesis

Why A is right

The replication crisis refers to the widespread finding that a significant proportion of published psychological studies fail to replicate when attempted by independent researchers. This has prompted major reforms in research practices, including pre-registration, larger samples, and open data sharing.
Why the other 3 choices miss
B
Incorrect because publication bias refers to journals preferring positive (significant) results, which would cause the original significant finding to be published; publication bias would make replication failures less visible, not more.
C
Incorrect because demand characteristics are a concern in the original study's design, not the explanation for a broader pattern of replication failures across psychology.
D
Incorrect because a Type II error is possible, but the question asks what issue this best illustrates at a broader level; the pattern of failed replications across psychology defines the replication crisis.

Studied alongside

Other concepts from Mental and Physical Health.