Skip to content
AP Statistics · Inference for Categorical Data: Proportions

Alpha

Alpha appears in 6 reviewed AP Statistics questions and sits in Inference for Categorical Data: Proportions. Below are worked examples showing exactly how the exam tests it, with the answer marked and every distractor explained.

How Alpha is tested

Example 1Medium
A significance level of α = 0.05 in hypothesis testing means:
  1. AThe probability the null hypothesis is true is 5%.
  2. BThere is a 5% chance of rejecting a null hypothesis that is true.Correct
  3. COnly 5% of experiments conducted at this level will produce correct results.
  4. DThe study has a 95% probability of detecting a real effect if one exists.

Why B is right

The significance level α is the probability of committing a Type I error — rejecting the null hypothesis when it is actually true. Setting α = 0.05 means the researcher accepts a 5% chance of a false positive (incorrectly concluding there is an effect when there is none).
Why the other 3 choices miss
A
The significance level is not a statement about the probability that H0 is true. That would require prior probabilities and Bayesian reasoning. Alpha is the long-run false positive rate under the frequentist framework.
C
The significance level has nothing to do with the proportion of experiments producing correct results. It specifically controls the false positive rate.
D
The probability of detecting a real effect is statistical power (1 - β), not α. Power depends on α, the effect size, and the sample size, but α alone does not determine power.
Example 2Harder
A researcher conducts 20 separate hypothesis tests at α = 0.05, all testing true null hypotheses (no real effects exist). On average, how many of these tests will produce statistically significant results?
  1. A0
  2. B1Correct
  3. C10
  4. D20

Why B is right

With α = 0.05 and all null hypotheses true, each test has a 5% probability of producing a false positive (Type I error). Over 20 tests, the expected number of false positives = 20 × 0.05 = 1. This illustrates the problem of multiple comparisons: running many tests inflates the overall chance of at least one false positive.
Why the other 3 choices miss
A
With a 5% error rate per test, we expect some false positives. Expecting zero false positives over 20 tests ignores the Type I error rate.
C
10 would correspond to a 50% Type I error rate per test, far above α = 0.05.
D
If all 20 tests produced significant results despite true nulls, that would mean the Type I error rate is 100%, not 5%. This contradicts the definition of α.
Example 3Harder
A researcher decreases the significance level from α = 0.05 to α = 0.01 (making it harder to reject H0). What effect does this have on the probability of Type I and Type II errors?
  1. AType I error probability decreases; Type II error probability decreases.
  2. BType I error probability increases; Type II error probability decreases.
  3. CBoth error probabilities remain unchanged.
  4. DType I error probability decreases; Type II error probability increases.Correct

Why D is right

Decreasing α makes it harder to reject H0, which reduces the probability of a Type I error (false positive). However, by setting a more stringent threshold, the test also becomes less likely to reject H0 even when it is false — increasing the probability of a Type II error (false negative). There is an inherent tradeoff between Type I and Type II error rates.
Why the other 3 choices miss
A
Decreasing α does lower Type I error probability, but the Type II error probability INCREASES (not decreases) because the test becomes less sensitive.
B
Decreasing α (a smaller significance level) makes it HARDER to reject H0, which REDUCES Type I error probability, not increases it.
C
Both error probabilities change when α changes. They are inversely related: reducing Type I error by lowering α increases Type II error probability.

Studied alongside

Other concepts from Inference for Categorical Data: Proportions.