A researcher finds a statistically significant chi-square test for independence (p < 0.001) between gender and career preference in a sample of 10,000 adults. However, Cramer's V (a measure of effect size for categorical variables) is only 0.04. This result illustrates that
- Athe sample size of 10,000 is too large for a valid chi-square test of independence
- Bthe chi-square test must be incorrect because the reported effect size is so small
- Cwith large samples, trivial associations can still be statistically significantCorrect
- Dgender and career preference are strongly related because the p-value is below 0.001
Why C is right
With very large samples, chi-square tests have enormous power and can detect even tiny departures from independence. A Cramer's V of 0.04 indicates an extremely weak association that, while statistically significant (unlikely due to chance), has virtually no practical importance. This demonstrates the critical distinction between statistical significance and practical significance.
Why the other 3 choices miss
- A
- Incorrect. There is no upper limit on sample size for chi-square tests. The test is valid; the issue is interpreting a significant but practically meaningless result.
- B
- Incorrect. The test is not incorrect. The tiny effect truly exists in the sample; it is just too small to be meaningful. Statistical significance correctly identifies a non-zero association.
- D
- Incorrect. A very small p-value indicates that the association is unlikely to be due to chance, but it says nothing about the strength of the association. Cramer's V = 0.04 shows the association is very weak.