On this page · 5 sections
Underneath the wording, every experimental-design question asks the same thing: what can this study legitimately prove, and which feature decides that? The exam hands you a described study and then asks you to name its method, name its variables, name the flaw, or judge the conclusion someone drew from it. You are not being asked whether the finding sounds true. You are being asked whether the design earns the claim.
These questions are scored in two places. In Section I they are one point each out of 75, no partial credit and no guessing penalty, at about 72 seconds per question. Some are pure method questions. Many more hide inside content questions, where the stem describes Bandura's films or a twin study and asks what conclusion it supports. In Section II, the Article Analysis Question runs almost entirely on this skill. Its lettered parts ask you to identify the research method, state an operational definition, describe what a named statistic indicates in the context of the study, name an ethical guideline the researchers followed, judge how far the findings generalize, and explain whether a finding supports or refutes a claim. The Article Analysis Question is worth 7 points across parts A through F. Parts A through E carry one point each. Part F carries two, and you need both halves to collect both. Name a specific result and interpret it accurately, then explain how it supports or refutes the claim. A result with no explanation earns 1. An explanation with no result earns 1. The other Section II item, the Evidence-Based Question, is also worth 7 points, and it runs on this same design reasoning. Section II is 14 raw points in 70 minutes. Budget about 25 minutes for the Article Analysis Question and about 45 for the Evidence-Based Question, with the first 10 and 15 of those spent reading.
The scoring logic differs between the sections and you should exploit that. Multiple choice rewards discrimination: four choices are all real psychology terms and only one fits this study. The free response rewards naming: a reader scans for the concept word and the link, so a sentence that gestures at the right idea without naming it earns zero, while a plain sentence that names the method and points to the specific feature earns the point.
The steps
- 1
Read the verb before you read the study
Stems in this family ask four distinct jobs, and they look similar until you notice the verb. "Which design would provide the strongest evidence that X causes Y" wants you to pick the design with random assignment. "Which limitation is most inherent to this design" wants the flaw that comes from the structure itself, not any flaw you can imagine. "Which flaw undermines this conclusion" wants the gap between what was measured and what was claimed. "The researcher is measuring which variable" wants a label. Decide which job you have first, then read the scenario looking for that one thing. Reading the scenario first wastes your 72 seconds because you re-read it once you learn what was wanted.
- 2
Strip the study to five slots
Write or think five labels, fast. Method: experiment, correlational study, naturalistic observation, case study, survey, longitudinal, cross-sectional, or meta-analysis. Independent variable: the thing the researcher changed, and its levels. Dependent variable: the thing measured, and how it was measured. Assignment: were participants randomly assigned to conditions, or did they arrive in groups already formed? Sample: who they were and how they were recruited. Most wrong answers die at this stage. If nothing was manipulated, no answer choice containing the word "causes" can be right, and you can eliminate on that basis without understanding the topic at all.
- 3
Run the causation test on assignment, not on effort
Only one feature licenses a causal claim: random assignment of participants to conditions. It works by spreading pre-existing differences — motivation, trait anxiety, prior skill, family income — evenly across groups on average. It does not erase them from any one study. What survives is the treatment plus chance variation. That is why results arrive with a p-value. The p-value goes after the chance explanation. It asks how often a difference this large or larger would show up if the treatment did nothing at all. It does not tell you what fraction of your result came from chance. Nothing else substitutes for random assignment. A five-year longitudinal survey is still correlational because participants chose their own exposure. A meta-analysis of correlations is still correlational, pooled. A study that compares naturally formed groups — adults malnourished as infants versus adults who were not — has no random assignment at all. Poverty, schooling, and healthcare stay alive as confounding variables, and no statistic removes them after the fact. Effort, size, and duration do not buy causation. Assignment does.
- 4
Sort the flaw: inside the study or outside it
Internal validity asks whether the independent variable actually caused the change in the dependent variable inside this study. It is threatened by confounding variables, self-selection into conditions, experimenter bias, demand characteristics, and the absence of a control group. External validity asks whether the result extends beyond this room. It is threatened by a narrow sample, an artificial task, and volunteer bias. The exam separates these deliberately. Drawing participants from one university limits generalizability, so it is external. Letting participants choose whether to receive the therapy creates a motivation confound, so it is internal. When a stem says "internally invalid," every choice about the sample or the setting is a trap.
- 5
Split the look-alike terms before you commit
Four families produce most missed points. Expectancy family: the placebo effect is the participant improving because they believe they got treatment; demand characteristics are the participant guessing the hypothesis and adjusting behavior to match; experimenter bias is the researcher's expectation shaping how data are delivered or scored. A double-blind design blocks the placebo effect and experimenter bias at once. Participants do not know which condition they are in, so expectancy is held equal across groups. The researchers running or scoring the study do not know either, so their expectations cannot bend the results. Demand characteristics are reduced but not eliminated, because a blinded participant can still work out the hypothesis and play along. Next, keep two different questions apart. Some biases are about who ends up in the data. Volunteer bias is about who agreed to take part, and it threatens generalizability. Attrition is about who dropped out partway through, and when one group drops out more than the other it threatens internal validity. Social desirability bias is a different animal. It is a response bias: participants under-report answers that make them look bad and over-report answers that make them look good, so the measure is distorted rather than the sample. Watch the term self-selection, which the exam uses in two opposite places. Self-selection into the study, meaning who volunteered, is a sampling problem and threatens external validity. Self-selection into a condition, meaning who chose the treatment, is a confound and threatens internal validity. Randomness family: random sampling is drawing participants from a population and buys generalizability; random assignment sorts them into conditions and buys causal inference. Time family: cross-sectional compares different ages at once and confounds age with cohort; longitudinal follows one cohort and suffers attrition. You may see cross-sequential, which combines the two, but treat it as background vocabulary rather than a required term.
- 6
Handle any statistics sentence as its own question
When a stem hands you a p-value or an effect size, treat that clause separately from the design. A p-value of .01 means that if the treatment truly did nothing, a difference this large or larger would turn up about one time in a hundred. It is not the probability that the treatment fails, and it says nothing about how big the effect is. Effect size answers the size question. Cohen's d of about 0.2 is conventionally small, 0.5 medium, 0.8 large, so d = 0.08 with a big enough sample can be statistically significant and practically worthless. A correlation coefficient near +0.78 tells you the relationship is strong and positive, and still tells you nothing about direction of causation.
- 7
Answer free-response parts in the rubric's own verbs
On the Article Analysis Question, write one or two sentences per lettered part and label them A through F to match. Answer only what that part asks. If asked to identify the method, name it flatly: "This was an experiment, because participants were randomly assigned to conditions." If asked for an operational definition, give the measurement procedure, not the concept: "Anxiety was operationally defined as the score on a 20-item self-report scale." If asked about generalizability, take a position and cite a specific sample fact from the passage. If asked whether a finding supports a claim, state supports or refutes, then point to the specific result that decides it. Hedging costs points; readers award them for named moves.
- 8
Use the same reasoning on the Evidence-Based Question
The other Section II item, the Evidence-Based Question, is worth up to 7 points. It hands you summaries of studies as sources and asks you to take a position on a psychological question. Its parts are lettered A, B(i), B(ii), C(i), and C(ii). Part A is your claim, worth 1 point. Part B(i) is specific evidence from one source, worth 1 point. Part B(ii) is explanation and application, worth 2 points. Part C(i) is specific evidence from a different source, worth 1 point. Part C(ii) is explanation and application again, worth 2 points. Label your booklet with those exact letters.
Same skill, different packaging. To use a source you must know what its design licenses. A correlational source cannot be cited as proof that one variable caused another, and a result from one narrow sample cannot be cited as proof about everyone. Write the claim as one flat sentence that picks a side. For each source you use, name the specific finding, then add a separate sentence explaining the link.
The 2-point rows are a ladder, not two loose halves. The first point goes to interpreting the evidence correctly and explaining how it relates to your claim. The second point sits on top of that: you also apply a named psychological perspective, theory, concept, or research finding. You cannot bank the application point without the relationship point underneath it. The concept you apply must be one the AP Psychology Course and Exam Description (2024) actually names. A concept you invented, or one borrowed from outside the course, does not score.
Evidence with no explanation still earns its own point, since parts B(i) and C(i) pay 1 point each for correctly cited, specific evidence. What it forfeits is the 2 points in part B(ii) or C(ii), the explanation-and-application rows. The reverse is also true, and it is useful under time pressure. If your citation goes missing or is too vague to score B(i) or C(i), you can still earn the points in B(ii) or C(ii). So write the reasoning even when you doubt the citation lands.
Two constraints are printed in the prompt and enforced by the rubric. Part C(i) must take its evidence from a different source than part B(i). Part C(ii) must apply a different perspective, theory, concept, or research finding than the one you used in B(ii). The penalties are not the same. Reuse the source and you lose the whole 1 point in C(i). Reuse the concept and C(ii) drops from 2 points to 1, because you keep the point for explaining how the evidence relates to your claim and lose only the application point.
Worked through
Here is a study built to look like an exam stimulus.
Dr. Reyes wants to know whether a ten-minute guided breathing recording reduces test anxiety. She recruits 120 volunteers from the junior class at one high school. Before assigning anyone, she obtains approval from an institutional review board, written parental consent, and each student's own assent. She randomly assigns 60 to hear the breathing recording and 60 to hear a ten-minute recording of a neutral news article read aloud. Everyone then takes a practice math test, after which anxiety is measured with a 20-item self-report scale scored from 20 to 100. Individual scores are kept confidential and reported only as group averages. The breathing group averages 48; the article group averages 57. She reports p = .01 and d = 0.48.
Five slots first. Method: experiment. Independent variable: recording type, two levels, breathing versus neutral article. Dependent variable: score on the 20-item anxiety scale. Assignment: random. Sample: 120 volunteers, one grade, one school.
Why the second group matters. The article condition is an active control, not an empty one. Everyone sits for ten minutes, everyone hears a voice, everyone knows a researcher is doing something to them. Holding those constant means the only thing left differing between groups is the breathing content. If the control group had simply gone straight to the test, an improvement could be explained by ten quiet minutes rather than by the technique.
What the numbers license. The breathing group scored nine points lower. p = .01 says a gap this large or larger would appear about once in a hundred runs if the recording did nothing at all. d = 0.48 sits just under the conventional medium mark, so the gap is a moderate one and not a trivial effect propped up by a huge sample. Both clauses are needed; either alone is incomplete.
Internal validity. Random assignment means trait anxiety, sleep, and math ability are spread evenly across groups on average, so the recording is the leading explanation for the difference. One threat survives: participants can tell which recording they got, and the breathing group may guess the hypothesis and report feeling calmer to be helpful. That is demand characteristics. The fix is a cover story — present both recordings as candidate relaxation methods, so no participant can tell which one is the treatment. A separate fix targets experimenter bias instead. Have a research assistant who does not know which recording is the treatment play the recordings and hand out the scale. Then no difference in the researcher's tone or manner can leak into one group. Blind scoring would not help here, because a 20-item scale is simply summed and leaves a scorer nothing to bend. Do not swap the two fixes.
External validity. The sample is volunteers from one grade at one school, and the outcome is a low-stakes practice test with no consequences. So the finding does not yet support a claim about adults, about other schools, or about performance on a real graded exam where the stakes change the anxiety being measured.
Ethics. The ethics part of the Article Analysis Question asks for a guideline the researchers actually applied, so you may only name a safeguard the article reports. This article reports four: review board approval, parental consent, each student's own assent, and confidentiality of individual scores. Consent from a parent plus assent from the student is the standard pairing when participants are minors. A full ethics list also includes protection from harm, the right to withdraw at any time, and a debriefing, and debriefing is routine practice rather than something only deception triggers. But if the source does not report those, you cannot claim them. Inventing an ethical safeguard scores zero on this part.
Now the multiple-choice version. If the stem asks which limitation most restricts the conclusion that breathing lowers test anxiety in high school students generally, the answer is the volunteer sample from a single school — external validity. If the stem instead asks what threatens internal validity, the sample choice is a trap and demand characteristics is the answer. Same study, different verb, different point.
The free-response version, in rubric sentences. Method: an experiment, because participants were randomly assigned to two conditions. Operational definition of the dependent variable: the score on the 20-item self-report anxiety scale, range 20 to 100. Statistic: p = .01 means that if the recording had no real effect, a difference at least this large would show up in about 1 study in 100 like this one. Ethical guideline: the researchers obtained parental consent and each student's own assent before collecting any data, which is the consent standard for minors. Generalizability: limited, because all participants volunteered from one grade at one school and the task was a practice test rather than a graded exam. Support or refute: the finding supports the hypothesis. The breathing group averaged 48 on the anxiety scale and the neutral-article group averaged 57, a nine-point difference with p = .01, so the students who heard guided breathing reported lower test anxiety than the students who heard neutral content.
Common mistakes
Treating a long or expensive correlational study as if it proved causation. A five-year longitudinal survey of gaming and aggression still lets participants choose their own exposure, so reverse causation and third variables both stay alive. Duration and sample size never substitute for random assignment.
Answering an internal-validity question with an external-validity flaw. When a stem says a critic calls the study internally invalid, "the sample came from one university" is the designed trap. The correct answer names something that could have caused the group difference itself, such as participants self-selecting into the treatment condition because they were more motivated to improve.
Swapping the three expectancy terms. Improvement because you believe you got the real pill is the placebo effect. Changing your answers because you worked out that the study is about conformity is demand characteristics. A researcher scoring the treatment group more generously because she knows who they are is experimenter bias. A double-blind design controls the first and the third at once, so in a double-blind trial neither the placebo effect nor experimenter bias can be the explanation.
Calling a low survey response rate attrition bias. Attrition means participants dropped out of a study that followed them over time. When 200 of 1,000 people return a survey and those 200 are heavy users of the thing being surveyed, that is volunteer or self-selection bias, and it inflates the apparent prevalence in the population.
Misreading p. A p-value of .03 does not mean a 3 percent chance the program fails, and it does not mean the effect is worth having. It means results this extreme or more extreme would occur 3 percent of the time if the null hypothesis were true. Practical significance is a separate judgment made from effect size.
Blurring random sampling and random assignment. Randomly selecting 500 names from a national registry buys representativeness and generalizability. Randomly sorting those 500 into treatment and control buys causal inference. A study can have one without the other, and the exam writes choices that offer the wrong one.
Writing an Evidence-Based Question answer that stacks quotations without explanation. The citation itself does earn a point, because parts B(i) and C(i) pay 1 point each for specific cited evidence. The explanation is scored separately, in parts B(ii) and C(ii), worth 2 points each. Those 2 points are a ladder. The first goes to interpreting the finding and saying how it supports your claim. The second comes only on top of that, for applying a named psychological perspective, theory, concept, or research finding from the course. Stacking quotations therefore cashes in one third of what each source is worth.
Checklist
- Can you look at a described study and say, in ten seconds, its method, independent variable, dependent variable, assignment procedure, and sample?
- Can you state why random assignment licenses a causal claim while random sampling does not, and give an example of a study that has one but not the other?
- Given a flaw, can you say instantly whether it threatens internal or external validity, and defend the call?
- Can you distinguish the placebo effect, demand characteristics, and experimenter bias using three different one-sentence scenarios, and say which two of them a double-blind design controls?
- Can you explain what a p-value of .01 does and does not mean, and say why a result with d = 0.08 can be significant and still not matter?
- Given a study that compares naturally formed groups with no random assignment, can you name the confounding variables that survive and explain why statistical analysis does not repair the design?
- Can you write an operational definition that gives the measurement procedure rather than restating the concept?
- Can you name the ethical requirements a researcher must satisfy — informed consent, IRB approval, protection from harm, the right to withdraw, confidentiality, and debriefing — and match each to a feature of a described study? Debriefing is standard practice at the end of a study, not something triggered only by deception. With minors you need parental consent plus the participants' own assent.
- For the Evidence-Based Question, can you name what each part scores — A the claim, B(i) and C(i) the evidence, B(ii) and C(ii) the explanation and application — and write a one-sentence defensible claim with a separate explanation sentence attached to each piece of source evidence?