Problem-Solving and Data Analysis · Math
Evaluating statistical claims
Evaluating study design and generalisability: which population a sample speaks for, and which designs support a cause-and-effect claim.
9 questions in the bank · 3 easier · 3 medium · 3 harder · this domain is ≈15% of the section
What to watch for
Only random assignment supports a causal conclusion; random selection supports generalising to the population sampled. A larger sample fixes neither — it measures the same bias more precisely.
How the question is worded
These are the actual stems used in the bank — the SAT reuses a small set of wordings, and recognising them saves time on test day.
- A survey was given to 120 randomly selected students at Ridgeview High School. The results of this survey can best be generalized to which population?
- A survey was given to 200 randomly chosen visitors to one public library branch. The results of this survey can best be generalized to which population?
- A survey was given to 85 randomly selected members of a community garden. The results of this survey can best be generalized to which population?
- A researcher wants to determine whether a new stretching routine reduces soreness after workouts. Which study design would best allow a cause-and-effect conclusion?
Worked examples
easyA survey was given to 120 randomly selected students at Ridgeview High School.…
A survey was given to 120 randomly selected students at Ridgeview High School. The results of this survey can best be generalized to which population?
- A. All high school students in the state
- B. The 120 students who were surveyed and no one else
- C. All teenagers in the country
- D. All students at ridgeview high school
Answer: D. All students at ridgeview high school
Explanation
Random sampling supports conclusions about the population the sample was drawn FROM — no further. The survey sampled 120 randomly selected students at Ridgeview High School, so its results generalize to all students at Ridgeview High School. Extending to larger groups (the state, the country) assumes those groups resemble the sampled one, which the survey cannot establish.
mediumA researcher wants survey results that generalize to all students at a high sc…
A researcher wants survey results that generalize to all students at a high school. Which sampling method best supports that goal?
- A. Select the sample at random from all students at a high school.
- B. Let people volunteer to respond online.
- C. Survey students leaving one club meeting, since they are easy to reach.
- D. Survey a very large number of whoever is nearby.
Answer: A. Select the sample at random from all students at a high school.
Explanation
Results generalize to the population that was randomly sampled — random selection gives every member a chance of inclusion, so the sample resembles the population. Convenience and volunteer samples over-represent whoever was easiest to reach or most motivated, and no sample size repairs that.
hardA researcher wants to determine whether a new stretching routine reduces soren…
A researcher wants to determine whether a new stretching routine reduces soreness after workouts. Which study design would best allow a cause-and-effect conclusion?
- A. Compare people who already stretch regularly with people who do not.
- B. Survey adults at one gym who volunteered to try the routine about whether they noticed a difference.
- C. Randomly assign volunteers to follow the routine or not, then compare their reported soreness.
- D. Recruit a much larger group of people who use the routine and measure their soreness precisely.
Answer: C. Randomly assign volunteers to follow the routine or not, then compare their reported soreness.
Explanation
Cause-and-effect conclusions require random ASSIGNMENT to treatment and control groups — randomization balances all other differences between the groups. Comparing people who chose for themselves (or surveying volunteers) leaves those differences in place, and a bigger sample only measures the biased comparison more precisely.