A group enters a rehabilitation study after unusually difficult assessments. Its average score improves at follow-up. The improvement is real as a description of the recorded scores. What remains unsettled is why it happened.
This lesson teaches one specific explanation: selecting people when their measurements are extreme can create an apparent improvement even when their underlying usual level does not change. Your task is to identify that selection mechanism, then distinguish the observed change from the effect attributable to an intervention.
Start with the selection rule
Regression to the mean concerns repeated measurements that vary imperfectly together: an extreme first value tends, on average, to be followed by a less extreme value. Selection on an unusually high or low baseline makes this especially relevant. It does not mean every individual must move toward an average. Barnett and colleagues, abstract
Before looking at a follow-up graph, read the eligibility criterion. Were participants selected because they had a particularly poor score on one occasion? Was the same score then used to demonstrate improvement? That combination creates a reason to examine regression to the mean, even if the measurement procedure was careful.
Temporary variation need not be an instrument malfunction. A score can reflect usual ability plus occasion-specific circumstances. An unusually difficult day can be followed by a more ordinary day without either observation being dishonest.
Worked example: improvement with no intervention
Original fictional teaching study. Ninety learners complete a simulated task scored from 0 to 100, with higher scores indicating greater difficulty. They receive no intervention between measurements.
For this deliberately simple model, everyone has an unchanged usual score of 50. Each assessment adds a temporary deviation of −10, 0 or +10, with equal probabilities. Deviations on the two occasions are independent. The following constructed dataset is exactly balanced across the nine possible combinations; actual random samples need not be this balanced.
| Baseline score | Follow-up 40 | Follow-up 50 | Follow-up 60 | People in row | Follow-up mean |
|---|---|---|---|---|---|
| 40 | 10 | 10 | 10 | 30 | 50 |
| 50 | 10 | 10 | 10 | 30 | 50 |
| 60 | 10 | 10 | 10 | 30 | 50 |
The full group has a mean of 50 on both occasions. Now apply a study's hypothetical entry rule: include only people with a baseline score of at least 60.
That selects the bottom row: 30 people with a baseline mean of 60. Their follow-up mean is:
(10 × 40 + 10 × 50 + 10 × 60) ÷ 30 = 50.
Their mean change is 50 − 60 = −10 points. Twenty improve and ten remain unchanged. A report could accurately describe a ten-point average improvement while incorrectly attributing it to an intervention delivered after selection. In our constructed example, no intervention occurred and nobody's usual score changed.
The error lies in the causal explanation, not in subtraction. Baseline selection captured only positive temporary deviations. The next occasion allowed all three deviations again.
Selecting the baseline-40 row gives the reverse pattern: its mean rises to 50. Regression to the mean is therefore not inherently beneficial. Whether the observed movement looks better or worse depends on the score's direction and which extreme was selected.
What the example assumes—and what it does not prove
We know the unchanged usual score because we invented the data-generating process. A real study cannot infer each person's stable underlying level from this table.
The equal usual scores and independent temporary deviations make the mechanism visible. Real participants differ in usual ability, and their occasion-specific circumstances may persist. A very high measurement might reflect both a genuinely high usual level and temporary variation; the expected return need not be all the way to the overall population mean.
As a boundary case, suppose each person's follow-up score always equals their baseline score exactly. Selecting extreme baseline scores then produces no regression through the mechanism illustrated here. Conversely, genuine recovery or an effective intervention can coexist with regression to the mean. Recognizing the possibility does not establish that every improvement is an artefact.
This is why “the group improved” and “the intervention caused the improvement” should remain separate statements. Neither automatic attribution to treatment nor automatic dismissal as regression is justified by a single before-and-after contrast.
What a concurrent randomized comparison adds
Consider a separate fictional trial, not an additional analysis of the 90-person table. All participants meet the same entry rule. They are then properly randomized to Programme A or comparison Programme B. Follow-up is complete, measurement is comparable, and both groups start at a mean of 60.
At follow-up, A averages 47 and B averages 51:
- A changes by 47 − 60 = −13 points.
- B changes by 51 − 60 = −9 points.
- The difference in changes is −13 − (−9) = −4 points.
Under the stated trial assumptions, the between-group comparison estimates the added effect of assignment to A rather than B. The thirteen-point improvement in A includes whatever would also have happened under B; it is not itself that added effect.
Randomizing people selected by the same criteria allows regression and other shared time-related influences to operate in both groups in expectation. It does not guarantee identical influences in a particular small sample. Repeated baseline measurements can also reduce dependence on a single extreme occasion, but averaging alone does not supply an untreated counterfactual. Morton and Torgerson
The four-point estimate still needs an uncertainty interval and an interpretation of practical importance. B may itself be an active programme. The result answers A versus B under these conditions, not A versus nothing or A for every possible learner.
Three appraisal errors to catch
Treating eligibility as a neutral administrative detail. Selection based on an unusually poor value can shape the expected follow-up pattern. Record the exact eligibility measure, timing and threshold.
Calling every non-treatment influence regression to the mean. Familiarity with the task, secular change, natural recovery and changed measurement procedures are different explanations. They can operate alongside regression but should not be collapsed into one label.
Choosing the baseline retrospectively. Taking the worst of several available pre-intervention scores and comparing it with an ordinary follow-up strengthens the selection problem. A defensible measurement schedule should be specified before inspecting which contrast looks largest.
Transfer exercise: a score that runs the other way
Original fictional exercise. A study selects participants with low performance scores; higher scores are better. Everyone starts at 20. Participants are randomized to A or B under the same conditions used above. Mean follow-up scores are 31 for A and 27 for B.
Which statement best matches these data?
A. Assignment to A caused an eleven-point benefit over B.
B. The estimated added benefit of assignment to A over B is four points; A's eleven-point within-group gain has a broader set of possible causes.
C. Regression to the mean proves that A has no effect.
Feedback: B is best. A gains 31 − 20 = 11, B gains 27 − 20 = 7, and the contrast is 11 − 7 = 4 points. A confuses improvement with a comparative effect. C turns a plausible mechanism into an unsupported explanation of the entire change. These point estimates alone establish neither precision nor practical importance.
A useful next reading habit
For the next before-and-after study you read, write three sentences: how participants were selected; what changed in the measured outcome; and what comparison supports the proposed causal explanation. If the third sentence is missing, report the improvement descriptively and keep its cause open.
Sources and further reading
- Barnett, van der Pols and Dobson: Regression to the mean—what it is and how to deal with it. Abstract consulted; full publisher text was not accessible.
- Morton and Torgerson: Effect of regression to the mean on decision making in health care. Relevant full-text statistical and design sections consulted.