The pooled diamond in a forest plot is an invitation to ask a question: what, exactly, has been combined? A narrow diamond can describe an average precisely while leaving substantial uncertainty about effects across settings.
For a physiotherapy student, useful interpretation connects the statistics to the review’s populations, programmes, comparators, outcomes and time points. Counting how many studies have “significant” results does not accomplish that task.
The review and every result below are fictional. They illustrate reading a report, not evidence that a particular rehabilitation programme works.
Orient the plot before judging it
Start with the effect measure. A mean difference uses the outcome’s units and has a no-difference value of zero. A risk ratio has a no-difference value of one. Read the axis labels and comparison order; the side marked as beneficial depends on what the outcome means.
Markers show study estimates; horizontal lines usually show confidence intervals. Marker area often indicates statistical weight, not quality. The pooled diamond shows the summary and its interval. See Cochrane Handbook chapter 10.
If the outcome is a 0–100 ability score with higher scores better, and the difference is new minus comparator, positive values favour the new programme. Make that sentence explicit before interpreting a single row.
A fictional review with a positive average
Suppose an eight-study review reports the following rounded outputs from its specified random-effects analysis:
| Reported quantity | Fictional result |
|---|---|
| Average mean difference | +3 ability points |
| 95% confidence interval for the average | +1 to +5 points |
| 95% prediction interval for an underlying effect in a comparable setting | −4 to +10 points |
| I² estimate | 65% |
These are supplied teaching outputs; the individual-study dataset is not supplied and the analysis cannot be reconstructed from this table.
The confidence interval for the average lies above zero. Under the analysis assumptions, the review provides evidence of a positive average difference on this scale. It does not establish that three points is important to patients, that every programme works, or that an individual participant will improve by three points.
Now read the prediction interval. It includes negative, near-zero and positive underlying effects. Its purpose differs from the confidence interval for the mean: it addresses variation in underlying effects for a comparable new setting under the model, rather than uncertainty only about their average. Prediction intervals are discussed by IntHout and colleagues. Plea for routinely presenting prediction intervals in meta-analysis.
A coherent appraisal can therefore say both: “The estimated average favours the new programme” and “The model permits effects in comparable settings that do not favour it.” Neither statement cancels the other.
Read variation in context
Imagine that the review includes programmes with different supervision, dose and comparator intensity. Some are compared with a substantial existing service; others with minimal support. Even if each trial is conducted carefully, the contrast called “new programme versus comparator” may represent different practical questions.
Your next task is to compare those characteristics with the decision being considered. A Canadian outpatient placement may resemble only part of the evidence. That does not justify selecting whichever trial has the most attractive result. It justifies asking whether the pooled question is coherent and how directly the studies address this setting.
Likewise, a large study marker deserves attention because of its influence on the estimate, but size cannot answer whether the programme was delivered as intended or whether outcomes were selectively reported. A small marker is not a licence to ignore an informative boundary case.
What I² adds—and what it does not
I² expresses the proportion of variability in effect estimates attributed to heterogeneity rather than sampling error under its calculation. It is not the percentage of studies that disagree, the percentage of patients who respond differently, or the probability that the conclusion is wrong. The statistic was introduced to help interpret inconsistency across studies. Higgins et al.: Measuring inconsistency in meta-analyses.
In our case, 65% does not mean that 65% of the eight studies failed. Nor does it identify supervision, dose or comparator choice as the cause of variation. Those are hypotheses requiring investigation.
I² is relative to within-study uncertainty. Its value does not tell you the magnitude of variation in ability points. The between-study variance, often written τ², and the prediction interval help address different aspects of that question. With few studies, heterogeneity estimates are themselves uncertain; a value of zero does not prove all underlying effects are identical. Avoid using an I² cutoff as an automatic switch between models. Cochrane Handbook chapter 10.
A random-effects model does not repair a biased evidence base
Random-effects modelling allows related underlying effects to vary. It cannot correct selective publication, poor allocation procedures or inconsistent outcome definitions simply by producing a wider interval. Under some conditions it also gives smaller studies relatively more weight than a fixed-effect analysis. Cochrane Handbook chapter 10.
For the fictional review, suppose four unpublished studies all found less favourable results. Changing the model applied to the eight available studies would not retrieve those missing results. Or suppose one study accidentally entered a standard error as a standard deviation. An elaborate model would not make that input correct.
This is why a useful reading sequence examines the question and data before celebrating the pooled estimate. Look for transparent eligibility decisions, consistent outcome direction, appropriate handling of dependent estimates and a reasoned assessment of bias. Statistical precision is only one part of credibility.
Transfer exercise: choose the defensible conclusion
A second fictional review of the same higher-is-better scale reports an average difference of +2 points, a 95% confidence interval of +0.5 to +3.5, and a 95% prediction interval of −1 to +5. Which statement is best supported?
A. At least 95% of patients gain between −1 and +5 points.
B. Every comparable setting benefits because the confidence interval excludes zero.
C. The average favours the programme, while the prediction interval includes underlying effects that do not; applicability and model assumptions still matter.
Reasoned solution: C distinguishes the two targets. A turns an interval about setting-level underlying effects into an interval for individual outcomes. B replaces an average with a universal claim. Neither interval identifies individual benefit or establishes that the measured difference is worthwhile.
As a counterexample, imagine the prediction interval had been +1 to +3. That would strengthen the model-based case for consistently positive underlying effects in comparable settings. It still would not guarantee benefit in an unlike population or remove bias in the included studies. Prediction intervals also depend on assumptions and estimation methods, particularly when evidence is sparse.
Take a three-part note
Record the average and its uncertainty, the variation across studies, and the match to the question you need answered. In this example: “Average +3 points; confidence interval +1 to +5; prediction interval −4 to +10; programme and comparator differences need explanation.” That note is more useful than “meta-analysis positive.”
Sources and further reading
- Cochrane Handbook, chapter 10 — forest plots, weighting and heterogeneity; relevant full-text sections reviewed.
- Higgins et al.: Measuring inconsistency in meta-analyses, 2003 — I² and inconsistency; indexed publisher excerpts reviewed.
- IntHout et al.: Plea for routinely presenting prediction intervals in meta-analysis, 2016 — confidence versus prediction intervals; indexed publisher excerpts reviewed.