Two methods can rank people almost perfectly while giving substantially different numbers for the same person. A measurement comparison must therefore ask how far apart the paired readings are, not only whether high readings from one method accompany high readings from the other.
Your learning task is to interpret a difference-versus-mean plot, distinguish average bias from limits of agreement, and compare those limits with a stated purpose. This is a different question from interpreting change over time or choosing the construct an outcome measure should capture.
See why a perfect correlation is insufficient
Original fictional counterexample: Method A gives readings of 20, 40, 60 and 80 units. Method B gives 30, 50, 70 and 90 units for the same four people. Pearson's correlation is exactly 1, because B is always A plus 10.
Yet substituting B for A changes every recorded value by ten units. If a study's prespecified acceptable difference were only three units, perfect correlation would not rescue the comparison.
Correlation describes association; agreement concerns the closeness of paired measurements. Bland and Altman's original paper explains why correlation alone cannot establish agreement. Bland and Altman, 1986.
A related boundary runs the other way: sampling people with a narrow range of underlying values can produce a modest correlation even when the absolute differences are small for the intended use. Do not use a universal correlation cutoff as permission to exchange methods.
Define the replacement decision before calculating
Original fictional worked case: A Canadian rehabilitation research team compares two devices measuring the same standardized joint angle in degrees. Method A is the established procedure; method B is more portable. Neither is assumed to reveal the error-free true angle.
Forty independent participants each provide one reading from each method under closely matched conditions. Positioning and instructions are standardized, measurement order is randomized, and the protocol aims to avoid meaningful change between readings. These are supplied case assumptions, not observations from a real study.
Before examining results, the team defines differences from −5° to +5° as acceptable for its particular research use. This invented range is not a clinical threshold for joint measurement. Its justification would need to reflect the actual decisions and consequences in a real project. An acceptable difference should be specified independently of the observed results. Bland and Altman, 1986.
The comparison concerns single readings using these protocols. A study of three-reading averages would answer a different replacement question.
Keep the sign and the pair intact
Define each difference as d = B − A and each pair's mean as (A + B) ÷ 2. Plot the difference vertically against the paired mean horizontally.
For an illustrative pair with A = 42° and B = 47°, the plotted point is (44.5°, +5°). A second pair with A = 74° and B = 73° becomes (73.5°, −1°). The first coordinate describes the measured magnitude; the second describes disagreement and its direction.
A positive difference means B reads higher. Reversing the subtraction reverses signs; it does not improve agreement. Check the axis label before interpreting “positive bias.”
Plotting B − A against A alone shares A between both axes and can create a misleading relationship. The paired mean is generally the more useful starting point when both methods contain error. Even then, substantially different measurement-error variances can contribute to a trend; the graph does not automatically identify its cause. Bland and Altman, 1995.
Calculate bias and limits separately
The fictional forty-person report supplies a mean difference of +2° and a sample standard deviation of the differences of 3°. For the calculation, assume approximately normal differences, no material magnitude-related change in their mean or spread, and independent participants.
The estimated bias is +2°: B reads two degrees higher on average. It does not follow that every B reading is two degrees higher.
The conventional estimated 95% limits of agreement are:
Mean difference ± 1.96 × SD of differences
2 ± 1.96 × 3 = −3.88° to +7.88°.
Under the assumptions, these estimate the range containing about 95% of differences for comparable paired measurements. They are not guarantees for a particular person or confidence limits around the average bias. The basic method and its assumptions are described by Bland and Altman. 1986 paper.
Compare the whole interval with the fictional acceptable range of −5° to +5°. The upper estimated limit exceeds +5°. The result does not support interchangeability under the team's stated criterion, despite a small-looking average difference.
The established method's name does not tell us which device is responsible. Agreement between two imperfect methods is also not proof that either measures the true angle accurately.
Do not confuse uncertainty about the mean with individual disagreement
For these supplied numbers, the standard error of the mean difference is 3 ÷ √40 = 0.474°. Using a t multiplier of approximately 2.023 with 39 degrees of freedom gives a 95% confidence interval for the bias of approximately +1.04° to +2.96°.
That narrow interval concerns the population's average difference. It does not replace the much wider −3.88° to +7.88° limits describing individual paired differences.
The limits of agreement are themselves estimated and need uncertainty intervals when judging their precision. Forty is not a universal adequate sample size, and observing estimated limits just inside an acceptance range would not settle the decision if their uncertainty were large. Bland and Altman, 1986.
Increasing sample size can estimate disagreement more precisely without making the methods agree more closely. This is why a statistically significant average bias and a practically unacceptable spread are separate findings.
Challenge a simple correction
A team member proposes subtracting two degrees from every B reading. In the worked example, that would centre the estimated differences on zero while leaving their standard deviation at three degrees. The shifted limits would be −5.88° to +5.88°, still outside the chosen acceptance range.
Removing average bias does not remove person-to-person disagreement. Moreover, a correction developed from these data would need validation under the intended conditions; fitting it and celebrating its performance on the same observations is insufficient.
Now consider a different invented device where B = 1.10 × A. The difference is +2° at A = 20° but +8° at A = 80°. Subtracting one constant cannot remove this proportional discrepancy. The exact numerical relationship is supplied here; in real data, an apparent slope needs investigation rather than immediate causal interpretation.
Distinguish a changing average difference from a changing spread. A plot might slope, widen at larger values, or do both. A justified transformation or a model allowing magnitude-dependent limits may be needed; one pair of horizontal limits can conceal poor performance in part of the range. Bland and Altman, 1995.
Check what the observations represent
If forty people each supplied five pairs, there would be 200 pairs but not 200 independent people. Repeated-measurement methods must account for that structure. Averaging five readings per method can also narrow the apparent disagreement while changing the target from a single reading to an average. Bland and Altman's later paper addresses these distinctions. Bland and Altman, 2007.
Likewise, if one method is always used after a tiring task, differences could mix method effects with real change. A technically correct calculation cannot separate explanations the collection procedure has combined.
Transfer exercise: improve the conclusion, not the headline
Original fictional exercise: A new comparison has mean B − A of −0.5° and SD of differences of 1.2°. The same ±5° acceptance range was specified in advance. Assume independent pairs, approximately normal differences and no important trend or widening across the observed range.
Calculate the estimated limits and choose the most defensible interpretation:
- A: The methods are proven interchangeable in every rehabilitation setting.
- B: The estimated limits fit this study's acceptance range; their precision and applicability still need appraisal.
- C: The negative average difference proves the new method is inaccurate.
The limits are −0.5 ± 1.96 × 1.2 = −2.852° to +1.852°, making B the supported answer. A extends beyond the studied protocol and ignores uncertainty. C mistakes a between-method difference for error against known truth.
For your next measurement paper, record the subtraction direction, paired-difference spread, prespecified acceptable range and intended use. Then ask whether the design and uncertainty support that particular substitution. A high correlation alone cannot answer it.
Sources and further reading
- Bland and Altman, 1986 — Author-hosted corrected full-text reproduction of the foundational agreement paper.
- Bland and Altman, 1995 — Difference-versus-mean plots and artefacts; author-hosted full text.
- Bland and Altman, 2007 — Repeated observations and single-reading versus averaged-reading agreement; introductory and concepts sections consulted.