Can two highly correlated methods disagree? Bland–Altman limits
Read a Bland–Altman plot, calculate limits of agreement and compare paired measurement differences with a tolerance you set in advance.
AcademicFix Editorial Team · Sources reviewed
- Pearson correlation across the five pairs
- r = 1
- Method B minus method A in every pair
- +10 units
Fictional readings illustrate how a fixed offset can leave correlation unchanged.
See the five paired readingsWould you swap two instruments if the second always read 10 units higher? Their Pearson correlation could still be 1. To decide whether you can use two methods interchangeably, compare their paired differences with an acceptable difference set in the original units. This guide is for researchers comparing two methods that measure the same continuous quantity. Bland and Altman; Parker et al. on agreement and tolerance.
On this page
How can r = 1 hide a 10-unit difference?
In these five hypothetical pairs, B is exactly A plus 10. Every point sits on the line B = A + 10. Changing methods raises every reading by 10 units. A separate reference is needed if your question is which method is closer to the true value. Original method comparison, introduction and measuring agreement; Parker et al., methods and application.
| Unit | Method A | Method B | Pair mean | B − A |
|---|---|---|---|---|
| 1 | 10 | 20 | 15 | +10 |
| 2 | 20 | 30 | 25 | +10 |
| 3 | 30 | 40 | 35 | +10 |
| 4 | 40 | 50 | 45 | +10 |
| 5 | 50 | 60 | 55 | +10 |
Put the mean on the x-axis and the difference on the y-axis
Plot (A + B) / 2 horizontally and B − A vertically. Keep this subtraction order in your data, plot and report. The zero line marks equal readings. The mean difference line shows the average offset. In the five-pair example, every point lies at +10 on the vertical axis. Bland and Altman, measuring agreement; NIST, axes and reference lines.
Look for a slope, a fan shape and unusual points. A slope can indicate that the typical difference changes with measurement size. A fan can indicate that the spread changes. Investigate these patterns before applying one pair of limits across the full measurement range. The plot identifies patterns to examine further. NIST, plot interpretation; Parker et al., discussion.
Calculate the approximate 95% limits of agreement
The simple calculation assumes one pair per independent unit and differences that are approximately normal, with roughly constant mean and spread across the relevant range. The assumptions concern the differences. Bland and Altman, measuring agreement; NCSS PASS, technical details; Bland and Altman, independent pairs.
Estimated limits = mean difference ± 1.96 × sample standard deviation of the differences. Under the model, these endpoints estimate the central 95% range for individual differences in comparable conditions. Bland and Altman, introduction; Shieh, normal percentile estimates.
For a second fictional example, take ten differences: 6, 7, 8, 9, 10, 10, 11, 12, 13 and 14 units. The mean is +10, the sample SD is about 2.58, and the estimated limits are 4.9 to 15.1 units. These numbers illustrate arithmetic. The data cannot establish agreement or assess assumptions for any real instrument.
| Question | Report |
|---|---|
| How much higher does B read on average? | Mean difference, here +10 units. |
| How far apart might individual pairs be? | Estimated limits of agreement under the model. |
| How uncertain are those estimates? | A confidence interval for the mean difference and separate intervals for each limit. |
In the first example the differences have zero spread, so the formula gives +10 for both limits. This is a feature of the constructed example. NIST, separate confidence and agreement limits; Shieh, uncertainty in each endpoint.
Choose the tolerance before looking at the results
Suppose the research team sets an acceptable range of −5 to +5 units for a specific use. The second example’s upper limit, 15.1 units, exceeds that range. The estimates therefore fail the illustrative criterion. In a real study, justify the tolerance in terms of the decision affected and consider uncertainty in both limits before concluding interchangeability. Bland and Altman, acceptable differences; Parker et al., acceptable differences and uncertainty; NCSS PASS, comparison with a preselected tolerance.
Report the number of independent units, subtraction order, measurement units, mean difference, limits, uncertainty and tolerance. When you collect several pairs from the same person, choose whether the target is agreement for one reading or for an average of readings. Use a method that accounts for clustering instead of treating every repeat as a new participant. Bland and Altman, clustered observations; Parker et al., mixed effects limits.
Questions that change the conclusion
What if the mean difference is near zero? Large positive and negative differences can cancel. Inspect the scatter and the limits as well as the mean. Bland and Altman; NIST.
Are the limits a confidence interval for the mean difference? The limits estimate the spread of individual differences. A confidence interval for the mean measures how precisely you estimated the average offset. Each limit also has its own sampling uncertainty. NIST; Shieh.
Does subtracting the mean difference fix agreement? Subtracting a constant shifts all differences equally, leaving their spread unchanged. If you calibrate the method, assess the remaining differences over the intended range. Consider using new data for that check. Bland and Altman; NIST on calibration.
Check your method comparison before reporting it
- Do A and B refer to the same unit, quantity and comparable conditions?
- Did you specify the tolerance and subtraction order before analysis?
- Does the plot show any trend, widening spread or unusual points?
- Did you distinguish agreement limits from confidence intervals and account for repeated measurements?
For association and prediction questions, read correlation vs regression. For the separate role of estimation uncertainty, see p-values, effect sizes and confidence intervals. Then use the APA 7 table formatter to lay out your report.
Sources
- Bland and Altman (1986), original method comparison paper
- NIST Dataplot, Bland–Altman plot documentation
- Shieh (2018), confidence intervals for limits of agreement
- Bland and Altman (2007), multiple observations per individual
- Parker et al. (2020), agreement with continuous repeated measurements
- NCSS PASS, method comparison assumptions and decision rules