Dataset Information

On Jones et al.'s method for extending Bland-Altman plots to limits of agreement with the mean for multiple observers.

ABSTRACT:

Background

To assess the agreement of continuous measurements between a number of observers, Jones et al. introduced limits of agreement with the mean (LOAM) for multiple observers, representing how much an individual observer can deviate from the mean measurement of all observers. Besides the graphical visualisation of LOAM, suggested by Jones et al., it is desirable to supply LOAM with confidence intervals and to extend the method to the case of multiple measurements per observer.

Methods

We reformulate LOAM under the assumption the measurements follow an additive two-way random effects model. Assuming this model, we provide estimates and confidence intervals for the proposed LOAM. Further, this approach is easily extended to the case of multiple measurements per observer.

Results

The proposed method is applied on two data sets to illustrate its use. Specifically, we consider agreement between measurements regarding tumour size and aortic diameter. For the latter study, three measurement methods are considered.

Conclusions

The proposed LOAM and the associated confidence intervals are useful for assessing agreement between continuous measurements.

SUBMITTER: Christensen HS

PROVIDER: S-EPMC7730774 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

On Jones et al.'s method for extending Bland-Altman plots to limits of agreement with the mean for multiple observers.

Christensen Heidi S HS Borgbjerg Jens J Børty Lars L Bøgsted Martin M

BMC medical research methodology 20201211 1

<h4>Background</h4>To assess the agreement of continuous measurements between a number of observers, Jones et al. introduced limits of agreement with the mean (LOAM) for multiple observers, representing how much an individual observer can deviate from the mean measurement of all observers. Besides the graphical visualisation of LOAM, suggested by Jones et al., it is desirable to supply LOAM with confidence intervals and to extend the method to the case of multiple measurements per observer.<h4>M ...[more]

PMID: 33308154

Similar Datasets

Project description:ObjectiveThis study aims to propose a comprehensive alternative to the Bland-Altman plot method, addressing its limitations and providing a statistical framework for evaluating the equivalences of measurement techniques. This involves introducing an innovative three-step approach for assessing accuracy, precision, and agreement between techniques, which enhances objectivity in equivalence assessment. Additionally, the development of an R package that is easy to use enables researchers to efficiently analyze and interpret technique equivalences.MethodsInferential statistics support for equivalence between measurement techniques was proposed in three nested tests. These were based on structural regressions with the goal to assess the equivalence of structural means (accuracy), the equivalence of structural variances (precision), and concordance with the structural bisector line (agreement in measurements obtained from the same subject), using analytical methods and robust approach by bootstrapping. To promote better understanding, graphical outputs following Bland and Altman's principles were also implemented.ResultsThe performance of this method was shown and confronted by five data sets from previously published articles that used Bland and Altman's method. One case demonstrated strict equivalence, three cases showed partial equivalence, and one showed poor equivalence. The developed R package containing open codes and data are available for free and with installation instructions at Harvard Dataverse at https://doi.org/10.7910/DVN/AGJPZH.ConclusionAlthough easy to communicate, the widely cited and applied Bland and Altman plot method is often misinterpreted, since it lacks suitable inferential statistical support. Common alternatives, such as Pearson's correlation or ordinal least-square linear regression, also fail to locate the weakness of each measurement technique. It may be possible to test whether two techniques have full equivalence by preserving graphical communication, in accordance with Bland and Altman's principles, but also adding robust and suitable inferential statistics. Decomposing equivalence into three features (accuracy, precision, and agreement) helps to locate the sources of the problem when fixing a new technique.

Project description:Reliable quantification in biological systems of endogenous low- and high-molecular substances, drugs and their metabolites, is of particular importance in diagnosis and therapy, and in basic and clinical research. The analytical characteristics of analytical approaches have many differences, including in core features such as accuracy, precision, specificity, and limits of detection (LOD) and quantitation (LOQ). Several different mathematic approaches were developed and used for the comparison of two analytical methods applied to the same chemical compound in the same biological sample. Generally, comparisons of results obtained by two analytical methods yields different quantitative results. Yet, which mathematical approach gives the most reliable results? Which mathematical approach is best suited to demonstrate agreement between the methods, or the superiority of an analytical method A over analytical method B? The simplest and most frequently used method of comparison is the linear regression analysis of data observed by method A (y) and the data observed by method B (x): y = α + βx. In 1986, Bland and Altman indicated that linear regression analysis, notably the use of the correlation coefficient, is inappropriate for method-comparison. Instead, Bland and Altman have suggested an alternative approach, which is generally known as the Bland-Altman approach. Originally, this method of comparison was applied in medicine, for instance, to measure blood pressure by two devices. The Bland-Altman approach was rapidly adapted in analytical chemistry and in clinical chemistry. To date, the approach suggested by Bland-Altman approach is one of the most widely used mathematical approaches for method-comparison. With about 37,000 citations, the original paper published in the journal The Lancet in 1986 is among the most frequently cited scientific papers in this area to date. Nevertheless, the Bland-Altman approach has not been really set on a quantitative basis. No criteria have been proposed thus far, in which the Bland-Altman approach can form the basis on which analytical agreement or the better analytical method can be demonstrated. In this article, the Bland-Altman approach is re-valuated from a quantitative bioanalytical perspective, and an attempt is made to propose acceptance criteria. For this purpose, different analytical methods were compared with Gold Standard analytical methods based on mass spectrometry (MS) and tandem mass spectrometry (MS/MS), i.e., GC-MS, GC-MS/MS, LC-MS and LC-MS/MS. Other chromatographic and non-chromatographic methods were also considered. The results for several different endogenous substances, including nitrate, anandamide, homoarginine, creatinine and malondialdehyde in human plasma, serum and urine are discussed. In addition to the Bland-Altman approach, linear regression analysis and the Oldham-Eksborg method-comparison approaches were used and compared. Special emphasis was given to the relation of difference and mean in the Bland-Altman approach. Currently available guidelines for method validation were also considered. Acceptance criteria for method agreement were proposed, including the slope and correlation coefficient in linear regression, and the coefficient of variation for the percentage difference in the Bland-Altman and Oldham-Eksborg approaches.

Dataset Information

On Jones et al.'s method for extending Bland-Altman plots to limits of agreement with the mean for multiple observers.

Background

Methods

Results

Conclusions

Publications

On Jones et al.'s method for extending Bland-Altman plots to limits of agreement with the mean for multiple observers.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets