Dataset Information

A Bayesian framework for performance assessment and comparison of imaging biomarker quantification methods.

ABSTRACT: Quantitative biomarkers derived from medical images are being used increasingly to help diagnose disease, guide treatment, and predict clinical outcomes. Measurement of quantitative imaging biomarkers is subject to bias and variability from multiple sources, including the scanner technologies that produce images, the approaches for identifying regions of interest in images, and the algorithms that calculate biomarkers from regions. Moreover, these sources may differ within and between the quantification methods employed by institutions, thus making it difficult to develop and implement multi-institutional standards. We present a Bayesian framework for assessing bias and variability in imaging biomarkers derived from different quantification methods, comparing agreement to a reference standard, studying prognostic performance, and estimating sample size for future clinical studies. The statistical methods are illustrated with data obtained from a positron emission tomography challenge conducted by members of the NCI's Quantitative Imaging Network program, in which tumor volumes were measured manually and with seven different semi-automated segmentation algorithms. Estimates and comparisons of bias and variability in the resulting measurements are provided along with an R software package for the technical performance analysis and an online web application for sample size and power analysis.

SUBMITTER: Smith BJ

PROVIDER: S-EPMC6045465 | biostudies-literature | 2019 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Bayesian framework for performance assessment and comparison of imaging biomarker quantification methods.

Smith Brian J BJ Beichel Reinhard R RR

Statistical methods in medical research 20171222 4

Quantitative biomarkers derived from medical images are being used increasingly to help diagnose disease, guide treatment, and predict clinical outcomes. Measurement of quantitative imaging biomarkers is subject to bias and variability from multiple sources, including the scanner technologies that produce images, the approaches for identifying regions of interest in images, and the algorithms that calculate biomarkers from regions. Moreover, these sources may differ within and between the quanti ...[more]

PMID: 29271301

Similar Datasets

Project description:Background: Evidence suggesting adverse drug reactions often emerges unsystematically and unpredictably in form of anecdotal reports, case series and survey data. Safety trials and observational studies also provide crucial information regarding the (un-)safety of drugs. Hence, integrating multiple types of pharmacovigilance evidence is key to minimising the risks of harm. Methods: In previous work, we began the development of a Bayesian framework for aggregating multiple types of evidence to assess the probability of a putative causal link between drugs and side effects. This framework arose out of a philosophical analysis of the Bradford Hill Guidelines. In this article, we expand the Bayesian framework and add "evidential modulators," which bear on the assessment of the reliability of incoming study results. The overall framework for evidence synthesis, "E-Synthesis", is then applied to a case study. Results: Theoretically and computationally, E-Synthesis exploits coherence of partly or fully independent evidence converging towards the hypothesis of interest (or of conflicting evidence with respect to it), in order to update its posterior probability. With respect to other frameworks for evidence synthesis, our Bayesian model has the unique feature of grounding its inferential machinery on a consolidated theory of hypothesis confirmation (Bayesian epistemology), and in allowing any data from heterogeneous sources (cell-data, clinical trials, epidemiological studies), and methods (e.g., frequentist hypothesis testing, Bayesian adaptive trials, etc.) to be quantitatively integrated into the same inferential framework. Conclusions: E-Synthesis is highly flexible concerning the allowed input, while at the same time relying on a consistent computational system, that is philosophically and statistically grounded. Furthermore, by introducing evidential modulators, and thereby breaking up the different dimensions of evidence (strength, relevance, reliability), E-Synthesis allows them to be explicitly tracked in updating causal hypotheses.

Project description:ObjectivesTo assess whether single-photon emission computed tomography (SPECT/CT) quantification of bone scintigraphy would improve diagnostic accuracy and offer a means of quantifying amyloid burden.BackgroundTransthyretin-related cardiac amyloidosis is common and can be diagnosed noninvasively using bone scintigraphy; interpretation, however, relies on planar images. SPECT/CT imaging offers 3-dimensional visualization.MethodsThis was a single-center, retrospective analysis of 99mTc-3,3-diphosphono-1,2-propanodicarboxylic acid (DPD) scans reported using the Perugini grading system (0 = negative; 1 to 3 = increasingly positive). Conventional planar quantification techniques (heart/contralateral lung, and heart/whole-body retention ratios) were performed. Heart, adjacent vertebra, paraspinal muscle and liver peak standardized uptake values (SUVpeak) were recorded from SPECT/CT acquisitions. An SUV retention index was also calculated: (cardiac SUVpeak/vertebral SUVpeak) × paraspinal muscle SUVpeak. In a subgroup of patients, SPECT/CT quantification was compared with myocardial extracellular volume quantification by CT imaging (ECVCT).ResultsA total of 100 DPD scans were analyzed (patient age 84 ± 9 years; 52% male): 40 were Perugini grade 0, 12 were grade 1, 41 were grade 2, and 7 were grade 3. Cardiac SUVpeak increased from grade 0 to grade 2; however, it plateaued between grades 2 and 3 (p < 0.001). Paraspinal muscle SUVpeak increased with grade (p < 0.001), whereas vertebral SUVpeak decreased (p < 0.001). The composite parameter of SUV retention index overcame the plateauing of the cardiac SUVpeak and increased across all grades (p < 0.001). Cardiac SUVpeak correlated well (r2 = 0.73; p < 0.001) with ECVCT. Both the cardiac SUVpeak and SUV retention index had excellent diagnostic accuracy (area under the curve [AUC]: 0.999). The heart to contralateral lung ratio performed the best of the planar quantification techniques (AUC: 0.987).ConclusionsSPECT/CT quantification in DPD scintigraphy is possible and outperforms planar quantification techniques. Differentiation of Perugini grade 2 or 3 is confounded by soft tissue uptake, which can be overcome by a composite SUV retention index. This index can help in the diagnosis of cardiac amyloidosis and may offer a means of monitoring response to therapy.

Project description:In order to have confidence in model-based phylogenetic methods, such as maximum likelihood (ML) and Bayesian analyses, one must use an appropriate model of molecular evolution identified using statistically rigorous criteria. Although model selection methods such as the likelihood ratio test and Akaike information criterion are widely used in the phylogenetic literature, model selection methods lack the ability to reject all models if they provide an inadequate fit to the data. There are two methods, however, that assess absolute model adequacy, the frequentist Goldman-Cox (GC) test and Bayesian posterior predictive simulations (PPSs), which are commonly used in conjunction with the multinomial log likelihood test statistic. In this study, we use empirical and simulated data to evaluate the adequacy of common substitution models using both frequentist and Bayesian methods and compare the results with those obtained with model selection methods. In addition, we investigate the relationship between model adequacy and performance in ML and Bayesian analyses in terms of topology, branch lengths, and bipartition support. We show that tests of model adequacy based on the multinomial likelihood often fail to reject simple substitution models, especially when the models incorporate among-site rate variation (ASRV), and normally fail to reject less complex models than those chosen by model selection methods. In addition, we find that PPSs often fail to reject simpler models than the GC test. Use of the simplest substitution models not rejected based on fit normally results in similar but divergent estimates of tree topology and branch lengths. In addition, use of the simplest adequate substitution models can affect estimates of bipartition support, although these differences are often small with the largest differences confined to poorly supported nodes. We also find that alternative assumptions about ASRV can affect tree topology, tree length, and bipartition support. Our results suggest that using the simplest substitution models not rejected based on fit may be a valid alternative to implementing more complex models identified by model selection methods. However, all common substitution models may fail to recover the correct topology and assign appropriate bipartition support if the true tree shape is difficult to estimate regardless of model adequacy.

Project description:BACKGROUND:Although the presence of late gadolinium enhancement (LGE) using cardiovascular magnetic resonance imaging (CMR) is a significant discriminator of events in patients with suspected myocarditis, no data are available on the optimal LGE quantification method. METHODS:Six hundred seventy consecutive patients (48?±?16?years, 59% male) with suspected myocarditis were enrolled between 2002 and 2015. We performed LGE quantitation using seven different signal intensity thresholding methods based either on 2, 3, 4, 5, 6, 7 standard deviations (SD) above remote myocardium or full width at half maximum (FWHM). In addition, a LGE visual presence score (LGE-VPS) (LGE present/absent in each segment) was assessed. For each of these methods, the strength of association of LGE results with major adverse cardiac events (MACE) was determined. Inter-and intra-rater variability using intraclass-correlation coefficient (ICC) was performed for all methods. RESULTS:Ninety-eight (15%) patients experienced a MACE at a medium follow-up of 4.7?years. LGE quantification by FWHM, 2- and 3-SD demonstrated univariable association with MACE (hazard ratio [HR] 1.05, 95% confidence interval [CI]:1.02-1.08, p?=?0.001; HR 1.02, 95%CI:1.00-1.04; p?=?0.001; HR 1.02, 95%CI: 1.00-1.05, p?=?0.035, respectively), whereas 4-SD through 7-SD methods did not reach significant association. LGE-VPS also demonstrated association with MACE (HR 1.09, 95%CI: 1.04-1.15, p?<?0.001). In the multivariable model, FWHM, 2-SD methods, and LGE-VPS each demonstrated significant association with MACE adjusted to age, sex, BMI and LVEF (adjusted HR of 1.04, 1.02, and 1.07; p?=?0.009, p?=?0.035; and p?=?0.005, respectively). In these, FWHM and LGE-VPS had the highest degrees of inter and intra-rater reproducibility based on their high ICC values. CONCLUSIONS:FWHM is the optimal semi-automated quantification method in risk-stratifying patients with suspected myocarditis, demonstrating the strongest association with MACE and the highest technical consistency. Visual LGE scoring is a reliable alternative method and is associated with a comparable association with MACE and reproducibility in these patients. TRIAL REGISTRATION NUMBER:NCT03470571 . Registered 13th March 2018. Retrospectively registered.

Dataset Information

A Bayesian framework for performance assessment and comparison of imaging biomarker quantification methods.

Publications

A Bayesian framework for performance assessment and comparison of imaging biomarker quantification methods.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets