Using Bayes theorem to estimate positive and negative predictive values for continuously and ordinally scaled diagnostic tests.
Ontology highlight
ABSTRACT: Objectives
Positive predictive values (PPVs) and negative predictive values (NPVs) are frequently reported to put estimates of accuracy of a diagnostic test in clinical context and to obtain risk estimates for a given patient taking into account baseline prevalence in the population. In order to calculate PPV and NPV, tests with ordinally or continuously scaled results are commonly dichotomized at the expense of a loss of information.Methods
Extending the rationale for the calculation of PPV and NPV, Bayesian theorem is used to calculate the probability of disease given the outcome of a continuously or ordinally scaled test. Probabilities of test results conditional on disease status are modeled in a Bayesian framework and subsequently transformed to probabilities of disease status conditional on test result.Results
Using publicly available data, probability of a clinical depression diagnosis given PROMIS Depression scores was estimated. Comparison with PPV and NPV based on dichotomized scores shows that a more fine-grained interpretation of test scores is possible.Conclusions
The proposed method bears the chance to facilitate accurate and meaningful interpretation of test results in clinical settings by avoiding unnecessary dichotomization of test scores.
SUBMITTER: Fischer F
PROVIDER: S-EPMC8170576 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The COVID19 crisis has provided a portal to revisit and understand qualities of screening tests and the importance of Bayes' theorem in understanding how to interpret results and implications of next actions.
| S-EPMC7315940 | biostudies-literature
Project description:Medicine is diagnosis, treatment and care. To diagnose is to consider the probability of the cause of discomfort experienced by the patient. The physician may face many options and all decisions are liable to uncertainty to some extent. The rational action is to perform selected tests and thereby increase the pre-test probability to reach a superior post-test probability of a particular option. To draw the right conclusions from a test, certain background information about the performance of the test is necessary. We set up a partially artificial dataset with measured results obtained from the laboratory information system and simulated diagnosis attached. The dataset is used to explore the use of contingency tables with a unique graphic design and software to establish and compare ROC graphs. The loss of information in the ROC curve is compensated by a cumulative data analysis (CDA) plot linked to a display of the efficiency and predictive values. A standard for the contingency table is suggested and the use of dynamic reference intervals discussed.
| S-EPMC5707117 | biostudies-literature
Project description:IntroductionSensitivity, specificity, and predictive values-the basic statistics behind using and interpreting screening and diagnostic tests-are taught in all medical schools, yet studies have shown that a majority of physicians cannot correctly define and apply these concepts. Previous work has not rigorously examined this disconnect and attempted to address it.MethodsWe used adult learning theory to design a case-based interactive workshop to review biostatistics and apply them to clinical decision-making using Bayes' theorem. Participants took an anonymous multiple-choice pretest, posttest, and delayed posttest on definitions and application of the concepts, and we compared the scores between the three tests. Several experiences with early iterations provided feedback to improve the workshop but were not included for analysis.ResultsWe conducted the finalized workshop with 54 pediatrics students, residents, and faculty. All learners completed the immediate pre- and posttests, and eight completed the delayed posttest. Average scores rose from 4.5/8 (56%) on the pretest to 6.5/8 (81%) on the posttest and 6.4/8 (80%) on the delayed posttest. Two-tailed t tests showed p < .001 for the difference between the pretest and both posttests, and post hoc power analysis showed a power of 99% to detect the observed differences. There was no significant difference (p = .8) between the posttest and delayed posttest.DiscussionOur work demonstrates that an interactive workshop reviewing basic biostatistics and teaching rational diagnostic testing using Bayes' theorem can be effective in connecting theoretical knowledge of biostatistics to evidence-based decision-making in real clinical practice.
| S-EPMC6346275 | biostudies-literature
Project description:Using classroom activities to motivate the teaching and learning of Bayes’ theorem is not new. However, many of the textbook exercises and published simulations gloss over how the requisite probabilities are determined. In our case study, Able Construction is a fictional company hoping to exploit historical bidding data to inform its own bidding strategy on a municipal construction project. Unlike most other classroom activities, we challenge students to calculate the necessary probabilities directly from a given dataset. In our experience with implementing this case in introductory business analytics courses at the undergraduate- and graduate-level, we find that this spreadsheet activity gives students the opportunity to exercise their own judgement regarding data manipulation and definition of states of nature. This autonomy in analysis develops in students a deeper appreciation for practical skills required for possible analytics careers after graduation, and leads to engaging discussions of the applicability of Bayes’ theorem in practice. Supplementary Information The online version contains supplementary material available at 10.1007/s43069-021-00119-3.
| S-EPMC8784234 | biostudies-literature
Project description:BackgroundMachine learning (ML) can be an effective tool to extract information from attribute-rich molecular datasets for the generation of molecular diagnostic tests. However, the way in which the resulting scores or classifications are produced from the input data may not be transparent. Algorithmic explainability or interpretability has become a focus of ML research. Shapley values, first introduced in game theory, can provide explanations of the result generated from a specific set of input data by a complex ML algorithm.MethodsFor a multivariate molecular diagnostic test in clinical use (the VeriStrat® test), we calculate and discuss the interpretation of exact Shapley values. We also employ some standard approximation techniques for Shapley value computation (local interpretable model-agnostic explanation (LIME) and Shapley Additive Explanations (SHAP) based methods) and compare the results with exact Shapley values.ResultsExact Shapley values calculated for data collected from a cohort of 256 patients showed that the relative importance of attributes for test classification varied by sample. While all eight features used in the VeriStrat® test contributed equally to classification for some samples, other samples showed more complex patterns of attribute importance for classification generation. Exact Shapley values and Shapley-based interaction metrics were able to provide interpretable classification explanations at the sample or patient level, while patient subgroups could be defined by comparing Shapley value profiles between patients. LIME and SHAP approximation approaches, even those seeking to include correlations between attributes, produced results that were quantitatively and, in some cases qualitatively, different from the exact Shapley values.ConclusionsShapley values can be used to determine the relative importance of input attributes to the result generated by a multivariate molecular diagnostic test for an individual sample or patient. Patient subgroups defined by Shapley value profiles may motivate translational research. However, correlations inherent in molecular data and the typically small ML training sets available for molecular diagnostic test development may cause some approximation methods to produce approximate Shapley values that differ both qualitatively and quantitatively from exact Shapley values. Hence, caution is advised when using approximate methods to evaluate Shapley explanations of the results of molecular diagnostic tests.
| S-EPMC8265031 | biostudies-literature
Project description:The inferior alveolar nerve block (IANB) is the most common anesthetic technique used on mandibular teeth during root canal treatment. Its success in the presence of preoperative inflammation is still controversial. The aim of this study was to evaluate the sensitivity, specificity, predictive values, and accuracy of three diagnostic tests used to predict IANB failure in symptomatic irreversible pulpitis (SIP).A cross-sectional study was carried out on the mandibular molars of 53 patients with SIP. All patients received a single cartridge of mepivacaine 2% with 1 : 100000 epinephrine using the IANB technique. Three diagnostic clinical tests were performed to detect anesthetic failure. Anesthetic failure was defined as a positive painful response to any of the three tests. Sensitivity, specificity, predictive values, accuracy, and ROC curves were calculated and compared and significant differences were analyzed.IANB failure was determined in 71.7% of the patients. The sensitivity scores for the three tests (lip numbness, the cold stimuli test, and responsiveness during endodontic access) were 0.03, 0.35, and 0.55, respectively, and the specificity score was determined as 1 for all of the tests. Clinically, none of the evaluated tests demonstrated a high enough accuracy (0.30, 0.53, and 0.68 for lip numbness, the cold stimuli test, and responsiveness during endodontic access, resp.). A comparison of the areas under the curve in the ROC analyses showed statistically significant differences between the three tests (p < 0.05).None of the analyzed tests demonstrated a high enough accuracy to be considered a reliable diagnostic tool for the prediction of anesthetic failure.
| S-EPMC5488488 | biostudies-other
Project description:Antibodies testing in the coronavirus era is frequently promoted, but the underlying statistics behind their validation has come under more scrutiny in recent weeks. We provide calculations, interpretations, and plots of positive and negative predictive values under a variety of scenarios. Prevalence, sensitivity, and specificity are estimated within ranges of values from researchers and antibodies manufacturers. Illustrative examples are highlighted, and interactive plots are provided in the Supplementary Information. Implications are discussed for society overall and across diverse locations with different levels of disease burden. Specifically, the proportion of positive serology tests that are false can differ drastically from up to 3%-88% for people from different places with different proportions of infected people in the populations while the false negative rate is typically under 10%.
| S-EPMC7943825 | biostudies-literature
Project description:BackgroundSepsis is an organ dysfunction syndrome caused by the body's dysregulated response to infection. Yet, due to the heterogeneity of this disease process, the diagnosis and definition of sepsis is a critical issue in clinical work. Existing methods for early diagnosis of sepsis have low specificity.AimsThis study evaluated the diagnostic and predictive values of pyroptosis-related genes in normal and sepsis patients and their role in the immune microenvironment using multiple bioinformatics analyses and machine-learning methods.MethodsPediatric sepsis microarray datasets were screened from the GEO database and the differentially expressed genes (DEGs) associated with pyroptosis were analyzed. DEGs were then subjected to multiple bioinformatics analyses. The differential immune landscape between sepsis and healthy controls was explored by screening diagnostic genes using various machine-learning models. Also, the diagnostic value of these diagnosis-related genes in sepsis (miRNAs that have regulatory relationships with genes and related drugs that have regulatory relationships) were analyzed in the internal test set and external test.ResultsEight genes (CLEC5A, MALT1, NAIP, NLRC4, SERPINB1, SIRT1, STAT3, and TLR2) related to sepsis diagnosis were screened by multiple machine learning algorithms. The CIBERSORT algorithm confirmed that these genes were significantly correlated with the infiltration abundance of some immune cells and immune checkpoint sites (all P<0.05). SIRT1, STAT3, and TLR2 were identified by the DGIdb database as potentially regulated by multiple drugs. Finally, 7 genes were verified to have significantly different expressions between the sepsis group and the control group (P<0.05).ConclusionThe pyroptosis-related genes identified and verified in this study may provide a useful reference for the prediction and assessment of sepsis.
| S-EPMC9932037 | biostudies-literature
Project description:BackgroundBronchopulmonary dysplasia (BPD) is the most common respiratory complication in preterm infants, and there is a lag in the diagnosis of BPD. Inflammation is a vital pathogenic factor for BPD; we aim to evaluate the predictive and diagnostic values of systemic inflammatory indices in BPD.MethodsBetween 1 January 2019 and 31 May 2023, the clinical data of 122 premature infants with a gestational age of <32 weeks in the Department of Neonatology, the Affiliated Huai'an No. 1 People's Hospital of Nanjing Medical University, were retrospectively collected and classified into non-BPD (n = 72) and BPD (n = 50) groups based on the National Institute of Child Health and Human Development 2018 criteria. To compare the general characteristics of each group, we identified the independent risk variables for BPD using multivariate logistic regression analysis, compared the systemic inflammatory indices at birth, 72 h, 1 week, 2 weeks, and 36 weeks postmenstrual age (PMA), and constructed the receiver operating characteristic curves of neutrophil-to-lymphocyte ratio (NLR) diagnosis of BPD at different time points.Results① The independent risk factors for BPD in preterm infants were birth weight, small for gestational age, and days of oxygen therapy (all p < 0.05). ② At 72 h and 1 week after birth, the serum NLR of the BPD group was higher than for the non-BPD group (p < 0.05). Furthermore, the neutrophil count (N), NLR, monocyte-to-lymphocyte ratio (MLR), systemic immune-inflammation index, systemic inflammation response index (SIRI), and pan-immune-inflammation value of infants with BPD were higher than the non-BPD group at 3 weeks after birth (p < 0.05). Moreover, at 36 weeks of PMA, the serum N, NLR, MLR, and SIRI of BPD infants were higher than those of non-BPD infants (p < 0.05). ③ The NLR of infants with and without BPD gradually increased after birth, reaching a peak at 72 h and 1 week, respectively. At 3 weeks postnatal, the NLR had the highest predictive power for BPD, with an area under the curve (AUC) of 0.717 (p < 0.001); the sensitivity was 56% and specificity was 86.1%. In addition, the NLR at 36 weeks of PMA exhibited some diagnostic value for BPD. The AUC was 0.693 (p < 0.001), the sensitivity was 54%, and specificity was 83.3%.ConclusionsAt 3 weeks after birth and 36 weeks of PMA, some systemic inflammation indices (like N, NLR, SIRI) of preterm infants with BPD have specific predictive and diagnostic values; these indices may help the management of high-risk preterm infants with BPD.
| S-EPMC10814477 | biostudies-literature
Project description:A marker's capacity to predict risk of a disease depends on disease prevalence in the target population and its classification accuracy, i.e. its ability to discriminate diseased subjects from non-diseased subjects. The latter is often considered an intrinsic property of the marker; it is independent of disease prevalence and hence more likely to be similar across populations than risk prediction measures. In this paper, we are interested in evaluating the population-specific performance of a risk prediction marker in terms of positive predictive value (PPV) and negative predictive value (NPV) at given thresholds, when samples are available from the target population as well as from another population. A default strategy is to estimate PPV and NPV using samples from the target population only. However, when the marker's classification accuracy as characterized by a specific point on the receiver operating characteristics (ROC) curve is similar across populations, borrowing information across populations allows increased efficiency in estimating PPV and NPV. We develop estimators that optimally combine information across populations. We apply this methodology to a cross-sectional study where we evaluate PCA3 as a risk prediction marker for prostate cancer among subjects with or without previous negative biopsy.
| S-EPMC3196635 | biostudies-literature