A Software Tool for Calculating the Uncertainty of Diagnostic Accuracy Measures.
Ontology highlight
ABSTRACT:
SUBMITTER: Chatzimichail T
PROVIDER: S-EPMC7997227 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Screening and diagnostic tests are used to classify people with and without a disease. Diagnostic accuracy measures are used to evaluate the correctness of a classification in clinical research and practice. Although this depends on the uncertainty of measurement, there has been limited research on their relation. The objective of this work was to develop an exploratory tool for the relation between diagnostic accuracy measures and measurement uncertainty, as diagnostic accuracy is fundamental to clinical decision-making, while measurement uncertainty is critical to quality and risk management in laboratory medicine. For this reason, a freely available interactive program was developed for calculating, optimizing, plotting and comparing various diagnostic accuracy measures and the corresponding risk of diagnostic or screening tests measuring a normally distributed measurand, applied at a single point in time in non-diseased and diseased populations. This is done for differing prevalence of the disease, mean and standard deviation of the measurand, diagnostic threshold, standard measurement uncertainty of the tests and expected loss. The application of the program is illustrated with a case study of glucose measurements in diabetic and non-diabetic populations. The program is user-friendly and can be used as an educational and research tool in medical decision-making.
| S-EPMC7555914 | biostudies-literature
Project description:Reactions to uncertainty in clinical medicine can affect decision making.To assess the extent to which radiologists' reactions to uncertainty influence diagnostic mammography interpretation.Cross-sectional responses to a mailed survey assessed reactions to uncertainty using a well-validated instrument. Responses were linked to radiologists' diagnostic mammography interpretive performance obtained from three regional mammography registries.One hundred thirty-two radiologists from New Hampshire, Colorado, and Washington.Mean scores and either standard errors or confidence intervals were used to assess physicians' reactions to uncertainty. Multivariable logistic regression models were fit via generalized estimating equations to assess the impact of uncertainty on diagnostic mammography interpretive performance while adjusting for potential confounders.When examining radiologists' interpretation of additional diagnostic mammograms (those after screening mammograms that detected abnormalities), a 5-point increase in the reactions to uncertainty score was associated with a 17% higher odds of having a positive mammogram given cancer was diagnosed during follow-up (sensitivity), a 6% lower odds of a negative mammogram given no cancer (specificity), a 4% lower odds (not significant) of a cancer diagnosis given a positive mammogram (positive predictive value [PPV]), and a 5% higher odds of having a positive mammogram (abnormal interpretation).Mammograms interpreted by radiologists who have more discomfort with uncertainty have higher likelihood of being recalled.
| S-EPMC1824735 | biostudies-other
Project description:The objective of this paper is to describe general approaches of diagnostic test accuracy (DTA) that are available for the quantitative synthesis of data using R software. We conduct a DTA that summarizes statistics for univariate analysis and bivariate analysis. The package commands of R software were "metaprop" and "metabin" for sensitivity, specificity, and diagnostic odds ratio; forest for forest plot; reitsma of "mada" for a summarized receiver-operating characteristic (ROC) curve; and "metareg" for meta-regression analysis. The estimated total effect sizes, test for heterogeneity and moderator effect, and a summarized ROC curve are reported using R software. In particular, we focus on how to calculate the effect sizes of target studies in DTA. This study focuses on the practical methods of DTA rather than theoretical concepts for researchers whose fields of study were non-statistics related. By performing this study, we hope that many researchers will use R software to determine the DTA more easily, and that there will be greater interest in related research.
| S-EPMC6545496 | biostudies-literature
Project description:Nowadays, the gold standard method for malaria diagnosis is a staining of thick and thin blood film examined by expert laboratorists. It requires well-trained laboratorists, which is a time consuming task, and is un-automated protocol. For this study, Maladiag Software was developed to predict malaria infection in suspected malaria patients. The demographic data of patients, examination for malaria parasites, and complete blood count (CBC) profiles were analyzed. Binary logistic regression was used to create the equation for the malaria diagnosis. The diagnostic parameters of the equation were tested on 4,985 samples (703 infected and 4,282 control samples). The equation indicated 81.2% sensitivity and 80.3% specificity for predicting infection of malaria. The positive likelihood and negative likelihood ratio were 4.12 (95% CI = 4.01-4.23) and 0.23 (95% CI = 0.22-0.25), respectively. This parameter also had odds ratios (P value < 0.0001, OR = 17.6, 95% CI = 16.0-19.3). The equation can predict malaria infection after adjust for age, gender, nationality, monocyte (%), platelet count, neutrophil (%), lymphocyte (%), and the RBC count of patients. The diagnostic accuracy was 0.877 (Area under curve, AUC) (95% CI = 0.871-0.883). The system, when used in combination with other clinical and microscopy methods, might improve malaria diagnoses and enhance prompt treatment.
| S-EPMC4642325 | biostudies-other
Project description:Despite the high prevalence of traumatic brain injuries (TBI), there are few rapid and straightforward tests to improve its assessment. To this end, we developed a tablet-based software battery ("BrainCheck") for concussion detection that is well suited to sports, emergency department, and clinical settings. This article is a study of the diagnostic accuracy of BrainCheck. We administered BrainCheck to 30 TBI patients and 30 pain-matched controls at a hospital Emergency Department (ED), and 538 healthy individuals at 10 control test sites. We compared the results of the tablet-based assessment against physician diagnoses derived from brain scans, clinical examination, and the SCAT3 test, a traditional measure of TBI. We found consistent distributions of normative data and high test-retest reliability. Based on these assessments, we defined a composite score that distinguishes TBI from non-TBI individuals with high sensitivity (83%) and specificity (87%). We conclude that our testing application provides a rapid, portable testing method for TBI.
| S-EPMC5501428 | biostudies-literature
Project description:Overall accuracy measures of medical tests are often used with unclear interpretations. To develop methods of calculating the overall accuracy of medical tests in the patient population. Algebraic equations based on Bayes' theorem. A new approach is proposed for calculating overall accuracy in the patient population. Examples and applications using published data are presented. The overall accuracy is the proportion of the correct test results. We introduce a clear distinction between the overall accuracy measures of medical tests that are aimed at the detection of a disease in a screening of populations for public health purposes in the general population and the overall accuracy measures of tests aimed at determining a diagnosis in individuals in a clinical setting. We show that the overall detection accuracy measure is obtained in a specific study that explores test accuracy among persons with known diagnoses and may be useful for public health screening tests. It is different from the overall diagnostic accuracy that could be calculated in the clinical setting for the evaluation of medical tests aimed at determining the individual patients' diagnoses. We show that the overall detection accuracy is constant and is not affected by the prevalence of the disease. In contrast, the overall diagnostic accuracy changes and is dependent on the prevalence. Moreover, it ranges according to the ratio between the sensitivity and specificity. Thus, when the sensitivity is greater than the specificity, the overall diagnostic accuracy increases with increasing prevalence, and vice versa, that is, when the sensitivity is lower than the specificity, the overall diagnostic accuracy decreases with increasing prevalence so that another test might be more useful for diagnostic procedures. Our paper suggests a new and more appropriate methodology for estimating the overall diagnostic accuracy of any medical test. This may be important for helping clinicians avoid errors.
| S-EPMC6186004 | biostudies-literature
Project description:BackgroundTimely and accurate discrimination of wide complex tachycardias (WCTs) into ventricular tachycardia (VT) or supraventricular WCT (SWCT) is critically important. Previously we developed and validated an automated VT Prediction Model that provides a VT probability estimate using the paired WCT and baseline 12-lead ECGs. Whether this model improves physicians' diagnostic accuracy has not been evaluated.ObjectiveWe sought to determine whether the VT Prediction Model improves physicians' WCT differentiation accuracy.MethodsOver four consecutive days, nine physicians independently interpreted fifty WCT ECGs (25 VTs and 25 SWCTs confirmed by electrophysiological study) as either VT or SWCT. Day 1 used the WCT ECG only, Day 2 used the WCT and baseline ECG, Day 3 used the WCT ECG and the VT Prediction Model's estimation of VT probability, and Day 4 used the WCT ECG, baseline ECG, and the VT Prediction Model's estimation of VT probability.ResultsInclusion of the VT Prediction Model data increased diagnostic accuracy versus the WCT ECG alone (Day 3: 84.2% vs. Day 1: 68.7%, p 0.009) and WCT and baseline ECGs together (Day 3: 84.2% vs. Day 2: 76.4%, p 0.003). There was no further improvement of accuracy with addition of the baseline ECG comparison to the VT Prediction Model (Day 3: 84.2% vs. Day 4: 84.0%, p 0.928). Overall sensitivity (Day 3: 78.2% vs. Day 1: 67.6%, p 0.005) and specificity (Day 3: 90.2% vs. Day 1: 69.8%, p 0.016) for VT were superior after the addition of the VT Prediction Model.ConclusionThe VT Prediction Model improves physician ECG diagnostic accuracy for discriminating WCTs.
| S-EPMC9799284 | biostudies-literature
Project description:BackgroundThere are a variety of ways in which accuracy of clinical tests can be summarised in systematic reviews. Variation in reporting of summary measures has only been assessed in a small survey restricted to meta-analyses of screening studies found in a single database. Therefore, we performed this study to assess the measures of accuracy used for reporting results of primary studies as well as their meta-analysis in systematic reviews of test accuracy studies.MethodsRelevant reviews on test accuracy were selected from the Database of Abstracts of Reviews of Effectiveness (1994-2000), which electronically searches seven bibliographic databases and manually searches key resources. The structured abstracts of these reviews were screened and information on accuracy measures was extracted from the full texts of 90 relevant reviews, 60 of which used meta-analysis.ResultsSensitivity or specificity was used for reporting the results of primary studies in 65/90 (72%) reviews, predictive values in 26/90 (28%), and likelihood ratios in 20/90 (22%). For meta-analysis, pooled sensitivity or specificity was used in 35/60 (58%) reviews, pooled predictive values in 11/60 (18%), pooled likelihood ratios in 13/60 (22%), and pooled diagnostic odds ratio in 5/60 (8%). Summary ROC was used in 44/60 (73%) of the meta-analyses. There were no significant differences in measures of test accuracy among reviews published earlier (1994-97) and those published later (1998-2000).ConclusionsThere is considerable variation in ways of reporting and summarising results of test accuracy studies in systematic reviews. There is a need for consensus about the best ways of reporting results of test accuracy studies in reviews.
| S-EPMC100326 | biostudies-literature
Project description:BACKGROUND:Traditional selection in livestock and crops focuses on additive genetic values or breeding values of the individuals. While traditional selection utilizes variation between individuals, differences between gametes within individuals have been less frequently exploited in selection programs. With the successful implementation of genomic selection in livestock and crops, estimation and selection for gametic variation is becoming possible. RESULTS:The gamevar.f90 software is designed to estimate individual-level variance of genetic values of gametes for complex traits in large populations. The software estimates the (co)variances of gametic diversity as well as other diversity parameters that are useful for selection programs and mating designs. The calculation is carried out chromosome by chromosome and can be easily parallelized. The gamevar.f90 program is written in Fortran with efficient computing algorithms in a user-friendly software package with easily-handled input and output files. Finally, we applied the program to estimate gametic variance for hundreds of bulls for lifetime net merit, productive life, and livability. The RPTA (relative predicted transmitting ability), assuming a future selection intensity (if) of 1.5, showed larger variance than GEBV/2, indicating that greater future genetic gains can be obtained using an index that includes gametic variances. We also used the relative coefficient of variation to estimate with 95% confidence the sample sizes required to observe 90% variability of the progeny for lifetime net merit (or to allow at maximum 10% of change in the EBV predicted from progeny data). CONCLUSIONS:Collectively, we develop an efficient computer program package, gamevar.f90, for estimating gametic variance for large numbers of individuals. The novel information on gametic variation will be useful in future animal and crop breeding programs.
| S-EPMC7060627 | biostudies-literature
Project description:There is a need for epidemiological and medical researchers to identify new biomarkers (biological markers) that are useful in determining exposure levels and/or for the purposes of disease detection. Often this process is stunted by high testing costs associated with evaluating new biomarkers. Traditionally, biomarker assessments are individually tested within a target population. Pooling has been proposed to help alleviate the testing costs, where pools are formed by combining several individual specimens. Methods for using pooled biomarker assessments to estimate discriminatory ability have been developed. However, all these procedures have failed to acknowledge confounding factors. In this paper, we propose a regression methodology based on pooled biomarker measurements that allow the assessment of the discriminatory ability of a biomarker of interest. In particular, we develop covariate-adjusted estimators of the receiver-operating characteristic curve, the area under the curve, and Youden's index. We establish the asymptotic properties of these estimators and develop inferential techniques that allow one to assess whether a biomarker is a good discriminator between cases and controls, while controlling for confounders. The finite sample performance of the proposed methodology is illustrated through simulation. We apply our methods to analyze myocardial infarction (MI) data, with the goal of determining whether the pro-inflammatory cytokine interleukin-6 is a good predictor of MI after controlling for the subjects' cholesterol levels.
| S-EPMC6234508 | biostudies-literature