Logit models, the area under receiver characteristic curves, sensitivity, and specificity for Co-enrollment density in college networks dataset.
Ontology highlight
ABSTRACT:
SUBMITTER: Huerta-Manzanilla E
PROVIDER: S-EPMC8573127 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundDisease prevalence is rarely explicitly considered in the early stages of the development of novel prognostic tests. Rather, researchers use the area under the receiver operating characteristic (AUROC) as the key metric to gauge and report predictive performance ability. Because this statistic does not account for disease prevalence, proposed tests may not appropriately address clinical requirements. This ultimately impedes the translation of prognostic tests into clinical practice.MethodsA method to express positive- and/or negative predictive value criteria (PPV, NPV) within the ROC space is presented. Equations are derived for so-called equi-PPV (and equi-NPV) lines. Herewith it is possible, for any given prevalence, to plot a series of sensitivity-specificity pairs which meet a specified PPV (or NPV) criterion onto the ROC space.This concept is introduced by firstly reviewing the well-established "mechanics", strengths and limitations of the ROC analysis in the context of developing prognostic models. Then, the use of PPV (and/or) NPV criteria to augment the ROC analysis is elaborated.Additionally, an interactive web tool was also created to enable people to explore the dynamics of lines of equi-predictive value in function of prevalence. The web tool also allows to gauge what ROC curve shapes best meet specific positive and/or negative predictive value criteria (http://d4ta.link/ppvnpv/).ResultsTo illustrate the merits and implications of this concept, an example on the prediction of pre-eclampsia risk in low-risk nulliparous pregnancies is elaborated.ConclusionsIn risk stratification, the clinical usefulness of a prognostic test can be expressed in positive- and negative predictive value criteria; the development of novel prognostic tests will be facilitated by the possibility to co-visualise such criteria together with ROC curves. To achieve clinically meaningful risk stratification, the development of separate tests to meet either a pre-specified positive value (rule-in) or a negative predictive value (rule-out) criteria should be considered: the characteristics of successful rule-in and rule-out tests may markedly differ.
| S-EPMC6460848 | biostudies-literature
Project description:BACKGROUND:Despite some advances, cardiovascular disease (CVD) remains the leading cause of death and healthcare costs in the United States. We therefore developed a comprehensive CVD policy simulation model that identifies cost-effective approaches for reducing CVD burden. This paper aims to: 1) describe our model in detail; and 2) perform model validation analyses. METHODS:The model simulates 1,000,000 adults (ages 35 to 80 years) using a variety of CVD-related epidemiological data, including previously calibrated Framingham-based risk scores for coronary heart disease and stroke. We validated our microsimulation model using recent National Health and Nutrition Examination Survey (NHANES) data, with baseline values collected in 1999-2000 and cause-specific mortality follow-up through 2011. Model-based (simulated) results were compared to observed all-cause and CVD-specific mortality data (from NHANES) for the same starting population using survival curves and, in a method not typically used for disease model validation, receiver operating characteristic (ROC) curves. RESULTS:Observed 10-year all-cause mortality in NHANES v. the simulation model was 11.2% (95% CI, 10.3% to 12.2%) v. 10.9%; corresponding results for CVD mortality were 2.2% (1.8% to 2.7%) v. 2.6%. Areas under the ROC curves for model-predicted 10-year all-cause and CVD mortality risks were 0.83 (0.81 to 0.85) and 0.84 (0.81 to 0.88), respectively; corresponding results for 5-year risks were 0.80 (0.77 to 0.83) and 0.81 (0.75 to 0.87), respectively. LIMITATIONS:The model is limited by the uncertainties in the data used to estimate its input parameters. Additionally, our validation analyses did not include non-fatal CVD outcomes due to NHANES data limitations. CONCLUSIONS:The simulation model performed well in matching to observed nationally representative longitudinal mortality data. ROC curve analysis, which has been traditionally used for risk prediction models, can also be used to assess discrimination for disease simulation models.
| S-EPMC5577377 | biostudies-literature
Project description:Receiver operating characteristic analysis provides an important methodology for assessing traditional (e.g., imaging technologies and clinical practices) and new (e.g., genomic studies, biomarker development) diagnostic problems. The area under the clinically/practically relevant part of the receiver operating characteristic curve (partial area or partial area under the receiver operating characteristic curve) is an important performance index summarizing diagnostic accuracy at multiple operating points (decision thresholds) that are relevant to actual clinical practice. A robust estimate of the partial area under the receiver operating characteristic curve is provided by the area under the corresponding part of the empirical receiver operating characteristic curve. We derive a closed-form expression for the jackknife variance of the partial area under the empirical receiver operating characteristic curve. Using the derived analytical expression, we investigate the differences between the jackknife variance and a conventional variance estimator. The relative properties in finite samples are demonstrated in a simulation study. The developed formula enables an easy way to estimate the variance of the empirical partial area under the receiver operating characteristic curve, thereby substantially reducing the computation burden, and provides important insight into the structure of the variability. We demonstrate that when compared with the conventional approach, the jackknife variance has substantially smaller bias, and leads to a more appropriate type I error rate of the Wald-type test. The use of the jackknife variance is illustrated in the analysis of a data set from a diagnostic imaging study.
| S-EPMC5133195 | biostudies-literature
Project description:The Alcohol Use Disorders Identification Test (AUDIT) is the gold standard screening measure. Recently, there has been increasing call to update the measure to reflect harmful drinking standards in the United States. The purpose of this study was to use receiver operating characteristic curve analysis to evaluate the AUDIT and the United States version (AUDIT-US). Participants were 382 traditional age (M = 20.2, SD = 1.5) college students (68.7% female, 64.9% White) who had consumed alcohol at least once in the 30 days prior to participating. Although results provide evidence for the AUDIT and the AUDIT-US as valid screening tools, the Consumption subscale of the AUDIT-US performed the best in predicting at-risk college drinkers. The Consumption subscale of the AUDIT-US with a single cutoff score of four appears to be the optimal and most parsimonious method of identifying at-risk college drinkers.
| S-EPMC7163156 | biostudies-literature
Project description:The area under the receiver operating characteristic curve is a widely used measure for evaluating the performance of a diagnostic test. Common approaches for inference on area under the receiver operating characteristic curve are usually based upon approximation. For example, the normal approximation based inference tends to suffer from the problem of low accuracy for small sample size. Frequentist empirical likelihood based approaches for area under the receiver operating characteristic curve estimation may perform better, but are usually conducted through approximation in order to reduce the computational burden, thus the inference is not exact. By contrast, we proposed an exact inferential procedure by adapting the empirical likelihood into a Bayesian framework and draw inference from the posterior samples of the area under the receiver operating characteristic curve obtained via a Gibbs sampler. The full conditional distributions within the Gibbs sampler only involve empirical likelihoods with linear constraints, which greatly simplify the computation. To further enhance the applicability and flexibility of the Bayesian empirical likelihood, we extend our method to the estimation of partial area under the receiver operating characteristic curve, comparison of multiple tests, and the doubly robust estimation of area under the receiver operating characteristic curve in the presence of missing test results. Simulation studies confirm the desirable performance of the proposed methods, and a real application is presented to illustrate its usefulness.
| S-EPMC9241147 | biostudies-literature
Project description:Receiver operating characteristic curves are widely used as a measure of accuracy of diagnostic tests and can be summarised using the area under the receiver operating characteristic curve (AUC). Often, it is useful to construct a confidence interval for the AUC; however, because there are a number of different proposed methods to measure variance of the AUC, there are thus many different resulting methods for constructing these intervals. In this article, we compare different methods of constructing Wald-type confidence interval in the presence of missing data where the missingness mechanism is ignorable. We find that constructing confidence intervals using multiple imputation based on logistic regression gives the most robust coverage probability and the choice of confidence interval method is less important. However, when missingness rate is less severe (e.g. less than 70%), we recommend using Newcombe's Wald method for constructing confidence intervals along with multiple imputation using predictive mean matching.
| S-EPMC6472951 | biostudies-literature
Project description:Unlabelled: Lu and Elston have recently proposed a procedure for developing optimal receiver operating characteristic curves that maximize the area under a receiver operating characteristic curve in the setting of a predictive genetic test. The method requires only summary data, not individual level genetic data. In an era of increased data sharing, we investigate the performance of this algorithm when individual level genetic data are available and compare this approach to more standard receiver operating characteristic curve-building methods.ConclusionThough the Lu-Elston method can produce an optimal area under the curve under some assumptions, the method typically has little advantage over standard multivariable logistic methods when data are available. Also, the standard approach easily allows comparison of nested models via likelihood ratio tests and incorporation of covariates - the Lu-Elston approach is shown to have some difficulties with such analyses. These conclusions are based on evaluations using the Genetic Analysis Workshop 16 rheumatoid arthritis data set.
| S-EPMC2795956 | biostudies-literature
Project description:Equations of state based on intermolecular potentials are often developed about the Lennard-Jones (LJ) potential. Many of such EOS have been proposed in the past. In this work, 20 LJ EOS were examined regarding their performance on Brown's characteristic curves and characteristic state points. Brown's characteristic curves are directly related to the virial coefficients at specific state points, which can be computed exactly from the intermolecular potential. Therefore, also the second and third virial coefficient of the LJ fluid were investigated. This approach allows a comparison of available LJ EOS at extreme conditions. Physically based, empirical, and semi-theoretical LJ EOS were examined. Most investigated LJ EOS exhibit some unphysical artifacts.
| S-EPMC7441092 | biostudies-literature
Project description:For censored survival outcomes, it can be of great interest to evaluate the predictive power of individual markers or their functions. Compared with alternative evaluation approaches, approaches based on the time-dependent receiver operating characteristics (ROC) rely on much weaker assumptions, can be more robust, and hence are preferred. In this article, we examine evaluation of markers' predictive power using the time-dependent ROC curve and a concordance measure that can be viewed as a weighted area under the time-dependent area under the ROC curve profile. This study significantly advances from existing time-dependent ROC studies by developing nonparametric estimators of the summary indexes and, more importantly, rigorously establishing their asymptotic properties. It reinforces the statistical foundation of the time-dependent ROC-based evaluation approaches for censored survival outcomes. Numerical studies, including simulations and application to an HIV clinical trial, demonstrate the satisfactory finite-sample performance of the proposed approaches.
| S-EPMC3743052 | biostudies-literature
Project description:Time-dependent receiver operating characteristic curves are often used to evaluate the classification performance of continuous measures when considering time-to-event data. When one is interested in evaluating the predictive performance of multiple covariates, it is common to use the Cox proportional hazards model to obtain risk scores; however, previous work has shown that when the model is mis-specified, the estimand corresponding to the partial likelihood estimator depends on the censoring distribution. In this manuscript, we show that when the risk score model is mis-specified, the AUC will also depend on the censoring distribution, leading to either over- or under-estimation of the risk score's predictive performance. We propose the use of censoring-robust estimators to remove the dependence on the censoring distribution and provide empirical results supporting the use of censoring-robust risk scores.
| S-EPMC8671363 | biostudies-literature