Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Censoring-robust time-dependent receiver operating characteristic curve estimators.

ABSTRACT: Time-dependent receiver operating characteristic curves are often used to evaluate the classification performance of continuous measures when considering time-to-event data. When one is interested in evaluating the predictive performance of multiple covariates, it is common to use the Cox proportional hazards model to obtain risk scores; however, previous work has shown that when the model is mis-specified, the estimand corresponding to the partial likelihood estimator depends on the censoring distribution. In this manuscript, we show that when the risk score model is mis-specified, the AUC will also depend on the censoring distribution, leading to either over- or under-estimation of the risk score's predictive performance. We propose the use of censoring-robust estimators to remove the dependence on the censoring distribution and provide empirical results supporting the use of censoring-robust risk scores.

SUBMITTER: Nuno MM

PROVIDER: S-EPMC8671363 | biostudies-literature | 2021 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Censoring-robust time-dependent receiver operating characteristic curve estimators.

Nuño Michelle M MM Gillen Daniel L DL

Statistics in medicine 20211017 30

Time-dependent receiver operating characteristic curves are often used to evaluate the classification performance of continuous measures when considering time-to-event data. When one is interested in evaluating the predictive performance of multiple covariates, it is common to use the Cox proportional hazards model to obtain risk scores; however, previous work has shown that when the model is mis-specified, the estimand corresponding to the partial likelihood estimator depends on the censoring d ...[more]

PMID: 34658036

Similar Datasets

Analyzing heterogeneity in biomarker discriminative performance through partial time-dependent receiver operating characteristic curve modeling.

Project description:This study investigates the heterogeneity of a biomarker's discriminative performance for predicting subsequent time-to-event outcomes across different patient subgroups. While the area under the curve (AUC) for the time-dependent receiver operating characteristic curve is commonly used to assess biomarker performance, the partial time-dependent AUC (PAUC) provides insights that are often more pertinent for population screening and diagnostic testing. To achieve this objective, we propose a regression model tailored for PAUC and develop two distinct estimation procedures for discrete and continuous covariates, employing a pseudo-partial likelihood method. Simulation studies are conducted to assess the performance of these procedures across various scenarios. We apply our model and inference procedure to the Alzheimer's Disease Neuroimaging Initiative data set to evaluate potential heterogeneities in the discriminative performance of biomarkers for early Alzheimer's disease diagnosis based on patients' characteristics.

| S-EPMC11449645 | biostudies-literature

Using the optimal robust receiver operating characteristic (ROC) curve for predictive genetic tests.

Project description:Current ongoing genome-wide association (GWA) studies represent a powerful approach to uncover common unknown genetic variants causing common complex diseases. The discovery of these genetic variants offers an important opportunity for early disease prediction, prevention, and individualized treatment. We describe here a method of combining multiple genetic variants for early disease prediction, based on the optimality theory of the likelihood ratio (LR). Such theory simply shows that the receiver operating characteristic (ROC) curve based on the LR has maximum performance at each cutoff point and that the area under the ROC curve so obtained is highest among that of all approaches. Through simulations and a real data application, we compared it with the commonly used logistic regression and classification tree approaches. The three approaches show similar performance if we know the underlying disease model. However, for most common diseases we have little prior knowledge of the disease model and in this situation the new method has an advantage over logistic regression and classification tree approaches. We applied the new method to the type 1 diabetes GWA data from the Wellcome Trust Case Control Consortium. Based on five single nucleotide polymorphisms, the test reaches medium level classification accuracy. With more genetic findings to be discovered in the future, we believe a predictive genetic test for type 1 diabetes can be successfully constructed and eventually implemented for clinical use.

| S-EPMC3039874 | biostudies-literature

Estimating the receiver operating characteristic curve in matched case control studies.

Project description:The matched case-control design is frequently used in the study of complex disorders and can result in significant gains in efficiency, especially in the context of measuring biomarkers; however, risk prediction in this setting is not straightforward. We propose an inverse-probability weighting approach to estimate the predictive ability associated with a set of covariates. In particular, we propose an algorithm for estimating the summary index, area under the curve corresponding to the Receiver Operating Characteristic curve associated with a set of pre-defined covariates for predicting a binary outcome. By combining data from the parent cohort with that generated in a matched case control study, we describe methods for estimation of the population parameters of interest and the corresponding area under the curve. We evaluate the bias associated with the proposed methods in simulations by considering a range of parameter settings. We illustrate the methods in two data applications: (1) a prospective cohort study of cardiovascular disease in women, the Women's Health Study, and (2) a matched case-control study nested within the Nurses' Health Study aimed at risk prediction of invasive breast cancer.

| S-EPMC6768691 | biostudies-literature

The length of the receiver operating characteristic curve and the two cutoff Youden index within a robust framework for discovery, evaluation, and cutoff estimation in biomarker studies involving improper receiver operating characteristic curves.

Project description:During the early stage of biomarker discovery, high throughput technologies allow for simultaneous input of thousands of biomarkers that attempt to discriminate between healthy and diseased subjects. In such cases, proper ranking of biomarkers is highly important. Common measures, such as the area under the receiver operating characteristic (ROC) curve (AUC), as well as affordable sensitivity and specificity levels, are often taken into consideration. Strictly speaking, such measures are appropriate under a stochastic ordering assumption, which implies, without loss of generality, that higher measurements are more indicative for the disease. Such an assumption is not always plausible and may lead to rejection of extremely useful biomarkers at this early discovery stage. We explore the length of a smooth ROC curve as a measure for biomarker ranking, which is not subject to directionality. We show that the length corresponds to a ϕ divergence, is identical to the corresponding length of the optimal (likelihood ratio) ROC curve, and is an appropriate measure for ranking biomarkers. We explore the relationship between the length measure and the AUC of the optimal ROC curve. We then provide a complete framework for the evaluation of a biomarker in terms of sensitivity and specificity through a proposed ROC analogue for use in improper settings. In the absence of any clinical insight regarding the appropriate cutoffs, we estimate the sensitivity and specificity under a two-cutoff extension of the Youden index and we further take into account the implied costs. We apply our approaches on two biomarker studies that relate to pancreatic and esophageal cancer.

| S-EPMC9976806 | biostudies-literature

Jackknife variance of the partial area under the empirical receiver operating characteristic curve.

Project description:Receiver operating characteristic analysis provides an important methodology for assessing traditional (e.g., imaging technologies and clinical practices) and new (e.g., genomic studies, biomarker development) diagnostic problems. The area under the clinically/practically relevant part of the receiver operating characteristic curve (partial area or partial area under the receiver operating characteristic curve) is an important performance index summarizing diagnostic accuracy at multiple operating points (decision thresholds) that are relevant to actual clinical practice. A robust estimate of the partial area under the receiver operating characteristic curve is provided by the area under the corresponding part of the empirical receiver operating characteristic curve. We derive a closed-form expression for the jackknife variance of the partial area under the empirical receiver operating characteristic curve. Using the derived analytical expression, we investigate the differences between the jackknife variance and a conventional variance estimator. The relative properties in finite samples are demonstrated in a simulation study. The developed formula enables an easy way to estimate the variance of the empirical partial area under the receiver operating characteristic curve, thereby substantially reducing the computation burden, and provides important insight into the structure of the variability. We demonstrate that when compared with the conventional approach, the jackknife variance has substantially smaller bias, and leads to a more appropriate type I error rate of the Wald-type test. The use of the jackknife variance is illustrated in the analysis of a data set from a diagnostic imaging study.

| S-EPMC5133195 | biostudies-literature

RE: "RECEIVER OPERATING CHARACTERISTIC CURVE INFERENCE FROM A SAMPLE WITH A LIMIT OF DETECTION".

Project description: Not available

| S-EPMC5860001 | biostudies-literature

Sojourn-time-corrected receiver operating characteristic curve (ROC) for prostate specific antigen (PSA) test in population-based prostate cancer screening.

Project description:Evaluating the performance of serum prostate-specific antigen (PSA) test in population-based screening with receiver operating characteristics (ROC) curve often neglects the time dimension. Asymptomatic cases with negative PSA test would have been missed if sojourn time is not taken into account to allow for cases surfacing into the clinical phase. Data included 20,796 men with PSA test at the first screening round was used from population-based Finnish prostate cancer screening trial during 1996-1999. Cancers detected at the first screen, together with interval cancers ascertained during 4-year follow-up were expediently used to estimate sensitivity and specificity. A sojourn-time-corrected model was applied to estimating the possible false negative cases for those with PSA < 4 ng/ml for correcting the ROC curve. The estimated sensitivity estimate was reduced from 94.4% without correction to 68.8% with correction but the estimated specificity was identical (89.4% vs. 89.2%) at cutoff of 3 ng/ml. The corrected area under curve (AUC) [77.0% (74.9-79.1%)] of the PSA test was significantly lower than the uncorrected AUC [95.9% (95.3-96.6%)]. The failure of considering the time since last negative screen due to incomplete ascertainment for asymptomatic cancer led to the overestimation of PSA test performance that further affects the cut-off value of PSA tests for population-based prostate cancer screening.

| S-EPMC7691355 | biostudies-literature

Assessment of dysregulated children using the Child Behavior Checklist: a receiver operating characteristic curve analysis.

Project description:Disorders of self-regulatory behavior are common reasons for referral to child and adolescent clinicians. Here, the authors sought to compare 2 methods of empirically based assessment of children with problems in self-regulatory behavior. Using parental reports on 2,028 children (53% boys) from a U.S. national probability sample of the Child Behavior Checklist (CBCL; T. M. Achenbach & L. A. Rescorla, 2001), the receiver operating characteristic curve analysis was applied to compare scores on the Posttraumatic Stress Problems Scale (PTSP) of the CBCL with the CBCL Dysregulation Profile (DP), identified using latent class analysis of the Attention Problems, Aggressive Behavior, and Anxious/Depressed scales of the CBCL. The CBCL-PTSP score demonstrated an area under the curve of between .88 and .91 for predicting membership in the CBCL-DP profile for boys and for girls. These findings suggest that the CBCL-PTSP, which others have shown does not uniquely identify children who have been traumatized, does identify the same profile of behavior as the CBCL-DP. Therefore, the authors recommend renaming the CBCL-PTSP the Dysregulation Short Scale and provide some guidelines for the use of the CBCL-DP scale and the CBCL-PTSP in clinical practice.

| S-EPMC2936710 | biostudies-literature

A unified Bayesian framework for exact inference of area under the receiver operating characteristic curve.

Project description:The area under the receiver operating characteristic curve is a widely used measure for evaluating the performance of a diagnostic test. Common approaches for inference on area under the receiver operating characteristic curve are usually based upon approximation. For example, the normal approximation based inference tends to suffer from the problem of low accuracy for small sample size. Frequentist empirical likelihood based approaches for area under the receiver operating characteristic curve estimation may perform better, but are usually conducted through approximation in order to reduce the computational burden, thus the inference is not exact. By contrast, we proposed an exact inferential procedure by adapting the empirical likelihood into a Bayesian framework and draw inference from the posterior samples of the area under the receiver operating characteristic curve obtained via a Gibbs sampler. The full conditional distributions within the Gibbs sampler only involve empirical likelihoods with linear constraints, which greatly simplify the computation. To further enhance the applicability and flexibility of the Bayesian empirical likelihood, we extend our method to the estimation of partial area under the receiver operating characteristic curve, comparison of multiple tests, and the doubly robust estimation of area under the receiver operating characteristic curve in the presence of missing test results. Simulation studies confirm the desirable performance of the proposed methods, and a real application is presented to illustrate its usefulness.

| S-EPMC9241147 | biostudies-literature

Identifying At-Risk College Student Drinkers With the AUDIT-US: A Receiver Operating Characteristic Curve Analysis.

Project description:The Alcohol Use Disorders Identification Test (AUDIT) is the gold standard screening measure. Recently, there has been increasing call to update the measure to reflect harmful drinking standards in the United States. The purpose of this study was to use receiver operating characteristic curve analysis to evaluate the AUDIT and the United States version (AUDIT-US). Participants were 382 traditional age (M = 20.2, SD = 1.5) college students (68.7% female, 64.9% White) who had consumed alcohol at least once in the 30 days prior to participating. Although results provide evidence for the AUDIT and the AUDIT-US as valid screening tools, the Consumption subscale of the AUDIT-US performed the best in predicting at-risk college drinkers. The Consumption subscale of the AUDIT-US with a single cutoff score of four appears to be the optimal and most parsimonious method of identifying at-risk college drinkers.

| S-EPMC7163156 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data