Dataset Information

Evaluation of an optimal receiver operating characteristic procedure.

ABSTRACT: UNLABELLED:: Lu and Elston have recently proposed a procedure for developing optimal receiver operating characteristic curves that maximize the area under a receiver operating characteristic curve in the setting of a predictive genetic test. The method requires only summary data, not individual level genetic data. In an era of increased data sharing, we investigate the performance of this algorithm when individual level genetic data are available and compare this approach to more standard receiver operating characteristic curve-building methods. CONCLUSION:Though the Lu-Elston method can produce an optimal area under the curve under some assumptions, the method typically has little advantage over standard multivariable logistic methods when data are available. Also, the standard approach easily allows comparison of nested models via likelihood ratio tests and incorporation of covariates - the Lu-Elston approach is shown to have some difficulties with such analyses. These conclusions are based on evaluations using the Genetic Analysis Workshop 16 rheumatoid arthritis data set.

SUBMITTER: Jeffries N

PROVIDER: S-EPMC2795956 | biostudies-literature | 2009 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluation of an optimal receiver operating characteristic procedure.

Jeffries Neal N Zheng Gang G

BMC proceedings 20091215

<h4>Unlabelled</h4>: Lu and Elston have recently proposed a procedure for developing optimal receiver operating characteristic curves that maximize the area under a receiver operating characteristic curve in the setting of a predictive genetic test. The method requires only summary data, not individual level genetic data. In an era of increased data sharing, we investigate the performance of this algorithm when individual level genetic data are available and compare this approach to more standar ...[more]

PMID: 20018049

Similar Datasets

Project description:BackgroundDisease prevalence is rarely explicitly considered in the early stages of the development of novel prognostic tests. Rather, researchers use the area under the receiver operating characteristic (AUROC) as the key metric to gauge and report predictive performance ability. Because this statistic does not account for disease prevalence, proposed tests may not appropriately address clinical requirements. This ultimately impedes the translation of prognostic tests into clinical practice.MethodsA method to express positive- and/or negative predictive value criteria (PPV, NPV) within the ROC space is presented. Equations are derived for so-called equi-PPV (and equi-NPV) lines. Herewith it is possible, for any given prevalence, to plot a series of sensitivity-specificity pairs which meet a specified PPV (or NPV) criterion onto the ROC space.This concept is introduced by firstly reviewing the well-established "mechanics", strengths and limitations of the ROC analysis in the context of developing prognostic models. Then, the use of PPV (and/or) NPV criteria to augment the ROC analysis is elaborated.Additionally, an interactive web tool was also created to enable people to explore the dynamics of lines of equi-predictive value in function of prevalence. The web tool also allows to gauge what ROC curve shapes best meet specific positive and/or negative predictive value criteria (http://d4ta.link/ppvnpv/).ResultsTo illustrate the merits and implications of this concept, an example on the prediction of pre-eclampsia risk in low-risk nulliparous pregnancies is elaborated.ConclusionsIn risk stratification, the clinical usefulness of a prognostic test can be expressed in positive- and negative predictive value criteria; the development of novel prognostic tests will be facilitated by the possibility to co-visualise such criteria together with ROC curves. To achieve clinically meaningful risk stratification, the development of separate tests to meet either a pre-specified positive value (rule-in) or a negative predictive value (rule-out) criteria should be considered: the characteristics of successful rule-in and rule-out tests may markedly differ.

Dataset Information

Evaluation of an optimal receiver operating characteristic procedure.

Publications

Evaluation of an optimal receiver operating characteristic procedure.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets