Dataset Information

Two criteria for evaluating risk prediction models.

ABSTRACT: We propose and study two criteria to assess the usefulness of models that predict risk of disease incidence for screening and prevention, or the usefulness of prognostic models for management following disease diagnosis. The first criterion, the proportion of cases followed?PCF?(q), is the proportion of individuals who will develop disease who are included in the proportion?q?of individuals in the population at highest risk. The second criterion is the proportion needed to follow-up,?PNF?(p), namely the proportion of the general population at highest risk that one needs to follow in order that a proportion?p?of those destined to become cases will be followed.?PCF?(q)?assesses the effectiveness of a program that follows?100q% of the population at highest risk.?PNF?(p)?assess the feasibility of covering?100p% of cases by indicating how much of the population at highest risk must be followed. We show the relationship of those two criteria to the Lorenz curve and its inverse, and present distribution theory for estimates of?PCF?and?PNF. We develop new methods, based on influence functions, for inference for a single risk model, and also for comparing the?PCFs and?PNFs of two risk models, both of which were evaluated in the same validation data.

SUBMITTER: Pfeiffer RM

PROVIDER: S-EPMC3135785 | biostudies-literature | 2011 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Two criteria for evaluating risk prediction models.

Pfeiffer R M RM Gail M H MH

Biometrics 20101214 3

We propose and study two criteria to assess the usefulness of models that predict risk of disease incidence for screening and prevention, or the usefulness of prognostic models for management following disease diagnosis. The first criterion, the proportion of cases followed PCF (q), is the proportion of individuals who will develop disease who are included in the proportion q of individuals in the population at highest risk. The second criterion is the proportion needed to follow-up, PNF (p), na ...[more]

PMID: 21155746

Similar Datasets

Project description:BACKGROUND:The ability to predict risk allows healthcare providers to propose which patients might benefit most from certain therapies, and is relevant to payers' demands to justify clinical and economic value. To understand the robustness of risk prediction models for heart failure (HF), we conducted a systematic literature review to (1) identify HF risk-prediction models, (2) assess statistical approach and extent of validation, (3) identify common variables, and (4) assess risk of bias (ROB). METHODS:Literature databases were searched from March 2013 to May 2018 to identify risk prediction models conducted in an out-of-hospital setting in adults with HF. Distinct risk prediction variables were ranked according to outcomes assessed and incorporation into the studies. ROB was assessed using Prediction model Risk Of Bias ASsessment Tool (PROBAST). RESULTS:Of 4720 non-duplicated citations, 40 risk-prediction publications were deemed relevant. Within the 40 publications, 58 models assessed 55 (co)primary outcomes, including all-cause mortality (n = 17), cardiovascular death (n = 9), HF hospitalizations (n = 15), and composite endpoints (n = 14). Few publications reported detail on handling missing data (n = 11; 28%). The discriminatory ability for predicting all-cause mortality, cardiovascular death, and composite endpoints was generally better than for HF hospitalization. 105 distinct predictor variables were identified. Predictors included in >5 publications were: N-terminal prohormone brain-natriuretic peptide, creatinine, blood urea nitrogen, systolic blood pressure, sodium, NYHA class, left ventricular ejection fraction, heart rate, and characteristics including male sex, diabetes, age, and BMI. Only 11/58 (19%) models had overall low ROB, based on our application of PROBAST. In total, 26/58 (45%) models discussed internal validation, and 14/58 (24%) external validation. CONCLUSIONS:The majority of the 58 identified risk-prediction models for HF present particular concerns according to ROB assessment, mainly due to lack of validation and calibration. The potential utility of novel approaches such as machine learning tools is yet to be determined. REGISTRATION NUMBER:The SLR was registered in Prospero (ID: CRD42018100709).

Dataset Information

Two criteria for evaluating risk prediction models.

Publications

Two criteria for evaluating risk prediction models.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets