Dataset Information

Quantifying risk stratification provided by diagnostic tests and risk predictions: Comparison to AUC and decision curve analysis.

ABSTRACT: A property of diagnostic tests and risk models deserving more attention is risk stratification, defined as the ability of a test or model to separate those at high absolute risk of disease from those at low absolute risk. Risk stratification fills a gap between measures of classification (ie, area under the curve (AUC)) that do not require absolute risks and decision analysis that requires not only absolute risks but also subjective specification of costs and utilities. We introduce mean risk stratification (MRS) as the average change in risk of disease (posttest-pretest) revealed by a diagnostic test or risk model dichotomized at a risk threshold. Mean risk stratification is particularly valuable for rare conditions, where AUC can be high but MRS can be low, identifying situations that temper overenthusiasm for screening with the new test/model. We apply MRS to the controversy over who should get testing for mutations in BRCA1/2 that cause high risks of breast and ovarian cancers. To reveal different properties of risk thresholds to refer women for BRCA1/2 testing, we propose an eclectic approach considering MRS and other metrics. The value of MRS is to interpret AUC in the context of BRCA1/2 mutation prevalence, providing a range of risk thresholds at which a risk model is "optimally informative," and to provide insight into why net benefit arrives to its conclusion.

SUBMITTER: Katki HA

PROVIDER: S-EPMC6827980 | biostudies-literature | 2019 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Quantifying risk stratification provided by diagnostic tests and risk predictions: Comparison to AUC and decision curve analysis.

Katki Hormuzd A HA

Statistics in medicine 20190430 16

A property of diagnostic tests and risk models deserving more attention is risk stratification, defined as the ability of a test or model to separate those at high absolute risk of disease from those at low absolute risk. Risk stratification fills a gap between measures of classification (ie, area under the curve (AUC)) that do not require absolute risks and decision analysis that requires not only absolute risks but also subjective specification of costs and utilities. We introduce mean risk st ...[more]

PMID: 31037749

Similar Datasets

Project description:BackgroundNon-alcoholic fatty liver disease (NAFLD) is the commonest liver condition in the western world and is directly linked to obesity and the metabolic syndrome. Elevated body mass index is regarded as a major risk factor of NAFL (steatosis) and NAFLD fibrosis. Using data from the Avon Longitudinal Study of Parents and Children (ALSPAC), we sought to investigate whether other variables from adolescence could improve prediction of future NAFL and NAFLD fibrosis risk at 24 years, above BMI and sex.MethodsAged 24 years, 4018 ALSPAC participants had transient elastography (TE) and controlled attenuation parameter (CAP) measurement using Echosens 502 Touch. 513 participants with harmful alcohol consumption were excluded. Logistic regression models examined which variables measured at 17 years were predictive of NAFL and NAFLD fibrosis in young adults. Predictors included sex, BMI, central adiposity, lipid profile, blood pressure, liver function tests, homeostatic model assessment for insulin resistance (HOMA-IR), and ultrasound defined NAFL at 17 years (when examining fibrosis outcomes). A model including all these variables was termed "routine clinical measures". Models were compared using area under the receiver operator curve (AUROC) and Bayesian Information Criterion (BIC), analysis, which penalises model complexity. Models were tested in all participants and those with overweight or obese standardised BMIs (BMI SDS) centiles at the 17-year time point. A decision curve analysis (DCA) was performed to evaluate the clinical utility of models in overweight and obese adolescents predicting NAFLD fibrosis at a threshold probability of 0.1.ResultsThe "routine clinical measures" model had the highest AUROC for predicting NAFL in all adolescent participants (AUROC 0.79 [SD 0.00]) and those with an overweight/obese BMI SDS centile (AUROC 0.77 [SD 0.01]). According to BIC analysis, insulin resistance was the best predictor of NAFL in all adolescents, whilst central adiposity was the best predictor in those with an overweight/obese BMI SDS centile. The "routine clinical measures" model also had the highest AUROC for predicting NAFLD fibrosis in all adolescent participants (AUROC 0.78 [SD 0.02]) and participants with an overweight/obese BMI SDS centile (AUROC 0.84 [SD 0.03]). However, following BIC analysis, BMI was the best predictor of NAFLD fibrosis in all adolescents including those with an overweight/obese BMI SDS centile. A decision curve analysis examining overweight/obese adolescent participants showed the model that had the greatest net benefit for increased NAFLD fibrosis detection, above a treat all overweight and obese adolescents' assumption, was the "routine clinical measures" model. However, the net benefit was marginal (0.0054 [0.0034-0.0075]).ConclusionIn adolescents, routine clinical measures were not superior to central adiposity and BMI at predicting NAFL and NAFLD fibrosis respectively in young adulthood. Additional routine clinical measurements do provide incremental benefit in detecting true positive fibrosis cases, but the benefit is small. Thus, to reduce morbidity and mortality associated with NASH cirrhosis in adults, the ultimate end point of NAFLD, the focus must be on obesity management at a population level.

Dataset Information

Quantifying risk stratification provided by diagnostic tests and risk predictions: Comparison to AUC and decision curve analysis.

Publications

Quantifying risk stratification provided by diagnostic tests and risk predictions: Comparison to AUC and decision curve analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets