Dataset Information

A Novel Method for Identifying a Parsimonious and Accurate Predictive Model for Multiple Clinical Outcomes.

ABSTRACT:

Background and objective

Most methods for developing clinical prognostic models focus on identifying parsimonious and accurate models to predict a single outcome; however, patients and providers often want to predict multiple outcomes simultaneously. As an example, for older adults one is often interested in predicting nursing home admission as well as mortality. We propose and evaluate a novel predictor-selection computing method for multiple outcomes and provide the code for its implementation.

Methods

Our proposed algorithm selected the best subset of common predictors based on the minimum average normalized Bayesian Information Criterion (BIC) across outcomes: the Best Average BIC (baBIC) method. We compared the predictive accuracy (Harrell's C-statistic) and parsimony (number of predictors) of the model obtained using the baBIC method with: 1) a subset of common predictors obtained from the union of optimal models for each outcome (Union method), 2) a subset obtained from the intersection of optimal models for each outcome (Intersection method), and 3) a model with no variable selection (Full method). We used a case-study data from the Health and Retirement Study (HRS) to demonstrate our method and conducted a simulation study to investigate performance.

Results

In the case-study data and simulations, the average Harrell's C-statistics across outcomes of the models obtained with the baBIC and Union methods were comparable. Despite the similar discrimination, the baBIC method produced more parsimonious models than the Union method. In contrast, the models selected with the Intersection method were the most parsimonious, but with worst predictive accuracy, and the opposite was true in the Full method. In the simulations, the baBIC method performed well by identifying many of the predictors selected in the baBIC model of the case-study data most of the time and excluding those not selected in the majority of the simulations.

Conclusions

Our method identified a common subset of variables to predict multiple clinical outcomes with superior balance between parsimony and predictive accuracy to current methods.

SUBMITTER: Diaz-Ramirez LG

PROVIDER: S-EPMC8098121 | biostudies-literature | 2021 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Novel Method for Identifying a Parsimonious and Accurate Predictive Model for Multiple Clinical Outcomes.

Diaz-Ramirez L Grisell LG Lee Sei J SJ Smith Alexander K AK Gan Siqi S Boscardin W John WJ

Computer methods and programs in biomedicine 20210327

<h4>Background and objective</h4>Most methods for developing clinical prognostic models focus on identifying parsimonious and accurate models to predict a single outcome; however, patients and providers often want to predict multiple outcomes simultaneously. As an example, for older adults one is often interested in predicting nursing home admission as well as mortality. We propose and evaluate a novel predictor-selection computing method for multiple outcomes and provide the code for its implem ...[more]

PMID: 33831724

Similar Datasets

Project description:The COVID-19 pandemic is impressively challenging the healthcare system. Several prognostic models have been validated but few of them are implemented in daily practice. The objective of the study was to validate a machine-learning risk prediction model using easy-to-obtain parameters to help to identify patients with COVID-19 who are at higher risk of death. The training cohort included all patients admitted to Fondazione Policlinico Gemelli with COVID-19 from March 5, 2020, to November 5, 2020. Afterward, the model was tested on all patients admitted to the same hospital with COVID-19 from November 6, 2020, to February 5, 2021. The primary outcome was in-hospital case-fatality risk. The out-of-sample performance of the model was estimated from the training set in terms of Area under the Receiving Operator Curve (AUROC) and classification matrix statistics by averaging the results of fivefold cross validation repeated 3-times and comparing the results with those obtained on the test set. An explanation analysis of the model, based on the SHapley Additive exPlanations (SHAP), is also presented. To assess the subsequent time evolution, the change in paO2/FiO2 (P/F) at 48 h after the baseline measurement was plotted against its baseline value. Among the 921 patients included in the training cohort, 120 died (13%). Variables selected for the model were age, platelet count, SpO2, blood urea nitrogen (BUN), hemoglobin, C-reactive protein, neutrophil count, and sodium. The results of the fivefold cross-validation repeated 3-times gave AUROC of 0.87, and statistics of the classification matrix to the Youden index as follows: sensitivity 0.840, specificity 0.774, negative predictive value 0.971. Then, the model was tested on a new population (n = 1463) in which the case-fatality rate was 22.6%. The test model showed AUROC 0.818, sensitivity 0.813, specificity 0.650, negative predictive value 0.922. Considering the first quartile of the predicted risk score (low-risk score group), the case-fatality rate was 1.6%, 17.8% in the second and third quartile (high-risk score group) and 53.5% in the fourth quartile (very high-risk score group). The three risk score groups showed good discrimination for the P/F value at admission, and a positive correlation was found for the low-risk class to P/F at 48 h after admission (adjusted R-squared = 0.48). We developed a predictive model of death for people with SARS-CoV-2 infection by including only easy-to-obtain variables (abnormal blood count, BUN, C-reactive protein, sodium and lower SpO2). It demonstrated good accuracy and high power of discrimination. The simplicity of the model makes the risk prediction applicable for patients in the Emergency Department, or during hospitalization. Although it is reasonable to assume that the model is also applicable in not-hospitalized persons, only appropriate studies can assess the accuracy of the model also for persons at home.

Project description:BackgroundThe application of sepsis subtypes to enhance personalized medicine in critically ill patients is hindered by the lack of validation across diverse cohorts and the absence of a simple classification model. We aimed to validate the previously identified SENECA clinical sepsis subtypes in multiple large ICU cohorts, and to develop parsimonious classifier models for δ-type adjudication in clinical practice.MethodsData from four cohorts between 2008 and 2023 were used to assign α, β, γ and δ-type in patients fulfilling the Sepsis-3 criteria using clinical variables: (I) The Molecular diAgnosis and Risk stratification of Sepsis (MARS, n = 2449), (II) a contemporary continuation of the MARS study (MARS2, n = 2445) (III) the Dutch National Intensive Care Evaluation registry (NICE, n = 28,621) and (IV) the Medical Information Mart for Intensive Care including (MIMIC-IV, n = 18,661). K-means clustering using clinical variables was conducted to assess the optimal number of classes and compared to the SENECA subtypes. Parsimonious models were built in the SENECA derivation cohort to predict subtype membership using logistic regression, and validated in MARS and MIMIC-IV.ResultsAmong 52.226 patients with sepsis, the subtype distribution in MARS, MARS2 and NICE was 2-6% for the α-type, 1-5% for the β-type, 49-65% for the γ-type and 26-48% for the δ-type compared to 33%, 27%, 27% and 13% in the original SENECA derivation cohort, whereas subtype distribution in MIMIC-IV was more similar at 25%, 24%, 27% and 25%, respectively. In-hospital mortality rates were significantly different between the four cohorts for α, γ and δ-type (p < 0.001). Method-based validation showed moderate overlap with the original subtypes in both MARS and MIMIC-IV. A parsimonious model for all four subtypes had moderate to low accuracy (accuracy 62.2%), while a parsimonious classifier model with 3 variables (aspartate aminotransferase, serum lactate, and bicarbonate) had excellent accuracy in predicting the δ-type patients from all other types in the derivation cohort and moderate accuracy in the validation cohorts (MARS: area under the receiver operator characteristic curve (AUC) 0.93, 95% CI [0.92-0.94], accuracy 85.5% [84.0-86.8%]; MIMIC-IV: AUC 0.86 [0.85-0.87], accuracy 82.9% [82.4-83.4%]).ConclusionsThe distribution and mortality rates of clinical sepsis subtypes varied between US and European cohorts. A three-variable model could accurately identify the δ-type sepsis patients.

Dataset Information

A Novel Method for Identifying a Parsimonious and Accurate Predictive Model for Multiple Clinical Outcomes.

Background and objective

Methods

Results

Conclusions

Publications

A Novel Method for Identifying a Parsimonious and Accurate Predictive Model for Multiple Clinical Outcomes.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets