Comparison of linear and non-linear machine learning models for time-dependent readmission or mortality prediction among hospitalized heart failure patients.
Ontology highlight
ABSTRACT:
SUBMITTER: Tong R
PROVIDER: S-EPMC10192765 | biostudies-literature | 2023 May
REPOSITORIES: biostudies-literature
Publications
Comparison of linear and non-linear machine learning models for time-dependent readmission or mortality prediction among hospitalized heart failure patients.
Tong Rui R Zhu Zhongsheng Z Ling Jia J
Heliyon 20230506 5
Although many models are available to predict prognosis of heart failure patients, most tools combining survival analysis are based on proportional hazard model. Non-linear machine learning algorithms would overcome the limitation of the time-independent hazard ratio assumption and provide more information in readmission or mortality prediction among heart failure patients. The present study collected the clinical information of 1796 hospitalized heart failure patients surviving during hospitali ...[more]
Similar Datasets
Project description:This study intends to predict in-hospital and 6-month mortality, as well as 30-day and 90-day hospital readmission, using Machine Learning (ML) approach via conventional features. A total of 737 patients remained after applying the exclusion criteria to 1101 heart failure patients. Thirty-four conventional features were collected for each patient. First, the data were divided into train and test cohorts with a 70-30% ratio. Then train data were normalized using the Z-score method, and its mean and standard deviation were applied to the test data. Subsequently, Boruta, RFE, and MRMR feature selection methods were utilized to select more important features in the training set. In the next step, eight ML approaches were used for modeling. Next, hyperparameters were optimized using tenfold cross-validation and grid search in the train dataset. All model development steps (normalization, feature selection, and hyperparameter optimization) were performed on a train set without touching the hold-out test set. Then, bootstrapping was done 1000 times on the hold-out test data. Finally, the obtained results were evaluated using four metrics: area under the ROC curve (AUC), accuracy (ACC), specificity (SPE), and sensitivity (SEN). The RFE-LR (AUC: 0.91, ACC: 0.84, SPE: 0.84, SEN: 0.83) and Boruta-LR (AUC: 0.90, ACC: 0.85, SPE: 0.85, SEN: 0.83) models generated the best results in terms of in-hospital mortality. In terms of 30-day rehospitalization, Boruta-SVM (AUC: 0.73, ACC: 0.81, SPE: 0.85, SEN: 0.50) and MRMR-LR (AUC: 0.71, ACC: 0.68, SPE: 0.69, SEN: 0.63) models performed the best. The best model for 3-month rehospitalization was MRMR-KNN (AUC: 0.60, ACC: 0.63, SPE: 0.66, SEN: 0.53) and regarding 6-month mortality, the MRMR-LR (AUC: 0.61, ACC: 0.63, SPE: 0.44, SEN: 0.66) and MRMR-NB (AUC: 0.59, ACC: 0.61, SPE: 0.48, SEN: 0.63) models outperformed the others. Reliable models were developed in 30-day rehospitalization and in-hospital mortality using conventional features and ML techniques. Such models can effectively personalize treatment, decision-making, and wiser budget allocation. Obtained results in 3-month rehospitalization and 6-month mortality endpoints were not astonishing and further experiments with additional information are needed to fetch promising results in these endpoints.
| S-EPMC10618467 | biostudies-literature
Project description:AimsThis study aimed to review the performance of machine learning (ML) methods compared with conventional statistical models (CSMs) for predicting readmission and mortality in patients with heart failure (HF) and to present an approach to formally evaluate the quality of studies using ML algorithms for prediction modelling.Methods and resultsFollowing Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we performed a systematic literature search using MEDLINE, EPUB, Cochrane CENTRAL, EMBASE, INSPEC, ACM Library, and Web of Science. Eligible studies included primary research articles published between January 2000 and July 2020 comparing ML and CSMs in mortality and readmission prognosis of initially hospitalized HF patients. Data were extracted and analysed by two independent reviewers. A modified CHARMS checklist was developed in consultation with ML and biostatistics experts for quality assessment and was utilized to evaluate studies for risk of bias. Of 4322 articles identified and screened by two independent reviewers, 172 were deemed eligible for a full-text review. The final set comprised 20 articles and 686 842 patients. ML methods included random forests (n = 11), decision trees (n = 5), regression trees (n = 3), support vector machines (n = 9), neural networks (n = 12), and Bayesian techniques (n = 3). CSMs included logistic regression (n = 16), Cox regression (n = 3), or Poisson regression (n = 3). In 15 studies, readmission was examined at multiple time points ranging from 30 to 180 day readmission, with the majority of studies (n = 12) presenting prediction models for 30 day readmission outcomes. Of a total of 21 time-point comparisons, ML-derived c-indices were higher than CSM-derived c-indices in 16 of the 21 comparisons. In seven studies, mortality was examined at 9 time points ranging from in-hospital mortality to 1 year survival; of these nine, seven reported higher c-indices using ML. Two of these seven studies reported survival analyses utilizing random survival forests in their ML prediction models. Both reported higher c-indices when using ML compared with CSMs. A limitation of studies using ML techniques was that the majority were not externally validated, and calibration was rarely assessed. In the only study that was externally validated in a separate dataset, ML was superior to CSMs (c-indices 0.913 vs. 0.835).ConclusionsML algorithms had better discrimination than CSMs in most studies aiming to predict risk of readmission and mortality in HF patients. Based on our review, there is a need for external validation of ML-based studies of prediction modelling. We suggest that ML-based studies should also be evaluated using clinical quality standards for prognosis research. Registration: PROSPERO CRD42020134867.
| S-EPMC7835549 | biostudies-literature
Project description:Heart failure (HF) inpatient mortality prediction models can help clinicians make treatment decisions and researchers conduct observational studies; however, published models have not been validated in external populations. We compared the performance of 7 models that predict inpatient mortality in patients hospitalized with acute decompensated heart failure: 4 HF-specific mortality prediction models developed from 3 clinical databases (ADHERE [Acute Decompensated Heart Failure National Registry], EFFECT study [Enhanced Feedback for Effective Cardiac Treatment], and GWTG-HF registry [Get With the Guidelines-Heart Failure]); 2 administrative HF mortality prediction models (Premier, Premier+); and a model that uses clinical data but is not specific for HF (Laboratory-Based Acute Physiology Score [LAPS2]). Using a multihospital, electronic health record-derived data set (HealthFacts [Cerner Corp], 2010-2012), we identified patients ≥18 years admitted with HF. Of 13 163 eligible patients, median age was 74 years; half were women; and 27% were black. In-hospital mortality was 4.3%. Model-predicted mortality ranges varied: Premier+ (0.8%-23.1%), LAPS2 (0.7%-19.0%), ADHERE (1.2%-17.4%), EFFECT (1.0%-12.8%), GWTG-Eapen (1.2%-13.8%), and GWTG-Peterson (1.1%-12.8%). The LAPS2 and Premier models outperformed the clinical models (C statistics: LAPS2 0.80 [95% confidence interval 0.78-0.82], Premier models 0.81 [95% confidence interval 0.79-0.83] and 0.76 [95% confidence interval 0.74-0.78], and clinical models 0.68 to 0.70). Four clinically derived, inpatient, HF mortality models exhibited similar performance, with C statistics near 0.70. Three other models, 1 developed in electronic health record data and 2 developed in administrative data, also were predictive, with C statistics from 0.76 to 0.80. Because every model performed acceptably, the decision to use a given model should depend on practical concerns and intended use.
| S-EPMC4988343 | biostudies-literature
Project description:Heart failure is a devastating disease that has high mortality rates and a negative impact on quality of life. Heart failure patients often experience emergency readmission after an initial episode, often due to inadequate management. A timely diagnosis and treatment of underlying issues can significantly reduce the risk of emergency readmissions. The purpose of this project was to predict emergency readmissions of discharged heart failure patients using classical machine learning (ML) models based on Electronic Health Record (EHR) data. The dataset used for this study consisted of 166 clinical biomarkers from 2008 patient records. Three feature selection techniques were studied along with 13 classical ML models using five-fold cross-validation. A stacking ML model was trained using the predictions of the three best-performing models for final classification. The stacking ML model provided an accuracy, precision, recall, specificity, F1-score, and area under the curve (AUC) of 89.41%, 90.10%, 89.41%, 87.83%, 89.28%, and 0.881, respectively. This indicates the effectiveness of the proposed model in predicting emergency readmissions. The healthcare providers can intervene pro-actively to reduce emergency hospital readmission risk and improve patient outcomes and decrease healthcare costs using the proposed model.
| S-EPMC10252957 | biostudies-literature
Project description:BackgroundSensitive and interpretable machine learning (ML) models can provide valuable assistance to clinicians in managing patients with heart failure (HF) at discharge by identifying individual factors associated with a high risk of readmission. In this cohort study, we delve into the factors driving the potential utility of classification models as decision support tools for predicting readmissions in patients with HF.ObjectiveThe primary objective of this study is to assess the trade-off between using deep learning (DL) and traditional ML models to identify the risk of 100-day readmissions in patients with HF. Additionally, the study aims to provide explanations for the model predictions by highlighting important features both on a global scale across the patient cohort and on a local level for individual patients.MethodsThe retrospective data for this study were obtained from the Regional Health Care Information Platform in Region Halland, Sweden. The study cohort consisted of patients diagnosed with HF who were over 40 years old and had been hospitalized at least once between 2017 and 2019. Data analysis encompassed the period from January 1, 2017, to December 31, 2019. Two ML models were developed and validated to predict 100-day readmissions, with a focus on the explainability of the model's decisions. These models were built based on decision trees and recurrent neural architecture. Model explainability was obtained using an ML explainer. The predictive performance of these models was compared against 2 risk assessment tools using multiple performance metrics.ResultsThe retrospective data set included a total of 15,612 admissions, and within these admissions, readmission occurred in 5597 cases, representing a readmission rate of 35.85%. It is noteworthy that a traditional and explainable model, informed by clinical knowledge, exhibited performance comparable to the DL model and surpassed conventional scoring methods in predicting readmission among patients with HF. The evaluation of predictive model performance was based on commonly used metrics, with an area under the precision-recall curve of 66% for the deep model and 68% for the traditional model on the holdout data set. Importantly, the explanations provided by the traditional model offer actionable insights that have the potential to enhance care planning.ConclusionsThis study found that a widely used deep prediction model did not outperform an explainable ML model when predicting readmissions among patients with HF. The results suggest that model transparency does not necessarily compromise performance, which could facilitate the clinical adoption of such models.
| S-EPMC10638630 | biostudies-literature
Project description:ObjectiveTo compare machine learning methods for predicting inpatient seizures risk and determine the feasibility of 1-h screening EEG to identify low-risk patients (<5% seizures risk in 48 h).MethodsThe Critical Care EEG Monitoring Research Consortium (CCEMRC) multicenter database contains 7716 continuous EEGs (cEEG). Neural networks (NN), elastic net logistic regression (EN), and sparse linear integer model (RiskSLIM) were trained to predict seizures. RiskSLIM was used previously to generate 2HELPS2B model of seizure predictions. Data were divided into training (60% for model fitting) and evaluation (40% for model evaluation) cohorts. Performance was measured using area under the receiver operating curve (AUC), mean risk calibration (CAL), and negative predictive value (NPV). A secondary analysis was performed using Monte Carlo simulation (MCS) to normalize all EEG recordings to 48 h and use only the first hour of EEG as a "screening EEG" to generate predictions.ResultsRiskSLIM recreated the 2HELPS2B model. All models had comparable AUC: evaluation cohort (NN: 0.85, EN: 0.84, 2HELPS2B: 0.83) and MCS (NN: 0.82, EN; 0.82, 2HELPS2B: 0.81) and NPV (absence of seizures in the group that the models predicted to be low risk): evaluation cohort (NN: 97%, EN: 97%, 2HELPS2B: 97%) and MCS (NN: 97%, EN: 99%, 2HELPS2B: 97%). 2HELPS2B model was able to identify the largest proportion of low-risk patients.InterpretationFor seizure risk stratification of hospitalized patients, the RiskSLIM generated 2HELPS2B model compares favorably to the complex NN and EN generated models. 2HELPS2B is able to accurately and quickly identify low-risk patients with only a 1-h screening EEG.
| S-EPMC6649418 | biostudies-literature
Project description:AimsRisk prediction indices used in worsening heart failure (HF) vary in complexity, performance, and the type of datasets in which they were validated. We compared the performance of seven risk prediction indices in a contemporary cohort of patients hospitalized for HF.Methods and resultsWe assessed the performance of the Length of stay and number of Emergency department visits in the prior 6 months (LE), Length of stay, number of Emergency department visits in the prior 6 months, and admission N-Terminal prohormone of brain natriuretic peptide (NT-proBNP (LENT), Length of stay, Acuity, Charlson co-morbidity index, and number of Emergency department visits in the prior 6 months (LACE), Get With The Guidelines Heart Failure (GWTG), Readmission Risk Score (RRS), Enhanced Feedback for Effective Cardiac Treatment model (EFFECT), and Acute Decompensated Heart Failure National Registry (ADHERE) risk indices among consecutive patients hospitalized for HF and discharged alive from January 2017 to December 2019 in a network of hospitals in England. The primary composite outcome was 30-day all-cause mortality or readmission. We assessed model discrimination and overall accuracy using the C-statistic (higher values, better) and Brier score (lower values, better), respectively. Among 1206 patients in the cohort, 45.0% were female, mean (SD) age was 76.6 (11.7) years, and mean (SD) left ventricular ejection fraction was 43.0% (11.6). At 30 days, 236 (19.6%) patients were readmitted and 28 (2.3%) patients died, with 264 (21.9%) patients experiencing either readmission or death. The LENT index offered the combination of greatest risk discrimination and accuracy for the primary composite outcome (C-statistic: 0.97; 95% CI 0.96, 0.98; 0.29; Brier score: 0.05). The LE (C-statistic: 0.95; 95% CI 0.93, 0.96; Brier score: 0.06) and LACE (C-statistic: 0.90; 95% CI 0.88, 0.92; Brier score 0.09) indices had high discrimination and accuracy. Discrimination and accuracy were modest with the RRS (C-statistic: 0.65; 95% CI 0.61, 0.69; Brier score: 0.16) and EFFECT (C-statistic: 0.64; 95% CI 0.60, 0.67; Brier score: 0.16) score; and poor with the GWTG-HF (C-statistic: 0.62; 95% CI 0.58, 0.66; Brier score: 0.17) and ADHERE (C-statistic: 0.54; 95% CI 0.50, 0.57; Brier score: 0.17) scores.ConclusionsIn a study that compared the performance of seven risk prediction indices in a contemporary cohort of patients hospitalized for HF, the simple LENT index offered the greatest combination of discrimination and accuracy for the primary composite outcome of 30-day all-cause mortality or readmission. This three-variable index -using length of hospital stay, preceding emergency department visits and admission NT-proBNP level- is a practical and reliable way to assess prognosis following hospitalization for HF.
| S-EPMC11911581 | biostudies-literature
Project description:The beta blockers carvedilol, bisoprolol, and sustained-release metoprolol succinate reduce readmissions and mortality among patients with heart failure with reduced ejection fraction (HFrEF), based upon clinical trial and registry studies. Results from these studies may not generalize to the typical patient with HFrEF. We conducted a retrospective cohort study of beneficiaries in the Medicare 5% sample hospitalized for HFrEF between 2007 and 2013 and were discharged alive. We compared the 30-day and 365-day heart failure (HF) readmission, all-cause readmission, and mortality rates between beneficiaries who filled a prescription for an evidence-based beta blocker and those who did not after being hospitalized for HFrEF. Out of 12,127 beneficiaries hospitalized for HFrEF, 20% were readmitted for HF, 62% were readmitted for any cause, and 27% died within 365 days. In competing risk models adjusted for demographics, healthcare utilization, and comorbidities, beta blocker use was associated with a lower risk of HF readmission between 8-365 days post discharge (hazard ratio 0.79 [95% confidence interval 0.76, 0.82]), but was not significantly associated with all-cause readmission (1.02 [0.97-1.07]). In Cox models adjusted for the same covariates, beta blocker use was associated with lower mortality 8-365 days post discharge (0.65 [0.60-0.71]). Results were similar when follow up was truncated at 30 days post discharge. Increasing the use of beta blockers following HFrEF hospitalization may not decrease all-cause readmissions among Medicare beneficiaries, but may reduce HF-specific readmissions and mortality.
| S-EPMC7347167 | biostudies-literature
Project description:BackgroundExisting risk assessment tools for heart failure (HF) outcomes use structured databases with static, single-timepoint clinical data and have limited accuracy.ObjectiveThe purpose of this study was to develop a comprehensive approach for accurate prediction of 30-day unplanned readmission and all-cause mortality (ACM) that integrates clinical and physiological data available in the electronic health record system.MethodsThree predictive models for 30-day unplanned readmissions or ACM were created using an extreme gradient boosting approach: (1) index admission model; (2) index discharge model; and (3) feature-aggregated model. Performance was assessed by the area under the curve (AUC) metric and compared with that of the HOSPITAL score, a widely used predictive model for hospital readmission.ResultsA total of 3774 patients with a primary billing diagnosis of HF were included (614 experienced the primary outcome), with 796 variables used in the admission and discharge models, and 2032 in the feature-aggregated model. The index admission model had AUC = 0.723, the index discharge model had AUC = 0.754, and the feature-aggregated model had AUC = 0.756 for prediction of 30-day unplanned readmission or ACM. For comparison, the HOSPITAL score had AUC = 0.666 (admission model: P = .093; discharge model: P = .022; feature aggregated: P = .012).ConclusionThese models predict risk of HF hospitalizations and ACM in patients admitted with HF and emphasize the importance of incorporating large numbers of variables in machine learning models to identify predictors for future investigation.
| S-EPMC8890080 | biostudies-literature
Project description:AimsCertain critical risk factors of heart failure with preserved ejection fraction (HFpEF) patients were significantly different from those of heart failure with reduced ejection fraction (HFrEF) patients, resulting in the limitations of existing predictive models in real-world situations. This study aimed to develop a machine learning model for predicting 90 day readmission for HFpEF patients.Methods and resultsData were extracted from electronic health records from 1 August 2020 to 1 August 2021 and follow-up records of patients with HFpEF within 3 months after discharge. Feature extraction was performed by univariate analysis combined with the least absolute shrinkage and selection operator (LASSO) algorithms. Machine learning models like eXtreme Gradient Boosting (XGBoost), random forest, neural network and logistic regression were adopted to construct models. The discrimination and calibration of each model were compared, and the Shapley Additive exPlanations (SHAP) method was used to explore the interpretability of the model. The cohort included 746 patients, of whom 103 (13.8%) were readmitted within 90 days. XGBoost owned the best performance [area under the curve (AUC) = 0.896, precision-recall area under the curve (PR-AUC) = 0.868, sensitivity = 0.817, specificity = 0.837, balanced accuracy = 0.827]. The Kolmogorov-Smirnov (KS) statistic was 0.694 at 0.468 in the XGBoost model. SHAP identified the top 12 risk features, including activities of daily living (ADL), left atrial dimension (LAD), left ventricular end-diastolic diameter (LVDD), shortness, nitrates, length of stay, nutritional risk, fall risk, accompanied by other symptoms, educational level, anticoagulants and edema.ConclusionsOur model could help medical agencies achieve the early identification of 90 day readmission risk in HFpEF patients and reveal risk factors that provide valuable insights for treatments.
| S-EPMC11631356 | biostudies-literature