Dataset Information

Comparison of machine-learning and logistic regression models for prediction of 30-day unplanned readmission in electronic health records: A development and validation study.

ABSTRACT: It is expected but unknown whether machine-learning models can outperform regression models, such as a logistic regression (LR) model, especially when the number and types of predictor variables increase in electronic health records (EHRs). We aimed to compare the predictive performance of gradient-boosted decision tree (GBDT), random forest (RF), deep neural network (DNN), and LR with the least absolute shrinkage and selection operator (LR-LASSO) for unplanned readmission. We used EHRs of patients discharged alive from 38 hospitals in 2015-2017 for derivation and in 2018 for validation, including basic characteristics, diagnosis, surgery, procedure, and drug codes, and blood-test results. The outcome was 30-day unplanned readmission. We created six patterns of data tables having different numbers of binary variables (that ≥5% or ≥1% of patients or ≥10 patients had) with and without blood-test results. For each pattern of data tables, we used the derivation data to establish the machine-learning and LR models, and used the validation data to evaluate the performance of each model. The incidence of outcome was 6.8% (23,108/339,513 discharges) and 6.4% (7,507/118,074 discharges) in the derivation and validation datasets, respectively. For the first data table with the smallest number of variables (102 variables that ≥5% of patients had, without blood-test results), the c-statistic was highest for GBDT (0.740), followed by RF (0.734), LR-LASSO (0.720), and DNN (0.664). For the last data table with the largest number of variables (1543 variables that ≥10 patients had, including blood-test results), the c-statistic was highest for GBDT (0.764), followed by LR-LASSO (0.755), RF (0.751), and DNN (0.720), suggesting that the difference between GBDT and LR-LASSO was small and their 95% confidence intervals overlapped. In conclusion, GBDT generally outperformed LR-LASSO to predict unplanned readmission, but the difference of c-statistic became smaller as the number of variables was increased and blood-test results were used.

SUBMITTER: Iwagami M

PROVIDER: S-EPMC11335098 | biostudies-literature | 2024 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparison of machine-learning and logistic regression models for prediction of 30-day unplanned readmission in electronic health records: A development and validation study.

Iwagami Masao M Inokuchi Ryota R Kawakami Eiryo E Yamada Tomohide T Goto Atsushi A Kuno Toshiki T Hashimoto Yohei Y Michihata Nobuaki N Goto Tadahiro T Shinozaki Tomohiro T Sun Yu Y Taniguchi Yuta Y Komiyama Jun J Uda Kazuaki K Abe Toshikazu T Tamiya Nanako N

PLOS digital health 20240820 8

It is expected but unknown whether machine-learning models can outperform regression models, such as a logistic regression (LR) model, especially when the number and types of predictor variables increase in electronic health records (EHRs). We aimed to compare the predictive performance of gradient-boosted decision tree (GBDT), random forest (RF), deep neural network (DNN), and LR with the least absolute shrinkage and selection operator (LR-LASSO) for unplanned readmission. We used EHRs of patie ...[more]

PMID: 39163277

Similar Datasets

Project description:ObjectiveUnplanned hospital readmissions following surgical interventions are associated with adverse events and contribute to increasing health care costs. Despite numerous studies defining risk factors following lower extremity bypass surgery, evidence regarding readmission after endovascular interventions is limited. This study aimed to identify predictors of 30-day unplanned readmission following infrainguinal endovascular interventions.MethodsWe identified all patients undergoing an infrainguinal endovascular intervention in the targeted vascular module of the American College of Surgeons National Surgical Quality Improvement Program between 2012 and 2014. Perioperative outcomes were stratified by symptom status (chronic limb-threatening ischemia [CLI] vs claudication). Patients who died during index admission and those who remained in the hospital after 30 days were excluded. Indications for unplanned readmission related to the index procedure were evaluated. Multivariable logistic regression was used to identify preoperative and in-hospital (during index admission) risk factors of 30-day unplanned readmission.ResultsThere were 4449 patients who underwent infrainguinal endovascular intervention, of whom 2802 (63%) had CLI (66% tissue loss) and 1647 (37%) had claudication. The unplanned readmission rates for CLI and claudication patients were 16% (n = 447) and 6.5% (n = 107), respectively. Mortality after index admission was higher for readmitted patients compared with those not readmitted (CLI, 3.4% vs 0.7% [P < .001]; claudication, 2.8% vs 0.1% [P < .01]). Approximately 50% of all unplanned readmissions were related to the index procedure. Among CLI patients, the most common indication for readmission related to the index procedure was wound or infection related (42%), whereas patients with claudication were mainly readmitted for recurrent symptoms of peripheral vascular disease (28%). In patients with CLI, predictors of unplanned readmission included diabetes (odds ratio, 1.3; 95% confidence interval, 1.01-1.6), congestive heart failure (1.6; 1.1-2.5), renal insufficiency (1.7; 1.3-2.2), preoperative dialysis (1.4; 1.02-1.9), tibial angioplasty/stenting (1.3; 1.04-1.6), in-hospital bleeding (1.9; 1.04-3.5), in-hospital unplanned return to the operating room (1.9; 1.1-3.5), and discharge other than to home (1.5; 1.1-2.0). Risk factors for those with claudication were dependent functional status (3.5; 1.4-8.7), smoking (1.6; 1.02-2.5), diabetes (1.5; 1.01-2.3), preoperative dialysis (3.6; 1.6-8.3), procedure time exceeding 120 minutes (1.8; 1.1-2.7), in-hospital bleeding (2.9; 1.2-7.4), and in-hospital unplanned return to the operating room (3.4; 1.2-9.4).ConclusionsUnplanned readmission after endovascular treatment is relatively common, especially in patients with CLI, and is associated with substantially increased mortality. Awareness of these risk factors will help providers identify patients at high risk who may benefit from early surveillance, and prophylactic measures focused on decreasing postoperative complications may reduce the rate of readmission.

Project description:Introduction: Preterm babies are a vulnerable population that experience significant short and long-term morbidity. Rehospitalisations constitute an important, potentially modifiable adverse event in this population. Improving the ability of clinicians to identify those patients at the greatest risk of rehospitalisation has the potential to improve outcomes and reduce costs. Machine-learning algorithms can provide potentially advantageous methods of prediction compared to conventional approaches like logistic regression. Objective: To compare two machine-learning methods (least absolute shrinkage and selection operator (LASSO) and random forest) to expert-opinion driven logistic regression modelling for predicting unplanned rehospitalisation within 30 days in a large French cohort of preterm babies. Design, Setting and Participants: This study used data derived exclusively from the population-based prospective cohort study of French preterm babies, EPIPAGE 2. Only those babies discharged home alive and whose parents completed the 1-year survey were eligible for inclusion in our study. All predictive models used a binary outcome, denoting a baby's status for an unplanned rehospitalisation within 30 days of discharge. Predictors included those quantifying clinical, treatment, maternal and socio-demographic factors. The predictive abilities of models constructed using LASSO and random forest algorithms were compared with a traditional logistic regression model. The logistic regression model comprised 10 predictors, selected by expert clinicians, while the LASSO and random forest included 75 predictors. Performance measures were derived using 10-fold cross-validation. Performance was quantified using area under the receiver operator characteristic curve, sensitivity, specificity, Tjur's coefficient of determination and calibration measures. Results: The rate of 30-day unplanned rehospitalisation in the eligible population used to construct the models was 9.1% (95% CI 8.2-10.1) (350/3,841). The random forest model demonstrated both an improved AUROC (0.65; 95% CI 0.59-0.7; p = 0.03) and specificity vs. logistic regression (AUROC 0.57; 95% CI 0.51-0.62, p = 0.04). The LASSO performed similarly (AUROC 0.59; 95% CI 0.53-0.65; p = 0.68) to logistic regression. Conclusions: Compared to an expert-specified logistic regression model, random forest offered improved prediction of 30-day unplanned rehospitalisation in preterm babies. However, all models offered relatively low levels of predictive ability, regardless of modelling method.

Project description:BackgroundExisting readmission reduction solutions tend to focus on complementing inpatient care with enhanced care transition and postdischarge interventions. These solutions are initiated near or after discharge, when clinicians' impact on inpatient care is ending. Preventive intervention during hospitalization is an underexplored area that holds potential for reducing readmission risk. However, it is challenging to predict readmission risk at the early stage of hospitalization because few data are available.ObjectiveThe objective of this study was to build an early prediction model of unplanned 30-day hospital readmission using a large and diverse sample. We were also interested in identifying novel readmission risk factors and protective factors.MethodsWe extracted the medical records of 96,550 patients in 205 participating Cerner client hospitals across four US census regions in 2016 from the Health Facts database. The model was built with index admission data that can become available within 24 hours and data from previous encounters up to 1 year before the index admission. The candidate models were evaluated for performance, timeliness, and generalizability. Multivariate logistic regression analysis was used to identify readmission risk factors and protective factors.ResultsWe developed six candidate readmission models with different machine learning algorithms. The best performing model of extreme gradient boosting (XGBoost) achieved an area under the receiver operating characteristic curve of 0.753 on the development data set and 0.742 on the validation data set. By multivariate logistic regression analysis, we identified 14 risk factors and 2 protective factors of readmission that have never been reported.ConclusionsThe performance of our model is better than that of the most widely used models in US health care settings. This model can help clinicians identify readmission risk at the early stage of hospitalization so that they can pay extra attention during the care process of high-risk patients. The 14 novel risk factors and 2 novel protective factors can aid understanding of the factors associated with readmission.

Project description:BACKGROUND:Electronic health record (EHR) systems generate large datasets that can significantly enrich the development of medical predictive models. Several attempts have been made to investigate the effect of glycated hemoglobin (HbA1c) elevation on the prediction of diabetes onset. However, there is still a need for validation of these models using EHR data collected from different populations. OBJECTIVE:The aim of this study is to perform a replication study to validate, evaluate, and identify the strengths and weaknesses of replicating a predictive model that employed multiple logistic regression with EHR data to forecast the levels of HbA1c. The original study used data from a population in the United States and this differentiated replication used a population in Saudi Arabia. METHODS:A total of 3 models were developed and compared with the model created in the original study. The models were trained and tested using a larger dataset from Saudi Arabia with 36,378 records. The 10-fold cross-validation approach was used for measuring the performance of the models. RESULTS:Applying the method employed in the original study achieved an accuracy of 74% to 75% when using the dataset collected from Saudi Arabia, compared with 77% obtained from using the population from the United States. The results also show a different ranking of importance for the predictors between the original study and the replication. The order of importance for the predictors with our population, from the most to the least importance, is age, random blood sugar, estimated glomerular filtration rate, total cholesterol, non-high-density lipoprotein, and body mass index. CONCLUSIONS:This replication study shows that direct use of the models (calculators) created using multiple logistic regression to predict the level of HbA1c may not be appropriate for all populations. This study reveals that the weighting of the predictors needs to be calibrated to the population used. However, the study does confirm that replicating the original study using a different population can help with predicting the levels of HbA1c by using the predictors that are routinely collected and stored in hospital EHR systems.

Dataset Information

Comparison of machine-learning and logistic regression models for prediction of 30-day unplanned readmission in electronic health records: A development and validation study.

Publications

Comparison of machine-learning and logistic regression models for prediction of 30-day unplanned readmission in electronic health records: A development and validation study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets