Machine learning-based prediction of coronary care unit readmission: A multihospital validation study.
Ontology highlight
ABSTRACT: Objective
Readmission to the coronary care unit (CCU) has significant implications for patient outcomes and healthcare expenditure, emphasizing the urgency to accurately identify patients at high readmission risk. This study aims to construct and externally validate a predictive model for CCU readmission using machine learning (ML) algorithms across multiple hospitals.Methods
Patient information, including demographics, medical history, and laboratory test results were collected from electronic health record system and contributed to a total of 40 features. Five ML models: logistic regression, random forest, support vector machine, gradient boosting, and multilayer perceptron were employed to estimate the readmission risk.Results
The gradient boosting model was selected demonstrated superior performance with an area under the receiver operating characteristic curve (AUC) of 0.887 in the internal validation set. Further external validation in hold-out test set and three other medical centers upheld the model's robustness with consistent high AUCs, ranging from 0.852 to 0.879.Conclusion
The results endorse the integration of ML algorithms in healthcare to enhance patient risk stratification, potentially optimizing clinical interventions, and diminishing the burden of CCU readmissions.
SUBMITTER: Yau FF
PROVIDER: S-EPMC11367690 | biostudies-literature | 2024 Jan-Dec
REPOSITORIES: biostudies-literature
Publications
Machine learning-based prediction of coronary care unit readmission: A multihospital validation study.
Yau Fei-Fei Flora FF Chiu I-Min IM Wu Kuan-Han KH Cheng Chi-Yung CY Lee Wei-Chieh WC Chen Huang-Chung HC Cheng Cheng-I CI Chen Tien-Yu TY
Digital health 20240101
<h4>Objective</h4>Readmission to the coronary care unit (CCU) has significant implications for patient outcomes and healthcare expenditure, emphasizing the urgency to accurately identify patients at high readmission risk. This study aims to construct and externally validate a predictive model for CCU readmission using machine learning (ML) algorithms across multiple hospitals.<h4>Methods</h4>Patient information, including demographics, medical history, and laboratory test results were collected ...[more]
Similar Datasets
Project description:BackgroundIntensive care unit (ICU) readmission is a crucial indicator of patient safety. However, discharge decisions often rely on subjective assessment due to a lack of standardized guidelines. We aimed to develop a machine-learning model to predict ICU readmission within 48 h and compare its performance to traditional scoring systems.MethodsWe developed an ensemble model, iREAD, that generates a probability score at ICU discharge, representing the likelihood of the patient being readmitted to the ICU within 48 h, using data from Seoul National University Hospital (SNUH) and validated it using the MIMIC-III and eICU-CRD datasets. From September 2007 to August 2021, a total of 70,842 patients were included from SNUH. The MIMIC-III datasets comprised 43,237 patients admitted to ICUs between 2001 and 2012 at Beth Israel Deaconess Medical Center, and the eICU-CRD datasets included 90,271 ICU admissions across 208 hospitals between 2014 and 2015. Patients younger than 18, those who died in ICUs, or who refused life-sustaining treatment were excluded from the final analysis. The model's performance was evaluated using the area under the receiver operating characteristic curve (AUROC) and compared to the traditional scores and conventional machine learning models. Kaplan-Meier analysis was performed to compare the outcome between the high-risk and low-risk groups.FindingsWe developed the iREAD, that utilized 30 input features, encompassing demographics, length of stay, vital signs, GCS, and laboratory values. iREAD demonstrated superior performance compared with other models across all cohorts (all P < 0.001). In the internal validation, iREAD achieved AUROCs of 0.771 (95% CI 0.743-0.798), 0.834 (0.821-0.846), and 0.820 (0.808-0.832) for early (≤48 h), late (>48 h), and overall ICU readmissions, respectively. External validations with MIMIC-III and eICU-CRD also showed modest performance with AUROCs of 0.768 (0.748-0.787) and 0.725 (0.712-0.739) for overall readmission in MIMIC-III and eICU-CRD respectively, demonstrating superior performance compared to other models (All P < 0.001; higher than other models). Kaplan-Meier analysis revealed that over 40% of high-risk patients predicted by iREAD were readmitted within 48 h, representing a more than four-fold increase in predictive performance compared to the traditional scores.InterpretationiREAD demonstrates superior performance in predicting ICU readmission within 48 h after discharge compared to traditional scoring systems or conventional machine learning models in both internal and external validations. While the performance degradation observed in the external validations suggests the need for further prospective validation on diverse patient populations, the robust performance and ability to identify high-risk patients have the potential to guide clinical decision-making.FundingThis work was supported by the Korea Health Technology Research & Development Project through the Korea Health Industry Development Institute, funded by the Ministry of Health and Welfare, Republic of Korea (grant number RS-2021-KH114109).
| S-EPMC11872568 | biostudies-literature
Project description:ObjectivesUnplanned readmissions to the intensive care unit (ICU) are highly undesirable, increasing variance in care, making resource planning difficult and potentially increasing length of stay and mortality in some settings. Identifying patients who are likely to suffer unplanned ICU readmission could reduce the frequency of this adverse event.SettingA single academic, tertiary care hospital in the UK.ParticipantsA set of 3326 ICU episodes collected between October 2014 and August 2016. All records were of patients who visited an ICU at some point during their stay. We excluded patients who were ≤16 years of age; visited ICUs other than the general and neurosciences ICU; were missing crucial electronic patient record measurements; or had indeterminate ICU discharge outcomes or very early or extremely late discharge times. After exclusion, 2018 outcome-labelled episodes remained.Primary and secondary outcome measuresArea under the receiver operating characteristic curve (AUROC) for prediction of unplanned ICU readmission or in-hospital death within 48 hours of first ICU discharge.ResultsIn 10-fold cross-validation, an ensemble predictor was trained on data from both the target hospital and the Medical Information Mart for Intensive Care (MIMIC-III) database and tested on the target hospital's data. This predictor discriminated between patients with the unplanned ICU readmission or death outcome and those without this outcome, attaining mean AUROC of 0.7095 (SE 0.0260), superior to the purpose-built Stability and Workload Index for Transfer (SWIFT) score (AUROC=0.6082, SE 0.0249; p=0.014, pairwise t-test).ConclusionsDespite the inherent difficulties, we demonstrate that a novel machine learning algorithm based on transfer learning could achieve good discrimination, over and above that of the treating clinicians or the value added by the SWIFT score. Accurate prediction of unplanned readmission could be used to target resources more efficiently.
| S-EPMC5640090 | biostudies-literature
Project description:IntroductionSeptic patients requiring intensive care unit (ICU) readmission are at high risk of mortality, but research focusing on the association of ICU readmission due to sepsis and mortality is limited. The aim of this study was to develop and validate a machine learning (ML) model for predicting in-hospital mortality in septic patients readmitted to the ICU using routinely available clinical data.MethodsThe data used in this study were obtained from the Medical Information Mart for Intensive Care (MIMIC-IV, v1.0) database, between 2008 and 2019. The study cohort included patients with sepsis requiring ICU readmission. The data were randomly split into a training (75%) data set and a validation (25%) data set. Nine popular ML models were developed to predict mortality in septic patients readmitted to the ICU. The model with the best accuracy and area under the curve (A.C.) in the validation cohort was defined as the optimal model and was selected for further prediction studies. The SHAPELY Additive explanations (SHAP) values and Local Interpretable Model-Agnostic Explanation (LIME) methods were used to improve the interpretability of the optimal model.ResultsA total of 1117 septic patients who had required ICU readmission during the study period were enrolled in the study. Of these participants, 434 (38.9%) were female, and the median (interquartile range [IQR]) age was 68.6 (58.4-79.2) years. The median (IQR) ICU interval duration was 2.60 (0.64-5.78) days. After feature selection, 31 of 47 clinical factors were ultimately chosen for use in model construction. Of the nine ML models tested, the best performance was achieved with the random forest (RF) model, with an A.C. of 0.81, an accuracy of 85% and a precision of 62% in the validation cohort. The SHAP summary analysis revealed that Glasgow Coma Scale score, urine output, blood urea nitrogen, lactate, platelet count and systolic blood pressure were the top six most important factors contributing to the RF model. Additionally, the LIME method demonstrated how the RF model works in terms of explaining risk of death prediction in septic patients requiring ICU readmission.ConclusionThe ML models reported here showed a good prognostic prediction ability in septic patients requiring ICU readmission. Of the features selected, the parameters related to organ perfusion made the largest contribution to outcome prediction during ICU readmission in septic patients.
| S-EPMC9282631 | biostudies-literature
Project description:RationalePatients transferred from the intensive care unit to the wards who are later readmitted to the intensive care unit have increased length of stay, healthcare expenditure, and mortality compared with those who are never readmitted. Improving risk stratification for patients transferred to the wards could have important benefits for critically ill hospitalized patients.ObjectivesWe aimed to use a machine-learning technique to derive and validate an intensive care unit readmission prediction model with variables available in the electronic health record in real time and compare it to previously published algorithms.MethodsThis observational cohort study was conducted at an academic hospital in the United States with approximately 600 inpatient beds. A total of 24,885 intensive care unit transfers to the wards were included, with 14,962 transfers (60%) in the training cohort and 9,923 transfers (40%) in the internal validation cohort. Patient characteristics, nursing assessments, International Classification of Diseases, Ninth Revision codes from prior admissions, medications, intensive care unit interventions, diagnostic tests, vital signs, and laboratory results were extracted from the electronic health record and used as predictor variables in a gradient-boosted machine model. Accuracy for predicting intensive care unit readmission was compared with the Stability and Workload Index for Transfer score and Modified Early Warning Score in the internal validation cohort and also externally using the Medical Information Mart for Intensive Care database (n = 42,303 intensive care unit transfers).ResultsEleven percent (2,834) of discharges to the wards were later readmitted to the intensive care unit. The machine-learning-derived model had significantly better performance (area under the receiver operating curve, 0.76) than either the Stability and Workload Index for Transfer score (area under the receiver operating curve, 0.65), or Modified Early Warning Score (area under the receiver operating curve, 0.58; P value < 0.0001 for all comparisons). At a specificity of 95%, the derived model had a sensitivity of 28% compared with 15% for Stability and Workload Index for Transfer score and 7% for the Modified Early Warning Score. Accuracy improvements with the derived model over Modified Early Warning Score and Stability and Workload Index for Transfer were similar in the Medical Information Mart for Intensive Care-III cohort.ConclusionsA machine learning approach to predicting intensive care unit readmission was significantly more accurate than previously published algorithms in both our internal validation and the Medical Information Mart for Intensive Care-III cohort. Implementation of this approach could target patients who may benefit from additional time in the intensive care unit or more frequent monitoring after transfer to the hospital ward.
| S-EPMC6207111 | biostudies-literature
Project description:This study intends to predict in-hospital and 6-month mortality, as well as 30-day and 90-day hospital readmission, using Machine Learning (ML) approach via conventional features. A total of 737 patients remained after applying the exclusion criteria to 1101 heart failure patients. Thirty-four conventional features were collected for each patient. First, the data were divided into train and test cohorts with a 70-30% ratio. Then train data were normalized using the Z-score method, and its mean and standard deviation were applied to the test data. Subsequently, Boruta, RFE, and MRMR feature selection methods were utilized to select more important features in the training set. In the next step, eight ML approaches were used for modeling. Next, hyperparameters were optimized using tenfold cross-validation and grid search in the train dataset. All model development steps (normalization, feature selection, and hyperparameter optimization) were performed on a train set without touching the hold-out test set. Then, bootstrapping was done 1000 times on the hold-out test data. Finally, the obtained results were evaluated using four metrics: area under the ROC curve (AUC), accuracy (ACC), specificity (SPE), and sensitivity (SEN). The RFE-LR (AUC: 0.91, ACC: 0.84, SPE: 0.84, SEN: 0.83) and Boruta-LR (AUC: 0.90, ACC: 0.85, SPE: 0.85, SEN: 0.83) models generated the best results in terms of in-hospital mortality. In terms of 30-day rehospitalization, Boruta-SVM (AUC: 0.73, ACC: 0.81, SPE: 0.85, SEN: 0.50) and MRMR-LR (AUC: 0.71, ACC: 0.68, SPE: 0.69, SEN: 0.63) models performed the best. The best model for 3-month rehospitalization was MRMR-KNN (AUC: 0.60, ACC: 0.63, SPE: 0.66, SEN: 0.53) and regarding 6-month mortality, the MRMR-LR (AUC: 0.61, ACC: 0.63, SPE: 0.44, SEN: 0.66) and MRMR-NB (AUC: 0.59, ACC: 0.61, SPE: 0.48, SEN: 0.63) models outperformed the others. Reliable models were developed in 30-day rehospitalization and in-hospital mortality using conventional features and ML techniques. Such models can effectively personalize treatment, decision-making, and wiser budget allocation. Obtained results in 3-month rehospitalization and 6-month mortality endpoints were not astonishing and further experiments with additional information are needed to fetch promising results in these endpoints.
| S-EPMC10618467 | biostudies-literature
Project description:BackgroundUse of statistical models for assessing the clinical risk of readmission to medical and surgical intensive care units is well established. However, models for predicting risk of coronary care unit (CCU) readmission are rarely reported. Therefore, this study investigated the characteristics and outcomes of patients readmitted to CCU to identify risk factors for CCU readmission and to establish a scoring system for identifying patients at high risk for CCU readmission.MethodsMedical data were collected for 27,841 patients with a history of readmission to the CCU of a single multi-center healthcare provider in Taiwan during 2001-2019. Characteristics and outcomes were compared between a readmission group and a non-readmission group. Data were segmented at a 9:1 ratio for model building and validation.ResultsThe number of patients with a CCU readmission history after transfer to a standard care ward was 1,790 (6.4%). The eleven factors that had the strongest associations with CCU readmission were used to develop and validate a CCU readmission risk scoring and prediction model. When the model was used to predict CCU readmission, the receiver-operating curve characteristic was 0.7038 for risk score model group and 0.7181 for the validation group. A CCU readmission risk score was assigned to each patient. The patients were then stratified by risk score into low risk (0-12), moderate risk (13-31) and high risk (32-40) cohorts check scores, which showed that CCU readmission risk significantly differed among the three groups.ConclusionsThis study developed a model for estimating CCU readmission risk. By using the proposed model, clinicians can improve CCU patient outcomes and medical care quality.
| S-EPMC8907527 | biostudies-literature
Project description:Purpose: To develop and validate machine learning models for predicting the length of stay (LOS) in the Pediatric Intensive Care Unit (PICU) using data from the Virtual Pediatric Systems (VPS) database. Methods: A retrospective study was conducted utilizing machine learning (ML) algorithms to analyze and predict PICU LOS based on historical patient data from the VPS database. The study included data from over 100 North American PICUs spanning the years 2015-2020. After excluding entries with missing variables and those indicating recovery from cardiac surgery, the dataset comprised 123,354 patient encounters. Various ML models, including Support Vector Machine, Stochastic Gradient Descent Classifier, K-Nearest Neighbors, Decision Tree, Gradient Boosting, CatBoost, and Recurrent Neural Networks (RNNs), were evaluated for their accuracy in predicting PICU LOS at thresholds of 24 h, 36 h, 48 h, 72 h, 5 days, and 7 days. Results: Gradient Boosting, CatBoost, and RNN models demonstrated the highest accuracy, particularly at the 36 h and 48 h thresholds, with accuracy rates between 70 and 73%. These results far outperform traditional statistical and existing prediction methods that report accuracy of only around 50%, which is effectively unusable in the practical setting. These models also exhibited balanced performance between sensitivity (up to 74%) and specificity (up to 82%) at these thresholds. Conclusions: ML models, particularly Gradient Boosting, CatBoost, and RNNs, show moderate effectiveness in predicting PICU LOS with accuracy slightly over 70%, outperforming previously reported human predictions. This suggests potential utility in enhancing resource and staffing management in PICUs. However, further improvements through training on specialized databases can potentially achieve better accuracy and clinical applicability.
| S-EPMC11504172 | biostudies-literature
Project description:BackgroundPatient safety in the intensive care unit (ICU) is one of the most critical issues, and unplanned extubation (UE) is considered the most adverse event for patient safety. Prevention and early detection of such an event is an essential but difficult component of quality care.ObjectiveThis study aimed to develop and validate prediction models for UE in ICU patients using machine learning.MethodsThis study was conducted in an academic tertiary hospital in Seoul, Republic of Korea. The hospital had approximately 2000 inpatient beds and 120 ICU beds. As of January 2019, the hospital had approximately 9000 outpatients on a daily basis. The number of annual ICU admissions was approximately 10,000. We conducted a retrospective study between January 1, 2010, and December 31, 2018. A total of 6914 extubation cases were included. We developed a UE prediction model using machine learning algorithms, which included random forest (RF), logistic regression (LR), artificial neural network (ANN), and support vector machine (SVM). For evaluating the model's performance, we used the area under the receiver operating characteristic curve (AUROC). The sensitivity, specificity, positive predictive value, negative predictive value, and F1 score were also determined for each model. For performance evaluation, we also used a calibration curve, the Brier score, and the integrated calibration index (ICI) to compare different models. The potential clinical usefulness of the best model at the best threshold was assessed through a net benefit approach using a decision curve.ResultsAmong the 6914 extubation cases, 248 underwent UE. In the UE group, there were more males than females, higher use of physical restraints, and fewer surgeries. The incidence of UE was higher during the night shift as compared to the planned extubation group. The rate of reintubation within 24 hours and hospital mortality were higher in the UE group. The UE prediction algorithm was developed, and the AUROC for RF was 0.787, for LR was 0.762, for ANN was 0.763, and for SVM was 0.740.ConclusionsWe successfully developed and validated machine learning-based prediction models to predict UE in ICU patients using electronic health record data. The best AUROC was 0.787 and the sensitivity was 0.949, which was obtained using the RF algorithm. The RF model was well-calibrated, and the Brier score and ICI were 0.129 and 0.048, respectively. The proposed prediction model uses widely available variables to limit the additional workload on the clinician. Further, this evaluation suggests that the model holds potential for clinical usefulness.
| S-EPMC8387891 | biostudies-literature
Project description:BackgroundEarly identification of impending in-hospital cardiac arrest (IHCA) improves clinical outcomes but remains elusive for practicing clinicians.ObjectiveWe aimed to develop a multimodal machine learning algorithm based on ensemble techniques to predict the occurrence of IHCA.MethodsOur model was developed by the Multiparameter Intelligent Monitoring of Intensive Care (MIMIC)-IV database and validated in the Electronic Intensive Care Unit Collaborative Research Database (eICU-CRD). Baseline features consisting of patient demographics, presenting illness, and comorbidities were collected to train a random forest model. Next, vital signs were extracted to train a long short-term memory model. A support vector machine algorithm then stacked the results to form the final prediction model.ResultsOf 23,909 patients in the MIMIC-IV database and 10,049 patients in the eICU-CRD database, 452 and 85 patients, respectively, had IHCA. At 13 hours in advance of an IHCA event, our algorithm had already demonstrated an area under the receiver operating characteristic curve of 0.85 (95% CI 0.815-0.885) in the MIMIC-IV database. External validation with the eICU-CRD and National Taiwan University Hospital databases also presented satisfactory results, showing area under the receiver operating characteristic curve values of 0.81 (95% CI 0.763-0.851) and 0.945 (95% CI 0.934-0.956), respectively.ConclusionsUsing only vital signs and information available in the electronic medical record, our model demonstrates it is possible to detect a trajectory of clinical deterioration up to 13 hours in advance. This predictive tool, which has undergone external validation, could forewarn and help clinicians identify patients in need of assessment to improve their overall prognosis.
| S-EPMC11287234 | biostudies-literature
Project description:BackgroundDelirium frequently occurs among patients admitted to the intensive care unit (ICU). There is limited evidence to support interventions to treat or resolve delirium in patients who have already developed delirium. Therefore, the early recognition and prevention of delirium are important in the management of critically ill patients.ObjectiveThis study aims to develop and validate a delirium prediction model within 24 hours of admission to the ICU using electronic health record data. The algorithm was named the Prediction of ICU Delirium (PRIDE).MethodsThis is a retrospective cohort study performed at a tertiary referral hospital with 120 ICU beds. We only included patients who were 18 years or older at the time of admission and who stayed in the medical or surgical ICU. Patients were excluded if they lacked a Confusion Assessment Method for the ICU record from the day of ICU admission or if they had a positive Confusion Assessment Method for the ICU record at the time of ICU admission. The algorithm to predict delirium was developed using patient data from the first 2 years of the study period and validated using patient data from the last 6 months. Random forest (RF), Extreme Gradient Boosting (XGBoost), deep neural network (DNN), and logistic regression (LR) were used. The algorithms were externally validated using MIMIC-III data, and the algorithm with the largest area under the receiver operating characteristics (AUROC) curve in the external data set was named the PRIDE algorithm.ResultsA total of 37,543 cases were collected. After patient exclusion, 12,409 remained as our study population, of which 3816 (30.8%) patients experienced delirium incidents during the study period. Based on the exclusion criteria, out of the 96,016 ICU admission cases in the MIMIC-III data set, 2061 cases were included, and 272 (13.2%) delirium incidents occurred. The average AUROCs and 95% CIs for internal validation were 0.916 (95% CI 0.916-0.916) for RF, 0.919 (95% CI 0.919-0.919) for XGBoost, 0.881 (95% CI 0.878-0.884) for DNN, and 0.875 (95% CI 0.875-0.875) for LR. Regarding the external validation, the best AUROC were 0.721 (95% CI 0.72-0.721) for RF, 0.697 (95% CI 0.695-0.699) for XGBoost, 0.655 (95% CI 0.654-0.657) for DNN, and 0.631 (95% CI 0.631-0.631) for LR. The Brier score of the RF model is 0.168, indicating that it is well-calibrated.ConclusionsA machine learning approach based on electronic health record data can be used to predict delirium within 24 hours of ICU admission. RF, XGBoost, DNN, and LR models were used, and they effectively predicted delirium. However, with the potential to advise ICU physicians and prevent ICU delirium, prospective studies are required to verify the algorithm's performance.
| S-EPMC8367129 | biostudies-literature