A comparison of general and disease-specific machine learning models for the prediction of unplanned hospital readmissions.
Ontology highlight
ABSTRACT:
SUBMITTER: Sutter T
PROVIDER: S-EPMC7973448 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ImportanceHospital readmissions are associated with patient harm and expense. Ways to prevent hospital readmissions have focused on identifying patients at greatest risk using prediction scores.ObjectiveTo identify the type of score that best predicts hospital readmissions.Design, setting, and participantsThis prognostic study included 14?062 consecutive adult hospital patients with 16?649 discharges from a tertiary care center, suburban community hospital, and urban critical access hospital in Maryland from September 1, 2016, through December 31, 2016. Patients not included as eligible discharges by the Centers for Medicare & Medicaid Services or the Chesapeake Regional Information System for Our Patients were excluded. A machine learning rank score, the Baltimore score (B score) developed using a machine learning technique, for each individual hospital using data from the 2 years before September 1, 2016, was compared with standard readmission risk assessment scores to predict 30-day unplanned readmissions.Main outcomes and measuresThe 30-day readmission rate evaluated using various readmission scores: B score, HOSPITAL score, modified LACE score, and Maxim/RightCare score.ResultsOf the 10?732 patients (5605 [52.2%] male; mean [SD] age, 54.56 [22.42] years) deemed to be eligible for the study, 1422 were readmitted. The area under the receiver operating characteristic curve (AUROC) for individual rules was 0.63 (95% CI, 0.61-0.65) for the HOSPITAL score, which was significantly lower than the 0.66 for modified LACE score (95% CI, 0.64-0.68; P?<?.001). The B score machine learning score was significantly better than all other scores; 48 hours after admission, the AUROC of the B score was 0.72 (95% CI, 0.70-0.73), which increased to 0.78 (95% CI, 0.77-0.79) at discharge (all P?<?.001). At the hospital using Maxim/RightCare score, the AUROC was 0.63 (95% CI, 0.59-0.69) for HOSPITAL, 0.64 (95% CI, 0.61-0.68) for Maxim/RightCare, and 0.66 (95% CI, 0.62-0.69) for modified LACE score. The B score was 0.72 (95% CI, 0.69-0.75) 48 hours after admission and 0.81 (95% CI, 0.79-0.84) at discharge. In directly comparing the B score with the sensitivity at cutoff values for modified LACE, HOSPITAL, and Maxim/RightCare scores, the B score was able to identify the same number of readmitted patients while flagging 25.5% to 54.9% fewer patients.Conclusions and relevanceAmong 3 hospitals in different settings, an automated machine learning score better predicted readmissions than commonly used readmission scores. More efficiently targeting patients at higher risk of readmission may be the first step toward potentially preventing readmissions.
| S-EPMC6484642 | biostudies-literature
Project description:Predicting unplanned rehospitalizations has traditionally employed logistic regression models. Machine learning (ML) methods have been introduced in health service research and may improve the prediction of health outcomes. The objective of this work was to develop a ML model to predict 30-day all-cause rehospitalizations based on the French hospital medico-administrative database.This was a retrospective cohort study of all discharges in the year 2015 from acute-care inpatient hospitalizations in a tertiary-care university center comprising 4 French hospitals. The study endpoint was unplanned 30-day all-cause rehospitalization. Logistic regression (LR), classification and regression trees (CART), random forest (RF), gradient boosting (GB), and neural networks (NN) were applied to the collected data. The predictive performance of the models was evaluated using the H-measure and the area under the ROC curve (AUC).Our analysis included 118,650 hospitalizations, of which 4127 (3.5%) led to rehospitalizations via emergency departments. The RF model was the most performant model according to the H-measure (0.29) and the AUC (0.79). The performances of the RF, GB and NN models (H-measures ranged from 0.18 to 0. 29, AUC ranged from 0.74 to 0.79) were better than those of the LR model (H-measure = 0.18, AUC = 0.74); all P values <.001. In contrast, LR was superior to CART (H-measure = 0.16, AUC = 0.70), P < .0001.The use of ML may be an alternative to regression models to predict health outcomes. The integration of ML, particularly the RF algorithm, in the prediction of unplanned rehospitalization may help health service providers target patients at high risk of rehospitalizations and propose effective interventions at the hospital level.
| S-EPMC7717815 | biostudies-literature
Project description:ObjectiveTo assess both the feasibility and potential impact of predicting preventable hospital readmissions using causal machine learning applied to data from the implementation of a readmissions prevention intervention (the Transitions Program).Data sourcesElectronic health records maintained by Kaiser Permanente Northern California (KPNC).Study designRetrospective causal forest analysis of postdischarge outcomes among KPNC inpatients. Using data from both before and after implementation, we apply causal forests to estimate individual-level treatment effects of the Transitions Program intervention on 30-day readmission. These estimates are used to characterize treatment effect heterogeneity and to assess the notional impacts of alternative targeting strategies in terms of the number of readmissions prevented.Data collection1 539 285 index hospitalizations meeting the inclusion criteria and occurring between June 2010 and December 2018 at 21 KPNC hospitals.Principal findingsThere appears to be substantial heterogeneity in patients' responses to the intervention (omnibus test for heterogeneity p = 2.23 × 10-7 ), particularly across levels of predicted risk. Notably, predicted treatment effects become more positive as predicted risk increases; patients at somewhat lower risk appear to have the largest predicted effects. Moreover, these estimates appear to be well calibrated, yielding the same estimate of annual readmissions prevented in the actual treatment subgroup (1246, 95% confidence interval [CI] 1110-1381) as did a formal evaluation of the Transitions Program (1210, 95% CI 990-1430). Estimates of the impacts of alternative targeting strategies suggest that as many as 4458 (95% CI 3925-4990) readmissions could be prevented annually, while decreasing the number needed to treat from 33 to 23, by targeting patients with the largest predicted effects rather than those at highest risk.ConclusionsCausal machine learning can be used to identify preventable hospital readmissions, if the requisite interventional data are available. Moreover, our results suggest a mismatch between risk and treatment effects.
| S-EPMC7704477 | biostudies-literature
Project description:OBJECTIVES:Unplanned readmissions to the intensive care unit (ICU) are highly undesirable, increasing variance in care, making resource planning difficult and potentially increasing length of stay and mortality in some settings. Identifying patients who are likely to suffer unplanned ICU readmission could reduce the frequency of this adverse event. SETTING:A single academic, tertiary care hospital in the UK. PARTICIPANTS:A set of 3326 ICU episodes collected between October 2014 and August 2016. All records were of patients who visited an ICU at some point during their stay. We excluded patients who were ?16 years of age; visited ICUs other than the general and neurosciences ICU; were missing crucial electronic patient record measurements; or had indeterminate ICU discharge outcomes or very early or extremely late discharge times. After exclusion, 2018 outcome-labelled episodes remained. PRIMARY AND SECONDARY OUTCOME MEASURES:Area under the receiver operating characteristic curve (AUROC) for prediction of unplanned ICU readmission or in-hospital death within 48?hours of first ICU discharge. RESULTS:In 10-fold cross-validation, an ensemble predictor was trained on data from both the target hospital and the Medical Information Mart for Intensive Care (MIMIC-III) database and tested on the target hospital's data. This predictor discriminated between patients with the unplanned ICU readmission or death outcome and those without this outcome, attaining mean AUROC of 0.7095 (SE 0.0260), superior to the purpose-built Stability and Workload Index for Transfer (SWIFT) score (AUROC=0.6082, SE 0.0249; p=0.014, pairwise t-test). CONCLUSIONS:Despite the inherent difficulties, we demonstrate that a novel machine learning algorithm based on transfer learning could achieve good discrimination, over and above that of the treating clinicians or the value added by the SWIFT score. Accurate prediction of unplanned readmission could be used to target resources more efficiently.
| S-EPMC5640090 | biostudies-other
Project description:BackgroundAlthough multiple prediction models have been developed to predict hospital admission to emergency departments (EDs) to address overcrowding and patient safety, only a few studies have examined prediction models for prehospital use. Development of institution-specific prediction models is feasible in this age of data science, provided that predictor-related information is readily collectable.ObjectiveWe aimed to develop a hospital admission prediction model based on patient information that is commonly available during ambulance transport before hospitalization.MethodsPatients transported by ambulance to our ED from April 2018 through March 2019 were enrolled. Candidate predictors were age, sex, chief complaint, vital signs, and patient medical history, all of which were recorded by emergency medical teams during ambulance transport. Patients were divided into two cohorts for derivation (3601/5145, 70.0%) and validation (1544/5145, 30.0%). For statistical models, logistic regression, logistic lasso, random forest, and gradient boosting machine were used. Prediction models were developed in the derivation cohort. Model performance was assessed by area under the receiver operating characteristic curve (AUROC) and association measures in the validation cohort.ResultsOf 5145 patients transported by ambulance, including deaths in the ED and hospital transfers, 2699 (52.5%) required hospital admission. Prediction performance was higher with the addition of predictive factors, attaining the best performance with an AUROC of 0.818 (95% CI 0.792-0.839) with a machine learning model and predictive factors of age, sex, chief complaint, and vital signs. Sensitivity and specificity of this model were 0.744 (95% CI 0.716-0.773) and 0.745 (95% CI 0.709-0.776), respectively.ConclusionsFor patients transferred to EDs, we developed a well-performing hospital admission prediction model based on routinely collected prehospital information including chief complaints.
| S-EPMC7655472 | biostudies-literature
Project description:ObjectiveTo compare machine learning methods for predicting inpatient seizures risk and determine the feasibility of 1-h screening EEG to identify low-risk patients (<5% seizures risk in 48 h).MethodsThe Critical Care EEG Monitoring Research Consortium (CCEMRC) multicenter database contains 7716 continuous EEGs (cEEG). Neural networks (NN), elastic net logistic regression (EN), and sparse linear integer model (RiskSLIM) were trained to predict seizures. RiskSLIM was used previously to generate 2HELPS2B model of seizure predictions. Data were divided into training (60% for model fitting) and evaluation (40% for model evaluation) cohorts. Performance was measured using area under the receiver operating curve (AUC), mean risk calibration (CAL), and negative predictive value (NPV). A secondary analysis was performed using Monte Carlo simulation (MCS) to normalize all EEG recordings to 48 h and use only the first hour of EEG as a "screening EEG" to generate predictions.ResultsRiskSLIM recreated the 2HELPS2B model. All models had comparable AUC: evaluation cohort (NN: 0.85, EN: 0.84, 2HELPS2B: 0.83) and MCS (NN: 0.82, EN; 0.82, 2HELPS2B: 0.81) and NPV (absence of seizures in the group that the models predicted to be low risk): evaluation cohort (NN: 97%, EN: 97%, 2HELPS2B: 97%) and MCS (NN: 97%, EN: 99%, 2HELPS2B: 97%). 2HELPS2B model was able to identify the largest proportion of low-risk patients.InterpretationFor seizure risk stratification of hospitalized patients, the RiskSLIM generated 2HELPS2B model compares favorably to the complex NN and EN generated models. 2HELPS2B is able to accurately and quickly identify low-risk patients with only a 1-h screening EEG.
| S-EPMC6649418 | biostudies-literature
Project description:It is expected but unknown whether machine-learning models can outperform regression models, such as a logistic regression (LR) model, especially when the number and types of predictor variables increase in electronic health records (EHRs). We aimed to compare the predictive performance of gradient-boosted decision tree (GBDT), random forest (RF), deep neural network (DNN), and LR with the least absolute shrinkage and selection operator (LR-LASSO) for unplanned readmission. We used EHRs of patients discharged alive from 38 hospitals in 2015-2017 for derivation and in 2018 for validation, including basic characteristics, diagnosis, surgery, procedure, and drug codes, and blood-test results. The outcome was 30-day unplanned readmission. We created six patterns of data tables having different numbers of binary variables (that ≥5% or ≥1% of patients or ≥10 patients had) with and without blood-test results. For each pattern of data tables, we used the derivation data to establish the machine-learning and LR models, and used the validation data to evaluate the performance of each model. The incidence of outcome was 6.8% (23,108/339,513 discharges) and 6.4% (7,507/118,074 discharges) in the derivation and validation datasets, respectively. For the first data table with the smallest number of variables (102 variables that ≥5% of patients had, without blood-test results), the c-statistic was highest for GBDT (0.740), followed by RF (0.734), LR-LASSO (0.720), and DNN (0.664). For the last data table with the largest number of variables (1543 variables that ≥10 patients had, including blood-test results), the c-statistic was highest for GBDT (0.764), followed by LR-LASSO (0.755), RF (0.751), and DNN (0.720), suggesting that the difference between GBDT and LR-LASSO was small and their 95% confidence intervals overlapped. In conclusion, GBDT generally outperformed LR-LASSO to predict unplanned readmission, but the difference of c-statistic became smaller as the number of variables was increased and blood-test results were used.
| S-EPMC11335098 | biostudies-literature
Project description:QuestionSevere asthma and COPD exacerbations requiring hospitalization are linked to increased disease morbidity and healthcare costs. We sought to identify Electronic Health Record (EHR) features of severe asthma and COPD exacerbations and evaluate the performance of four machine learning (ML) and one deep learning (DL) model in predicting readmissions using EHR data.Study design and methodsObservational study between September 30, 2012, and December 31, 2017, of patients hospitalized with asthma and COPD exacerbations.ResultsThis study included 5,794 patients, 1,893 with asthma and 3,901 with COPD. Patients with asthma were predominantly female (n = 1288 [68%]), 35% were Black (n = 669), and 25% (n = 479) were Hispanic. Black (44 vs. 33%, p = 0.01) and Hispanic patients (30 vs. 24%, p = 0.02) were more likely to be readmitted for asthma. Similarly, patients with COPD readmissions included a large percentage of Blacks (18 vs. 10%, p < 0.01) and Hispanics (8 vs. 5%, p < 0.01). To identify patients at high risk of readmission index hospitalization data of a subset of 2,682 patients, 777 with asthma and 1,905 with COPD, was analyzed with four ML models, and one DL model. We found that multilayer perceptron, the DL method, had the best sensitivity and specificity compared to the four ML methods implemented in the same dataset.InterpretationMultilayer perceptron, a deep learning method, had the best performance in predicting asthma and COPD readmissions, demonstrating that EHR and deep learning integration can improve high-risk patient detection.
| S-EPMC10720134 | biostudies-literature
Project description:BackgroundPatient safety in the intensive care unit (ICU) is one of the most critical issues, and unplanned extubation (UE) is considered the most adverse event for patient safety. Prevention and early detection of such an event is an essential but difficult component of quality care.ObjectiveThis study aimed to develop and validate prediction models for UE in ICU patients using machine learning.MethodsThis study was conducted in an academic tertiary hospital in Seoul, Republic of Korea. The hospital had approximately 2000 inpatient beds and 120 ICU beds. As of January 2019, the hospital had approximately 9000 outpatients on a daily basis. The number of annual ICU admissions was approximately 10,000. We conducted a retrospective study between January 1, 2010, and December 31, 2018. A total of 6914 extubation cases were included. We developed a UE prediction model using machine learning algorithms, which included random forest (RF), logistic regression (LR), artificial neural network (ANN), and support vector machine (SVM). For evaluating the model's performance, we used the area under the receiver operating characteristic curve (AUROC). The sensitivity, specificity, positive predictive value, negative predictive value, and F1 score were also determined for each model. For performance evaluation, we also used a calibration curve, the Brier score, and the integrated calibration index (ICI) to compare different models. The potential clinical usefulness of the best model at the best threshold was assessed through a net benefit approach using a decision curve.ResultsAmong the 6914 extubation cases, 248 underwent UE. In the UE group, there were more males than females, higher use of physical restraints, and fewer surgeries. The incidence of UE was higher during the night shift as compared to the planned extubation group. The rate of reintubation within 24 hours and hospital mortality were higher in the UE group. The UE prediction algorithm was developed, and the AUROC for RF was 0.787, for LR was 0.762, for ANN was 0.763, and for SVM was 0.740.ConclusionsWe successfully developed and validated machine learning-based prediction models to predict UE in ICU patients using electronic health record data. The best AUROC was 0.787 and the sensitivity was 0.949, which was obtained using the RF algorithm. The RF model was well-calibrated, and the Brier score and ICI were 0.129 and 0.048, respectively. The proposed prediction model uses widely available variables to limit the additional workload on the clinician. Further, this evaluation suggests that the model holds potential for clinical usefulness.
| S-EPMC8387891 | biostudies-literature
Project description:Machine Learning (ML) has been proposed for tissue fate prediction after acute ischemic stroke (AIS), with the aim to help treatment decision and patient management. We compared three different ML models to the clinical method based on diffusion-perfusion thresholding for the voxel-based prediction of final infarct, using a large MRI dataset obtained in a cohort of AIS patients prior to recanalization treatment. Baseline MRI (MRI0), including diffusion-weighted sequence (DWI) and Tmax maps from perfusion-weighted sequence, and 24-hr follow-up MRI (MRI24h) were retrospectively collected in consecutive 394 patients AIS patients (median age = 70 years; final infarct volume = 28mL). Manually segmented DWI24h lesion was considered the final infarct. Gradient Boosting, Random Forests and U-Net were trained using DWI, apparent diffusion coefficient (ADC) and Tmax maps on MRI0 as inputs to predict final infarct. Tissue outcome predictions were compared to final infarct using Dice score. Gradient Boosting had significantly better predictive performance (median [IQR] Dice Score as for median age, maybe you can replace the comma with an equal sign for consistency 0.53 [0.29-0.68]) than U-Net (0.48 [0.18-0.68]), Random Forests (0.51 [0.27-0.66]), and clinical thresholding method (0.45 [0.25-0.62]) (P < 0.001). In this benchmark of ML models for tissue outcome prediction in AIS, Gradient Boosting outperformed other ML models and clinical thresholding method and is thus promising for future decision-making.
| S-EPMC8756479 | biostudies-literature