Dataset Information

Predicting Intensive Care Unit Readmission with Machine Learning Using Electronic Health Record Data.

ABSTRACT:

Rationale

Patients transferred from the intensive care unit to the wards who are later readmitted to the intensive care unit have increased length of stay, healthcare expenditure, and mortality compared with those who are never readmitted. Improving risk stratification for patients transferred to the wards could have important benefits for critically ill hospitalized patients.

Objectives

We aimed to use a machine-learning technique to derive and validate an intensive care unit readmission prediction model with variables available in the electronic health record in real time and compare it to previously published algorithms.

Methods

This observational cohort study was conducted at an academic hospital in the United States with approximately 600 inpatient beds. A total of 24,885 intensive care unit transfers to the wards were included, with 14,962 transfers (60%) in the training cohort and 9,923 transfers (40%) in the internal validation cohort. Patient characteristics, nursing assessments, International Classification of Diseases, Ninth Revision codes from prior admissions, medications, intensive care unit interventions, diagnostic tests, vital signs, and laboratory results were extracted from the electronic health record and used as predictor variables in a gradient-boosted machine model. Accuracy for predicting intensive care unit readmission was compared with the Stability and Workload Index for Transfer score and Modified Early Warning Score in the internal validation cohort and also externally using the Medical Information Mart for Intensive Care database (n = 42,303 intensive care unit transfers).

Results

Eleven percent (2,834) of discharges to the wards were later readmitted to the intensive care unit. The machine-learning-derived model had significantly better performance (area under the receiver operating curve, 0.76) than either the Stability and Workload Index for Transfer score (area under the receiver operating curve, 0.65), or Modified Early Warning Score (area under the receiver operating curve, 0.58; P value < 0.0001 for all comparisons). At a specificity of 95%, the derived model had a sensitivity of 28% compared with 15% for Stability and Workload Index for Transfer score and 7% for the Modified Early Warning Score. Accuracy improvements with the derived model over Modified Early Warning Score and Stability and Workload Index for Transfer were similar in the Medical Information Mart for Intensive Care-III cohort.

Conclusions

A machine learning approach to predicting intensive care unit readmission was significantly more accurate than previously published algorithms in both our internal validation and the Medical Information Mart for Intensive Care-III cohort. Implementation of this approach could target patients who may benefit from additional time in the intensive care unit or more frequent monitoring after transfer to the hospital ward.

SUBMITTER: Rojas JC

PROVIDER: S-EPMC6207111 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundRapid antibiotic administration is known to improve sepsis outcomes, however early diagnosis remains challenging due to complex presentation. Our objective was to develop a model using readily available electronic health record (EHR) data capable of recognizing infant sepsis at least 4 hours prior to clinical recognition.Methods and findingsWe performed a retrospective case control study of infants hospitalized ≥48 hours in the Neonatal Intensive Care Unit (NICU) at the Children's Hospital of Philadelphia between September 2014 and November 2017 who received at least one sepsis evaluation before 12 months of age. We considered two evaluation outcomes as cases: culture positive-positive blood culture for a known pathogen (110 evaluations); and clinically positive-negative cultures but antibiotics administered for ≥120 hours (265 evaluations). Case data was taken from the 44-hour window ending 4 hours prior to evaluation. We randomly sampled 1,100 44-hour windows of control data from all times ≥10 days removed from any evaluation. Model inputs consisted of up to 36 features derived from routine EHR data. Using 10-fold nested cross-validation, 8 machine learning models were trained to classify inputs as sepsis positive or negative. When tasked with discriminating culture positive cases from controls, 6 models achieved a mean area under the receiver operating characteristic (AUC) between 0.80-0.82 with no significant differences between them. Including both culture and clinically positive cases, the same 6 models achieved an AUC between 0.85-0.87, again with no significant differences.ConclusionsMachine learning models can identify infants with sepsis in the NICU hours prior to clinical recognition. Learning curves indicate model improvement may be achieved with additional training examples. Additional input features may also improve performance. Further research is warranted to assess potential performance improvements and clinical efficacy in a prospective trial.

Project description:OBJECTIVES:Unplanned readmissions to the intensive care unit (ICU) are highly undesirable, increasing variance in care, making resource planning difficult and potentially increasing length of stay and mortality in some settings. Identifying patients who are likely to suffer unplanned ICU readmission could reduce the frequency of this adverse event. SETTING:A single academic, tertiary care hospital in the UK. PARTICIPANTS:A set of 3326 ICU episodes collected between October 2014 and August 2016. All records were of patients who visited an ICU at some point during their stay. We excluded patients who were ?16 years of age; visited ICUs other than the general and neurosciences ICU; were missing crucial electronic patient record measurements; or had indeterminate ICU discharge outcomes or very early or extremely late discharge times. After exclusion, 2018 outcome-labelled episodes remained. PRIMARY AND SECONDARY OUTCOME MEASURES:Area under the receiver operating characteristic curve (AUROC) for prediction of unplanned ICU readmission or in-hospital death within 48?hours of first ICU discharge. RESULTS:In 10-fold cross-validation, an ensemble predictor was trained on data from both the target hospital and the Medical Information Mart for Intensive Care (MIMIC-III) database and tested on the target hospital's data. This predictor discriminated between patients with the unplanned ICU readmission or death outcome and those without this outcome, attaining mean AUROC of 0.7095 (SE 0.0260), superior to the purpose-built Stability and Workload Index for Transfer (SWIFT) score (AUROC=0.6082, SE 0.0249; p=0.014, pairwise t-test). CONCLUSIONS:Despite the inherent difficulties, we demonstrate that a novel machine learning algorithm based on transfer learning could achieve good discrimination, over and above that of the treating clinicians or the value added by the SWIFT score. Accurate prediction of unplanned readmission could be used to target resources more efficiently.

Project description:IntroductionSeptic patients requiring intensive care unit (ICU) readmission are at high risk of mortality, but research focusing on the association of ICU readmission due to sepsis and mortality is limited. The aim of this study was to develop and validate a machine learning (ML) model for predicting in-hospital mortality in septic patients readmitted to the ICU using routinely available clinical data.MethodsThe data used in this study were obtained from the Medical Information Mart for Intensive Care (MIMIC-IV, v1.0) database, between 2008 and 2019. The study cohort included patients with sepsis requiring ICU readmission. The data were randomly split into a training (75%) data set and a validation (25%) data set. Nine popular ML models were developed to predict mortality in septic patients readmitted to the ICU. The model with the best accuracy and area under the curve (A.C.) in the validation cohort was defined as the optimal model and was selected for further prediction studies. The SHAPELY Additive explanations (SHAP) values and Local Interpretable Model-Agnostic Explanation (LIME) methods were used to improve the interpretability of the optimal model.ResultsA total of 1117 septic patients who had required ICU readmission during the study period were enrolled in the study. Of these participants, 434 (38.9%) were female, and the median (interquartile range [IQR]) age was 68.6 (58.4-79.2) years. The median (IQR) ICU interval duration was 2.60 (0.64-5.78) days. After feature selection, 31 of 47 clinical factors were ultimately chosen for use in model construction. Of the nine ML models tested, the best performance was achieved with the random forest (RF) model, with an A.C. of 0.81, an accuracy of 85% and a precision of 62% in the validation cohort. The SHAP summary analysis revealed that Glasgow Coma Scale score, urine output, blood urea nitrogen, lactate, platelet count and systolic blood pressure were the top six most important factors contributing to the RF model. Additionally, the LIME method demonstrated how the RF model works in terms of explaining risk of death prediction in septic patients requiring ICU readmission.ConclusionThe ML models reported here showed a good prognostic prediction ability in septic patients requiring ICU readmission. Of the features selected, the parameters related to organ perfusion made the largest contribution to outcome prediction during ICU readmission in septic patients.

Project description:BackgroundIntensive care units (ICUs) face financial, bed management, and staffing constraints. Detailed data covering all aspects of patients' journeys into and through intensive care are now collected and stored in electronic health records: machine learning has been used to analyse such data in order to provide decision support to clinicians.MethodsSystematic review of the applications of machine learning to routinely collected ICU data. Web of Science and MEDLINE databases were searched to identify candidate articles: those on image processing were excluded. The study aim, the type of machine learning used, the size of dataset analysed, whether and how the model was validated, and measures of predictive accuracy were extracted.ResultsOf 2450 papers identified, 258 fulfilled eligibility criteria. The most common study aims were predicting complications (77 papers [29.8% of studies]), predicting mortality (70 [27.1%]), improving prognostic models (43 [16.7%]), and classifying sub-populations (29 [11.2%]). Median sample size was 488 (IQR 108-4099): 41 studies analysed data on > 10,000 patients. Analyses focused on 169 (65.5%) papers that used machine learning to predict complications, mortality, length of stay, or improvement of health. Predictions were validated in 161 (95.2%) of these studies: the area under the ROC curve (AUC) was reported by 97 (60.2%) but only 10 (6.2%) validated predictions using independent data. The median AUC was 0.83 in studies of 1000-10,000 patients, rising to 0.94 in studies of > 100,000 patients. The most common machine learning methods were neural networks (72 studies [42.6%]), support vector machines (40 [23.7%]), and classification/decision trees (34 [20.1%]). Since 2015 (125 studies [48.4%]), the most common methods were support vector machines (37 studies [29.6%]) and random forests (29 [23.2%]).ConclusionsThe rate of publication of studies using machine learning to analyse routinely collected ICU data is increasing rapidly. The sample sizes used in many published studies are too small to exploit the potential of these methods. Methodological and reporting guidelines are needed, particularly with regard to the choice of method and validation of predictions, to increase confidence in reported findings and aid in translating findings towards routine use in clinical practice.

Project description:BackgroundPoor functional status is a key marker of morbidity, yet is not routinely captured in clinical encounters. We developed and evaluated the accuracy of a machine learning algorithm that leveraged electronic health record (EHR) data to provide a scalable process for identification of functional impairment.MethodsWe identified a cohort of patients with an electronically captured screening measure of functional status (Older Americans Resources and Services ADL/IADL) between 2018 and 2020 (N = 6484). Patients were classified using unsupervised learning K means and t-distributed Stochastic Neighbor Embedding into normal function (NF), mild to moderate functional impairment (MFI), and severe functional impairment (SFI) states. Using 11 EHR clinical variable domains (832 variable input features), we trained an Extreme Gradient Boosting supervised machine learning algorithm to distinguish functional status states, and measured prediction accuracies. Data were randomly split into training (80%) and test (20%) sets. The SHapley Additive Explanations (SHAP) feature importance analysis was used to list the EHR features in rank order of their contribution to the outcome.ResultsMedian age was 75.3 years, 62% female, 60% White. Patients were classified as 53% NF (n = 3453), 30% MFI (n = 1947), and 17% SFI (n = 1084). Summary of model performance for identifying functional status state (NF, MFI, SFI) was AUROC (area under the receiving operating characteristic curve) 0.92, 0.89, and 0.87, respectively. Age, falls, hospitalization, home health use, labs (e.g., albumin), comorbidities (e.g., dementia, heart failure, chronic kidney disease, chronic pain), and social determinants of health (e.g., alcohol use) were highly ranked features in predicting functional status states.ConclusionA machine learning algorithm run on EHR clinical data has potential utility for differentiating functional status in the clinical setting. Through further validation and refinement, such algorithms can complement traditional screening methods and result in a population-based strategy for identifying patients with poor functional status who need additional health resources.