Dataset Information

Development and Validation of a Machine-Learning Model for Prediction of Extubation Failure in Intensive Care Units.

ABSTRACT: Background: Extubation failure (EF) can lead to an increased chance of ventilator-associated pneumonia, longer hospital stays, and a higher mortality rate. This study aimed to develop and validate an accurate machine-learning model to predict EF in intensive care units (ICUs). Methods: Patients who underwent extubation in the Medical Information Mart for Intensive Care (MIMIC)-IV database were included. EF was defined as the need for ventilatory support (non-invasive ventilation or reintubation) or death within 48 h following extubation. A machine-learning model called Categorical Boosting (CatBoost) was developed based on 89 clinical and laboratory variables. SHapley Additive exPlanations (SHAP) values were calculated to evaluate feature importance and the recursive feature elimination (RFE) algorithm was used to select key features. Hyperparameter optimization was conducted using an automated machine-learning toolkit (Neural Network Intelligence). The final model was trained based on key features and compared with 10 other models. The model was then prospectively validated in patients enrolled in the Cardiac Surgical ICU of Zhongshan Hospital, Fudan University. In addition, a web-based tool was developed to help clinicians use our model. Results: Of 16,189 patients included in the MIMIC-IV cohort, 2,756 (17.0%) had EF. Nineteen key features were selected using the RFE algorithm, including age, body mass index, stroke, heart rate, respiratory rate, mean arterial pressure, peripheral oxygen saturation, temperature, pH, central venous pressure, tidal volume, positive end-expiratory pressure, mean airway pressure, pressure support ventilation (PSV) level, mechanical ventilation (MV) durations, spontaneous breathing trial success times, urine output, crystalloid amount, and antibiotic types. After hyperparameter optimization, our model had the greatest area under the receiver operating characteristic (AUROC: 0.835) in internal validation. Significant differences in mortality, reintubation rates, and NIV rates were shown between patients with a high predicted risk and those with a low predicted risk. In the prospective validation, the superiority of our model was also observed (AUROC: 0.803). According to the SHAP values, MV duration and PSV level were the most important features for prediction. Conclusions: In conclusion, this study developed and prospectively validated a CatBoost model, which better predicted EF in ICUs than other models.

SUBMITTER: Zhao QY

PROVIDER: S-EPMC8165178 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundExtubation is the process of removing tracheal tubes so that patients maintain oxygenation while they start to breathe spontaneously. However, hypoxemia after extubation is an important issue for critical care doctors and is associated with patients' oxygenation, circulation, recovery, and incidence of postoperative complications. Accuracy and specificity of most related conventional models remain unsatisfactory. We conducted a predictive analysis based on a supervised machine-learning algorithm for the precise prediction of hypoxemia after extubation in intensive care units (ICUs).MethodsData were extracted from the Medical Information Mart for Intensive Care (MIMIC)-IV database for patients over age 18 who underwent mechanical ventilation in the ICU. The primary outcome was hypoxemia after extubation, and it was defined as a partial pressure of oxygen <60 mmHg after extubation. Variables and individuals with missing values greater than 20% were excluded, and the remaining missing values were filled in using multiple imputation. The dataset was split into a training set (80%) and final test set (20%). All related clinical and laboratory variables were extracted, and logistics stepwise regression was performed to screen out the key features. Six different advanced machine-learning models, including logistics regression (LOG), random forest (RF), K-nearest neighbors (KNN), support-vector machine (SVM), eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM), were introduced for modelling. The best performance model in the first cross-validated dataset was further fine-tuned, and the final performance was assessed using the final test set.ResultsA total of 14,777 patients were included in the study, and 1,864 of the patients' experienced hypoxemia after extubation. After training, the RF and LightGBM models were the strongest initial performers, and the area under the curve (AUC) using RF was 0.780 [95% confidence interval (CI), 0.755-0.805] and using LightGBM was 0.779 (95% CI, 0.752-0.806). The final AUC using RF was 0.792 (95% CI, 0.771-0.814) and using LightGBM was 0.792 (95% CI, 0.770-0.815).ConclusionsOur machine learning models have considerable potential for predicting hypoxemia after extubation, which help to reduce ICU morbidity and mortality.

Project description:BackgroundPatient safety in the intensive care unit (ICU) is one of the most critical issues, and unplanned extubation (UE) is considered the most adverse event for patient safety. Prevention and early detection of such an event is an essential but difficult component of quality care.ObjectiveThis study aimed to develop and validate prediction models for UE in ICU patients using machine learning.MethodsThis study was conducted in an academic tertiary hospital in Seoul, Republic of Korea. The hospital had approximately 2000 inpatient beds and 120 ICU beds. As of January 2019, the hospital had approximately 9000 outpatients on a daily basis. The number of annual ICU admissions was approximately 10,000. We conducted a retrospective study between January 1, 2010, and December 31, 2018. A total of 6914 extubation cases were included. We developed a UE prediction model using machine learning algorithms, which included random forest (RF), logistic regression (LR), artificial neural network (ANN), and support vector machine (SVM). For evaluating the model's performance, we used the area under the receiver operating characteristic curve (AUROC). The sensitivity, specificity, positive predictive value, negative predictive value, and F1 score were also determined for each model. For performance evaluation, we also used a calibration curve, the Brier score, and the integrated calibration index (ICI) to compare different models. The potential clinical usefulness of the best model at the best threshold was assessed through a net benefit approach using a decision curve.ResultsAmong the 6914 extubation cases, 248 underwent UE. In the UE group, there were more males than females, higher use of physical restraints, and fewer surgeries. The incidence of UE was higher during the night shift as compared to the planned extubation group. The rate of reintubation within 24 hours and hospital mortality were higher in the UE group. The UE prediction algorithm was developed, and the AUROC for RF was 0.787, for LR was 0.762, for ANN was 0.763, and for SVM was 0.740.ConclusionsWe successfully developed and validated machine learning-based prediction models to predict UE in ICU patients using electronic health record data. The best AUROC was 0.787 and the sensitivity was 0.949, which was obtained using the RF algorithm. The RF model was well-calibrated, and the Brier score and ICI were 0.129 and 0.048, respectively. The proposed prediction model uses widely available variables to limit the additional workload on the clinician. Further, this evaluation suggests that the model holds potential for clinical usefulness.

Project description:BackgroundNoninvasive ventilation (NIV) has been widely used in critically ill patients after extubation. However, NIV failure is associated with poor outcomes. This study aimed to determine early predictors of NIV failure and to construct an accurate machine-learning model to identify patients at risks of NIV failure after extubation in intensive care units (ICUs).MethodsPatients who underwent NIV after extubation in the eICU Collaborative Research Database (eICU-CRD) were included. NIV failure was defined as need for invasive ventilatory support (reintubation or tracheotomy) or death after NIV initiation. A total of 93 clinical and laboratory variables were assessed, and the recursive feature elimination algorithm was used to select key features. Hyperparameter optimization was conducted with an automated machine-learning toolkit called Neural Network Intelligence. A machine-learning model called Categorical Boosting (CatBoost) was developed and compared with nine other models. The model was then prospectively validated among patients enrolled in the Cardiac Surgical ICU of Zhongshan Hospital, Fudan University.ResultsOf 929 patients included in the eICU-CRD cohort, 248 (26.7%) had NIV failure. The time from extubation to NIV, age, Glasgow Coma Scale (GCS) score, heart rate, respiratory rate, mean blood pressure (MBP), saturation of pulse oxygen (SpO2), temperature, glucose, pH, pressure of oxygen in blood (PaO2), urine output, input volume, ventilation duration, and mean airway pressure were selected. After hyperparameter optimization, our model showed the greatest accuracy in predicting NIV failure (AUROC: 0.872 [95% CI 0.82-0.92]) among all predictive methods in an internal validation. In the prospective validation cohort, our model was also superior (AUROC: 0.846 [95% CI 0.80-0.89]). The sensitivity and specificity in the prediction group is 89% and 75%, while in the validation group they are 90% and 70%. MV duration and respiratory rate were the most important features. Additionally, we developed a web-based tool to help clinicians use our model.ConclusionsThis study developed and prospectively validated the CatBoost model, which can be used to identify patients who are at risk of NIV failure. Thus, those patients might benefit from early triage and more intensive monitoring.Trial registrationNCT03704324. Registered 1 September 2018, https://register.Clinicaltrialsgov .

Project description:Hypothermia is a life-threatening condition where the temperature of the body drops below 35°C and is a key source of concern in Intensive Care Units (ICUs). Early identification can help to nudge clinical management to initiate early interventions. Despite its importance, very few studies have focused on the early prediction of hypothermia. In this study, we aim to monitor and predict Hypothermia (30 min-4 h) ahead of its onset using machine learning (ML) models developed on physiological vitals and to prospectively validate the best performing model in the pediatric ICU. We developed and evaluated ML algorithms for the early prediction of hypothermia in a pediatric ICU. Sepsis advanced forecasting engine ICU Database (SafeICU) data resource is an in-house ICU source of data built in the Pediatric ICU at the All-India Institute of Medical Science (AIIMS), New Delhi. Each time-stamp at 1-min resolution was labeled for the presence of hypothermia to construct a retrospective cohort of pediatric patients in the SafeICU data resource. The training set consisted of windows of the length of 4.2 h with a lead time of 30 min-4 h from the onset of hypothermia. A set of 3,835 hand-engineered time-series features were calculated to capture physiological features from the time series. Features selection using the Boruta algorithm was performed to select the most important predictors of hypothermia. A battery of models such as gradient boosting machine, random forest, AdaBoost, and support vector machine (SVM) was evaluated utilizing five-fold test sets. The best-performing model was prospectively validated. A total of 148 patients with 193 ICU stays were eligible for the model development cohort. Of 3,939 features, 726 were statistically significant in the Boruta analysis for the prediction of Hypothermia. The gradient boosting model performed best with an Area Under the Receiver Operating Characteristic curve (AUROC) of 85% (SD = 1.6) and a precision of 59.2% (SD = 8.8) for a 30-min lead time before the onset of Hypothermia onset. As expected, the model showed a decline in model performance at higher lead times, such as AUROC of 77.2% (SD = 2.3) and precision of 41.34% (SD = 4.8) for 4 h ahead of Hypothermia onset. Our GBM(gradient boosting machine) model produced equal and superior results for the prospective validation, where an AUROC of 79.8% and a precision of 53% for a 30-min lead time before the onset of Hypothermia whereas an AUROC of 69.6% and a precision of 38.52% for a (30 min-4 h) lead time prospective validation of Hypothermia. Therefore, this work establishes a pipeline termed ThermoGnose for predicting hypothermia, a major complication in pediatric ICUs.

Project description:BACKGROUND:Prognostication is an essential tool for risk adjustment and decision making in the intensive care unit (ICU). Research into prognostication in ICU has so far been limited to data from admission or the first 24 hours. Most ICU admissions last longer than this, decisions are made throughout an admission, and some admissions are explicitly intended as time-limited prognostic trials. Despite this, temporal changes in prognostic ability during ICU admission has received little attention to date. Current predictive models, in the form of prognostic clinical tools, are typically derived from linear models and do not explicitly handle incremental information from trends. Machine learning (ML) allows predictive models to be developed which use non-linear predictors and complex interactions between variables, thus allowing incorporation of trends in measured variables over time; this has made it possible to investigate prognosis throughout an admission. METHODS AND FINDINGS:This study uses ML to assess the predictability of ICU mortality as a function of time. Logistic regression against physiological data alone outperformed APACHE-II and demonstrated several important interactions including between lactate & noradrenaline dose, between lactate & MAP, and between age & MAP consistent with the current sepsis definitions. ML models consistently outperformed logistic regression with Deep Learning giving the best results. Predictive power was maximal on the second day and was further improved by incorporating trend data. Using a limited range of physiological and demographic variables, the best machine learning model on the first day showed an area under the receiver-operator characteristic curve (AUC) of 0.883 (σ = 0.008), compared to 0.846 (σ = 0.010) for a logistic regression from the same predictors and 0.836 (σ = 0.007) for a logistic regression based on the APACHE-II score. Adding information gathered on the second day of admission improved the maximum AUC to 0.895 (σ = 0.008). Beyond the second day, predictive ability declined. CONCLUSION:This has implications for decision making in intensive care and provides a justification for time-limited trials of ICU therapy; the assessment of prognosis over more than one day may be a valuable strategy as new information on the second day helps to differentiate outcomes. New ML models based on trend data beyond the first day could greatly improve upon current risk stratification tools.

Dataset Information

Development and Validation of a Machine-Learning Model for Prediction of Extubation Failure in Intensive Care Units.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets