Dataset Information

Predicting mortality among septic patients presenting to the emergency department-a cross sectional analysis using machine learning.

ABSTRACT:

Background

Sepsis is a life-threatening condition, causing almost one fifth of all deaths worldwide. The aim of the current study was to identify variables predictive of 7- and 30-day mortality among variables reflective of the presentation of septic patients arriving to the emergency department (ED) using machine learning.

Methods

Retrospective cross-sectional design, including all patients arriving to the ED at Södersjukhuset in Sweden during 2013 and discharged with an International Classification of Diseases (ICD)-10 code corresponding to sepsis. All predictions were made using a Balanced Random Forest Classifier and 91 variables reflecting ED presentation. An exhaustive search was used to remove unnecessary variables in the final model. A 10-fold cross validation was performed and the accuracy was described using the mean value of the following: AUC, sensitivity, specificity, PPV, NPV, positive LR and negative LR.

Results

The study population included 445 septic patients, randomised to a training (n = 356, 80%) and a validation set (n = 89, 20%). The six most important variables for predicting 7-day mortality were: "fever", "abnormal verbal response", "low saturation", "arrival by emergency medical services (EMS)", "abnormal behaviour or level of consciousness" and "chills". The model including these variables had an AUC of 0.83 (95% CI: 0.80-0.86). The final model predicting 30-day mortality used similar six variables, however, including "breathing difficulties" instead of "abnormal behaviour or level of consciousness". This model achieved an AUC = 0.80 (CI 95%, 0.78-0.82).

Conclusions

The results suggest that six specific variables were predictive of 7- and 30-day mortality with good accuracy which suggests that these symptoms, observations and mode of arrival may be important components to include along with vital signs in a future prediction tool of mortality among septic patients presenting to the ED. In addition, the Random Forests appears to be a suitable machine learning method on which to build future studies.

SUBMITTER: Karlsson A

PROVIDER: S-EPMC8276466 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.

Project description:BackgroundUrinary tract infection (UTI) is a common emergency department (ED) diagnosis with reported high diagnostic error rates. Because a urine culture, part of the gold standard for diagnosis of UTI, is usually not available for 24-48 hours after an ED visit, diagnosis and treatment decisions are based on symptoms, physical findings, and other laboratory results, potentially leading to overutilization, antibiotic resistance, and delayed treatment. Previous research has demonstrated inadequate diagnostic performance for both individual laboratory tests and prediction tools.ObjectiveOur aim, was to train, validate, and compare machine-learning based predictive models for UTI in a large diverse set of ED patients.MethodsSingle-center, multi-site, retrospective cohort analysis of 80,387 adult ED visits with urine culture results and UTI symptoms. We developed models for UTI prediction with six machine learning algorithms using demographic information, vitals, laboratory results, medications, past medical history, chief complaint, and structured historical and physical exam findings. Models were developed with both the full set of 211 variables and a reduced set of 10 variables. UTI predictions were compared between models and to proxies of provider judgment (documentation of UTI diagnosis and antibiotic administration).ResultsThe machine learning models had an area under the curve ranging from 0.826-0.904, with extreme gradient boosting (XGBoost) the top performing algorithm for both full and reduced models. The XGBoost full and reduced models demonstrated greatly improved specificity when compared to the provider judgment proxy of UTI diagnosis OR antibiotic administration with specificity differences of 33.3 (31.3-34.3) and 29.6 (28.5-30.6), while also demonstrating superior sensitivity when compared to documentation of UTI diagnosis with sensitivity differences of 38.7 (38.1-39.4) and 33.2 (32.5-33.9). In the admission and discharge cohorts using the full XGboost model, approximately 1 in 4 patients (4109/15855) would be re-categorized from a false positive to a true negative and approximately 1 in 11 patients (1372/15855) would be re-categorized from a false negative to a true positive.ConclusionThe best performing machine learning algorithm, XGBoost, accurately diagnosed positive urine culture results, and outperformed previously developed models in the literature and several proxies for provider judgment. Future prospective validation is warranted.

Project description:Emergency department triage is the first point in time when a patient's acuity level is determined. The time to assign a priority at triage is short and it is vital to accurately stratify patients at this stage, since under-triage can lead to increased morbidity, mortality and costs. Our aim was to present a model that can assist healthcare professionals in triage decision making, namely in the stratification of patients through the risk prediction of a composite critical outcome-mortality and cardiopulmonary arrest. Our study cohort consisted of 235826 adult patients triaged at a Portuguese Emergency Department from 2012 to 2016. Patients were assigned to emergent, very urgent or urgent priorities of the Manchester Triage System (MTS). Demographics, clinical variables routinely collected at triage and the patients' chief complaint were used. Logistic regression, random forests and extreme gradient boosting were developed using all available variables. The term frequency-inverse document frequency (TF-IDF) natural language processing weighting factor was applied to vectorize the chief complaint. Stratified random sampling was used to split the data into train (70%) and test (30%) data sets. Ten-fold cross validation was performed in train to optimize model hyper-parameters. The performance obtained with the best model was compared against the reference model-a regularized logistic regression trained using only triage priorities. Extreme gradient boosting exhibited good calibration properties and yielded areas under the receiver operating characteristic and precision-recall curves of 0.96 (95% CI 0.95-0.97) and 0.31 (95% CI 0.26-0.36), respectively. The predictors ranked with higher importance by this model were the Glasgow coma score, the patients' age, pulse oximetry and arrival mode. Compared to the reference, the extreme gradient boosting model using clinical variables and the chief complaint presented higher recall for patients assigned MTS-3 and can identify those who are at risk of the composite outcome.

Project description:ObjectivesTo predict 72-h and 9-day emergency department (ED) return by using gradient boosting on an expansive set of clinical variables from the electronic health record.MethodsThis retrospective study included all adult discharges from a level 1 trauma center ED and a community hospital ED covering the period of March 2013 to July 2017. A total of 1500 variables were extracted for each visit, and samples split randomly into training, validation, and test sets (80%, 10%, and 10%). Gradient boosting models were fit on 3 selections of the data: administrative data (demographics, prior hospital usage, and comorbidity categories), data available at triage, and the full set of data available at discharge. A logistic regression (LR) model built on administrative data was used for baseline comparison. Finally, the top 20 most informative variables identified from the full gradient boosting models were used to build a reduced model for each outcome.ResultsA total of 330 631 discharges were available for analysis, with 29 058 discharges (8.8%) resulting in 72-h return and 52 748 discharges (16.0%) resulting in 9-day return to either ED. LR models using administrative data yielded test AUCs of 0.69 (95% confidence interval [CI] 0.68-0.70) and 0.71(95% CI 0.70-0.72), while gradient boosting models using administrative data yielded test AUCs of 0.73 (95% CI 0.72-0.74) and 0.74 (95% CI 0.73-0.74) for 72-h and 9-day return, respectively. Gradient boosting models using variables available at triage yielded test AUCs of 0.75 (95% CI 0.74-0.76) and 0.75 (95% CI 0.74-0.75), while those using the full set of variables yielded test AUCs of 0.76 (95% CI 0.75-0.77) and 0.75 (95% CI 0.75-0.76). Reduced models using the top 20 variables yielded test AUCs of 0.73 (95% CI 0.71-0.74) and 0.73 (95% CI 0.72-0.74).Discussion and conclusionGradient boosting models leveraging clinical data are superior to LR models built on administrative data at predicting 72-h and 9-day returns.

Project description:BackgroundEmergency departments (ED) are becoming increasingly overwhelmed, increasing poor outcomes. Triage scores aim to optimize the waiting time and prioritize the resource usage. Artificial intelligence (AI) algorithms offer advantages for creating predictive clinical applications.ObjectiveEvaluate a state-of-the-art machine learning model for predicting mortality at the triage level and, by validating this automatic tool, improve the categorization of patients in the ED.DesignAn institutional review board (IRB) approval was granted for this retrospective study. Information of consecutive adult patients (ages 18-100) admitted at the emergency department (ED) of one hospital were retrieved (January 1, 2012-December 31, 2018). Features included the following: demographics, admission date, arrival mode, referral code, chief complaint, previous ED visits, previous hospitalizations, comorbidities, home medications, vital signs, and Emergency Severity Index (ESI). The following outcomes were evaluated: early mortality (up to 2 days post ED registration) and short-term mortality (2-30 days post ED registration). A gradient boosting model was trained on data from years 2012-2017 and examined on data from the final year (2018). The area under the curve (AUC) for mortality prediction was used as an outcome metric. Single-variable analysis was conducted to develop a nine-point triage score for early mortality.Key resultsOverall, 799,522 ED visits were available for analysis. The early and short-term mortality rates were 0.6% and 2.5%, respectively. Models trained on the full set of features yielded an AUC of 0.962 for early mortality and 0.923 for short-term mortality. A model that utilized the nine features with the highest single-variable AUC scores (age, arrival mode, chief complaint, five primary vital signs, and ESI) yielded an AUC of 0.962 for early mortality.ConclusionThe gradient boosting model shows high predictive ability for screening patients at risk of early mortality utilizing data available at the time of triage in the ED.

Project description:BackgroundMany prediction models have been developed to help identify emergency department (ED) patients at high risk of poor outcome. However, these models often underperform in clinical practice and their actual clinical impact has hardly ever been evaluated. We aim to perform a clinical trial to investigate the clinical impact of a prediction model based on machine learning (ML) technology.MethodsThe study is a prospective, randomized, open-label, non-inferiority pilot clinical trial. We will investigate the clinical impact of a prediction model based on ML technology, the RISKINDEX, which has been developed to predict the risk of 31-day mortality based on the results of laboratory tests and demographic characteristics. In previous studies, the RISKINDEX was shown to outperform internal medicine specialists and to have high discriminatory performance. Adults patients (18 years or older) will be recruited in the ED. All participants will be randomly assigned to the control group or the intervention group in a 1:1 ratio. Participants in the control group will receive care as usual in which the study team asks the attending physicians questions about their clinical intuition. Participants in the intervention group will also receive care as usual, but in addition to asking the clinical impression questions, the study team presents the RISKINDEX to the attending physician in order to assess the extent to which clinical treatment is influenced by the results.DiscussionThis pilot clinical trial investigates the clinical impact and implementation of an ML based prediction model in the ED. By assessing the clinical impact and prognostic accuracy of the RISKINDEX, this study aims to contribute valuable insights to optimize patient care and inform future research in the field of ML based clinical prediction models.Trial registrationClinicalTrials.gov NCT05497830. Machine Learning for Risk Stratification in the Emergency Department (MARS-ED). Registered on August 11, 2022. URL: https://clinicaltrials.gov/study/NCT05497830 .