Dataset Information

Prediction of Critical Care Outcome for Adult Patients Presenting to Emergency Department Using Initial Triage Information: An XGBoost Algorithm Analysis.

ABSTRACT:

Background

The emergency department (ED) triage system to classify and prioritize patients from high risk to less urgent continues to be a challenge.

Objective

This study, comprising 80,433 patients, aims to develop a machine learning algorithm prediction model of critical care outcomes for adult patients using information collected during ED triage and compare the performance with that of the baseline model using the Korean Triage and Acuity Scale (KTAS).

Methods

To predict the need for critical care, we used 13 predictors from triage information: age, gender, mode of ED arrival, the time interval between onset and ED arrival, reason of ED visit, chief complaints, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, oxygen saturation, and level of consciousness. The baseline model with KTAS was developed using logistic regression, and the machine learning model with 13 variables was generated using extreme gradient boosting (XGB) and deep neural network (DNN) algorithms. The discrimination was measured by the area under the receiver operating characteristic (AUROC) curve. The ability of calibration with Hosmer-Lemeshow test and reclassification with net reclassification index were evaluated. The calibration plot and partial dependence plot were used in the analysis.

Results

The AUROC of the model with the full set of variables (0.833-0.861) was better than that of the baseline model (0.796). The XGB model of AUROC 0.861 (95% CI 0.848-0.874) showed a higher discriminative performance than the DNN model of 0.833 (95% CI 0.819-0.848). The XGB and DNN models proved better reclassification than the baseline model with a positive net reclassification index. The XGB models were well-calibrated (Hosmer-Lemeshow test; P>.05); however, the DNN showed poor calibration power (Hosmer-Lemeshow test; P<.001). We further interpreted the nonlinear association between variables and critical care prediction.

Conclusions

Our study demonstrated that the performance of the XGB model using initial information at ED triage for predicting patients in need of critical care outperformed the conventional model with KTAS.

SUBMITTER: Yun H

PROVIDER: S-EPMC8491120 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Study objectiveTriage quickly identifies critically ill patients, facilitating timely interventions. Many emergency departments (EDs) use emergency severity index (ESI) or abnormal vital sign triggers to guide triage. However, both use fixed thresholds, and false activations are costly. Prior approaches using machinelearning have relied on information that is often unavailable during the triage process. We examined whether deep-learning approaches could identify critically ill patients only using data immediately available at triage.MethodsWe conducted a retrospective, cross-sectional study at an urban tertiary care center, from January 1, 2012-January 1, 2020. De-identified triage information included structured (age, sex, initial vital signs) and textual (chief complaint) data, with critical illness (mortality or ICU admission within 24 hours) as the outcome. Four progressively complex deep-learning models were trained and applied to triage information from all patients. We compared the accuracy of the models against ESI as the standard diagnostic test, using area under the receiver-operator curve (AUC).ResultsA total of 445,925 patients were included, with 60,901 (13.7%) critically ill. Vital sign thresholds identified critically ill patients with AUC 0.521 (95% confidence interval [CI] = 0.519-0.522), and ESI <3 demonstrated AUC 0.672 (95% CI = 0.671-0.674), logistic regression classified patients with AUC 0.803 (95% CI = 0.802-0.804), 2-layer neural network with structured data with AUC 0.811 (95% CI = 0.807-0.815), gradient tree boosting with AUC 0.820 (95% CI = 0.818-0.821), and the neural network model with textual data with AUC 0.851 (95% CI = 0.849-0.852). All successive increases in AUC were statistically significant.ConclusionDeep-learning techniques represent a promising method of augmenting triage, even with limited information. Further research is needed to determine if improved predictions yield clinical and operational benefits.

Project description:OBJECTIVE:To investigate the appropriateness of cases presenting to the emergency department (ED) following ambulance-based secondary telephone triage. DESIGN:A pragmatic retrospective cohort analysis of all the planned and unplanned ED presentations within 48 hours of a secondary telephone triage. SETTING:The secondary telephone triage service, called the Referral Service, and the hospitals were located in metropolitan Melbourne, Australia and operated 24 hours a day, servicing 4.25 million people. The Referral Service provides an in-depth secondary triage of cases classified as low acuity when calling the Australian emergency telephone number. POPULATION:Cases triaged by the Referral Service between September 2009 and June 2012 were linked to ED and hospital admission records (N=44,523). Planned ED presentations were cases referred to the ED following the secondary triage, unplanned ED presentations were cases that presented despite being referred to alternative care pathways. MAIN OUTCOME MEASURES:Appropriateness was measured using an ED suitability definition and hospital admission rates. These were compared with mean population data which consisted of all of the ED presentations for the state (termed the 'average Victorian ED presentation'). RESULTS:Planned ED presentations were more likely to be ED suitable than unplanned ED presentations (OR 1.62; 95% CI 1.5 to 1.7; p<0.001) and the average Victorian ED presentation (OR 1.85; 95% CI 1.01 to 3.4; p=0.046). They were also more likely to be admitted to the hospital than the unplanned ED presentation (OR 1.5; 95% CI 1.4 to 1.6; p<0.001) and the average Victorian ED presentation (OR 2.3, 95% CI 2.24 to 2.33; p<0.001). Just under 15% of cases diverted away from the emergency care pathways presented in the ED (unplanned ED attendances), and 9.5% of all the alternative care pathway cases were classified as ED suitable and 6.5% were admitted to hospital. CONCLUSIONS:Secondary telephone triage was able to appropriately identify many ED suitable cases, and while most cases referred to alternative care pathways did not present in the ED. Further research is required to establish that these were not inappropriately triaged away from the emergency care pathways.

Project description:IntroductionCanadian patients presenting to the emergency department (ED) typically undergo a triage process where they are assessed by a specially trained nurse and assigned a Canadian Triage and Acuity Scale (CTAS) score, indicating their level of acuity and urgency of assessment. We sought to assess the ability of patients to self-triage themselves through use of one of two of our proprietary self-triage tools, and how this would compare with the standard nurse-driven triage process.MethodsWe enrolled a convenience sample of ambulatory ED patients aged 17 years or older who presented with chief complaints of chest pain, abdominal pain, breathing problems, or musculoskeletal pain. Participants completed one, or both, of an algorithm generated self-triage (AGST) survey, or visual acuity scale (VAS) based self-triage tool which subsequently generated a CTAS score. Our primary outcome was to assess the accuracy of these tools to the CTAS score generated through the nurse-driven triage process.ResultsA total of 223 patients were included in our analysis. Of these, 32 (14.3%) presented with chest pain, 25 (11.2%) with shortness of breath, 75 (33.6%) with abdominal pain, and 91 (40.8%) with musculoskeletal pain. Of the total number of patients, 142 (47.2%) completed the AGST tool, 159 (52.8%) completed the VAS tool and 78 (25.9%) completed both tools. When compared to the nurse-driven triage standard, both the AGST and VAS tools had poor levels of agreement for each of the four presenting complaints.ConclusionsSelf-triage through use of an AGST or VAS tool is inaccurate compared to the established standard of nurse-driven triage. Although existing literature exists which suggests that self-triage tools developed for specific subsets of complaints may be feasible, our results would suggest that adopting the self-triage approach on a broader scale for all-comers to the ED does not appear to be a viable option to enhance the current triage process. Further study is required to show if self-triage can be used in the ED to optimize the triage process.

Project description:ImportanceEmergency department (ED) triage substantially affects how long patients wait for care but triage scoring relies on few objective criteria. Prior studies suggest that Black and Hispanic patients receive unequal triage scores, paralleled by disparities in the depth of physician evaluations.ObjectivesTo examine whether racial disparities in triage scores and physician evaluations are present across a multicenter network of academic and community hospitals and evaluate whether patients who do not speak English face similar disparities.Design, setting, and participantsThis was a cross-sectional, multicenter study examining adults presenting between February 28, 2019, and January 1, 2023, across the Mass General Brigham Integrated Health Care System, encompassing 7 EDs: 2 urban academic hospitals and 5 community hospitals. Analysis included all patients presenting with 1 of 5 common chief symptoms.ExposuresEmergency department nurse-led triage and physician evaluation.Main outcomes and measuresAverage Triage Emergency Severity Index [ESI] score and average visit work relative value units [wRVUs] were compared across symptoms and between individual minority racial and ethnic groups and White patients.ResultsThere were 249 829 visits (149 861 female [60%], American Indian or Alaska Native 0.2%, Asian 3.3%, Black 11.8%, Hispanic 18.8%, Native Hawaiian or Other Pacific Islander <0.1%, White 60.8%, and patients identifying as Other race or ethnicity 5.1%). Median age was 48 (IQR, 29-66) years. White patients had more acute ESI scores than Hispanic or Other patients across all symptoms (eg, chest pain: Hispanic, 2.68 [95% CI, 2.67-2.69]; White, 2.55 [95% CI, 2.55-2.56]; Other, 2.66 [95% CI, 2.64-2.68]; P < .001) and Black patients across most symptoms (nausea/vomiting: Black, 2.97 [95% CI, 2.96-2.99]; White: 2.90 [95% CI, 2.89-2.91]; P < .001). These differences were reversed for wRVUs (chest pain: Black, 4.32 [95% CI, 4.25-4.39]; Hispanic, 4.13 [95% CI, 4.08-4.18]; White 3.55 [95% CI, 3.52-3.58]; Other 3.96 [95% CI, 3.84-4.08]; P < .001). Similar patterns were seen for patients whose primary language was not English.Conclusions and relevanceIn this cross-sectional study, patients who identified as Black, Hispanic, and Other race and ethnicity were assigned less acute ESI scores than their White peers despite having received more involved physician workups, suggesting some degree of mistriage. Clinical decision support systems might reduce these disparities but would require careful calibration to avoid replicating bias.

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.