Dataset Information

Comparison of emergency department trauma triage performance of clinicians and clinical prediction models: a cohort study in India.

ABSTRACT: OBJECTIVE:The aim of this study was to evaluate and compare the abilities of clinicians and clinical prediction models to accurately triage emergency department (ED) trauma patients. We compared the decisions made by clinicians with the Revised Trauma Score (RTS), the Glasgow Coma Scale, Age and Systolic Blood Pressure (GAP) score, the Kampala Trauma Score (KTS) and the Gerdin et al model. DESIGN:Prospective cohort study. SETTING:Three hospitals in urban India. PARTICIPANTS:In total, 7697 adult patients who presented to participating hospitals with a history of trauma were approached for enrolment. The final study sample included 5155 patients. The majority (4023, 78.0%) were male. MAIN OUTCOME MEASURE:The patient outcome was mortality within 30 days of arrival at the participating hospital. A grid search was used to identify model cut-off values. Clinicians and categorised models were evaluated and compared using the area under the receiver operating characteristics curve (AUROCC) and net reclassification improvement in non-survivors (NRI+) and survivors (NRI-) separately. RESULTS:The differences in AUROCC between each categorised model and the clinicians were 0.016 (95% CI -0.014 to 0.045) for RTS, 0.019 (95% CI -0.007 to 0.058) for GAP, 0.054 (95% CI 0.033 to 0.077) for KTS and -0.007 (95% CI -0.035 to 0.03) for Gerdin et al. The NRI+ for each model were -0.235 (-0.37 to -0.116), 0.17 (-0.042 to 0.405), 0.55 (0.47 to 0.65) and 0.22 (0.11 to 0.717), respectively. The NRI- were 0.385 (0.348 to 0.4), -0.059 (-0.476 to -0.005), -0.162 (-0.18 to -0.146) and 0.039 (-0.229 to 0.06), respectively. CONCLUSION:The findings of this study suggest that there are no substantial differences in discrimination and net reclassification improvement between clinicians and all four clinical prediction models when using 30-day mortality as the outcome of ED trauma triage in adult patients. TRIAL REGISTRATION NUMBER:ClinicalTrials.gov Registry (NCT02838459).

SUBMITTER: Warnberg Gerdin L

PROVIDER: S-EPMC7044989 | biostudies-literature | 2020 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparison of emergency department trauma triage performance of clinicians and clinical prediction models: a cohort study in India.

Wärnberg Gerdin Ludvig L Khajanchi Monty M Kumar Vineet V Roy Nobhojit N Saha Makhan Lal ML Soni Kapil Dev KD Mishra Anurag A Kamble Jyoti J Borle Nitin N Verma Chandrika Prasad CP Gerdin Wärnberg Martin M

BMJ open 20200218 2

<h4>Objective</h4>The aim of this study was to evaluate and compare the abilities of clinicians and clinical prediction models to accurately triage emergency department (ED) trauma patients. We compared the decisions made by clinicians with the Revised Trauma Score (RTS), the Glasgow Coma Scale, Age and Systolic Blood Pressure (GAP) score, the Kampala Trauma Score (KTS) and the Gerdin <i>et al</i> model.<h4>Design</h4>Prospective cohort study.<h4>Setting</h4>Three hospitals in urban India.<h4>Pa ...[more]

PMID: 32075827

Similar Datasets

Project description:BackgroundDevelopment of emergency department (ED) triage systems that accurately differentiate and prioritize critically ill from stable patients remains challenging. We used machine learning models to predict clinical outcomes, and then compared their performance with that of a conventional approach-the Emergency Severity Index (ESI).MethodsUsing National Hospital and Ambulatory Medical Care Survey (NHAMCS) ED data, from 2007 through 2015, we identified all adult patients (aged ≥ 18 years). In the randomly sampled training set (70%), using routinely available triage data as predictors (e.g., demographics, triage vital signs, chief complaints, comorbidities), we developed four machine learning models: Lasso regression, random forest, gradient boosted decision tree, and deep neural network. As the reference model, we constructed a logistic regression model using the five-level ESI data. The clinical outcomes were critical care (admission to intensive care unit or in-hospital death) and hospitalization (direct hospital admission or transfer). In the test set (the remaining 30%), we measured the predictive performance, including area under the receiver-operating-characteristics curve (AUC) and net benefit (decision curves) for each model.ResultsOf 135,470 eligible ED visits, 2.1% had critical care outcome and 16.2% had hospitalization outcome. In the critical care outcome prediction, all four machine learning models outperformed the reference model (e.g., AUC, 0.86 [95%CI 0.85-0.87] in the deep neural network vs 0.74 [95%CI 0.72-0.75] in the reference model), with less under-triaged patients in ESI triage levels 3 to 5 (urgent to non-urgent). Likewise, in the hospitalization outcome prediction, all machine learning models outperformed the reference model (e.g., AUC, 0.82 [95%CI 0.82-0.83] in the deep neural network vs 0.69 [95%CI 0.68-0.69] in the reference model) with less over-triages in ESI triage levels 1 to 3 (immediate to urgent). In the decision curve analysis, all machine learning models consistently achieved a greater net benefit-a larger number of appropriate triages considering a trade-off with over-triages-across the range of clinical thresholds.ConclusionsCompared to the conventional approach, the machine learning models demonstrated a superior performance to predict critical care and hospitalization outcomes. The application of modern machine learning models may enhance clinicians' triage decision making, thereby achieving better clinical care and optimal resource utilization.

Project description:IntroductionCanadian patients presenting to the emergency department (ED) typically undergo a triage process where they are assessed by a specially trained nurse and assigned a Canadian Triage and Acuity Scale (CTAS) score, indicating their level of acuity and urgency of assessment. We sought to assess the ability of patients to self-triage themselves through use of one of two of our proprietary self-triage tools, and how this would compare with the standard nurse-driven triage process.MethodsWe enrolled a convenience sample of ambulatory ED patients aged 17 years or older who presented with chief complaints of chest pain, abdominal pain, breathing problems, or musculoskeletal pain. Participants completed one, or both, of an algorithm generated self-triage (AGST) survey, or visual acuity scale (VAS) based self-triage tool which subsequently generated a CTAS score. Our primary outcome was to assess the accuracy of these tools to the CTAS score generated through the nurse-driven triage process.ResultsA total of 223 patients were included in our analysis. Of these, 32 (14.3%) presented with chest pain, 25 (11.2%) with shortness of breath, 75 (33.6%) with abdominal pain, and 91 (40.8%) with musculoskeletal pain. Of the total number of patients, 142 (47.2%) completed the AGST tool, 159 (52.8%) completed the VAS tool and 78 (25.9%) completed both tools. When compared to the nurse-driven triage standard, both the AGST and VAS tools had poor levels of agreement for each of the four presenting complaints.ConclusionsSelf-triage through use of an AGST or VAS tool is inaccurate compared to the established standard of nurse-driven triage. Although existing literature exists which suggests that self-triage tools developed for specific subsets of complaints may be feasible, our results would suggest that adopting the self-triage approach on a broader scale for all-comers to the ED does not appear to be a viable option to enhance the current triage process. Further study is required to show if self-triage can be used in the ED to optimize the triage process.

Project description:BACKGROUND:Improved trauma management can reduce the time between injury and medical interventions, thus decreasing morbidity and mortality. Triage at the emergency department is essential to ensure prioritization and timely assessment of injured patients. The aim of the present study was to investigate how a lack of formal triage system impacts timely intervention and mortality in a sub-Saharan referral hospital. Further, the study attempts to assess potential benefits of triage towards efficient management of trauma patients in one middle income country. METHODS:A prospective descriptive study was conducted. Adult trauma patients admitted to the emergency department during an 8-month period at Moi Teaching and Referral Hospital in Eldoret, Kenya, were included. Mode of arrival and vital parameters were registered. Variables included in the analysis were Injury Severity Score, time before physician's assessment, length of hospital stay, and mortality. The patients were retrospectively categorized according to the Rapid Emergency Triage and Treatment System (RETTS) from patient records. RESULTS:A total of 571 patients were analyzed, with a mean Injury Severity Score of 12.2 (SD 7.7) with a mean length of stay of 11.6 (SD 18.3) days. The mortality rate was 1.8%. The results obtained in this study illustrate that trauma patients admitted to the emergency department at Eldoret are not assessed in a timely fashion, and the time frame recommendations postulated by RETTS are not adhered to. Assessment of patients according to the triage algorithm used revealed a significantly higher average Injury Severity Score in the red category than in the other color categories. CONCLUSION:The results from this study clearly illustrate a lack of correct prioritization of patients in relation to the need for timely assessment. This is further demonstrated by the retrospective triage classification of patients, which identified patients with high ISS as in urgent need of care. Since no significant difference in to time to assessment regardless of injury severity was observed, the need for a well-functioning triage system is apparent.

Project description:ImportanceWhile machine learning approaches may enhance prediction ability, little is known about their utility in emergency department (ED) triage.ObjectivesTo examine the performance of machine learning approaches to predict clinical outcomes and disposition in children in the ED and to compare their performance with conventional triage approaches.Design, setting, and participantsPrognostic study of ED data from the National Hospital Ambulatory Medical Care Survey from January 1, 2007, through December 31, 2015. A nationally representative sample of 52 037 children aged 18 years or younger who presented to the ED were included. Data analysis was performed in August 2018.Main outcomes and measuresThe outcomes were critical care (admission to an intensive care unit and/or in-hospital death) and hospitalization (direct hospital admission or transfer). In the training set (70% random sample), using routinely available triage data as predictors (eg, demographic characteristics and vital signs), we derived 4 machine learning-based models: lasso regression, random forest, gradient-boosted decision tree, and deep neural network. In the test set (the remaining 30% of the sample), we measured the models' prediction performance by computing C statistics, prospective prediction results, and decision curves. These machine learning models were built for each outcome and compared with the reference model using the conventional triage classification information.ResultsOf 52 037 eligible ED visits by children (median [interquartile range] age, 6 [2-14] years; 24 929 [48.0%] female), 163 (0.3%) had the critical care outcome and 2352 (4.5%) had the hospitalization outcome. For the critical care prediction, all machine learning approaches had higher discriminative ability compared with the reference model, although the difference was not statistically significant (eg, C statistics of 0.85 [95% CI, 0.78-0.92] for the deep neural network vs 0.78 [95% CI, 0.71-0.85] for the reference; P = .16), and lower number of undertriaged critically ill children in the conventional triage levels 3 to 5 (urgent to nonurgent). For the hospitalization prediction, all machine learning approaches had significantly higher discrimination ability (eg, C statistic, 0.80 [95% CI, 0.78-0.81] for the deep neural network vs 0.73 [95% CI, 0.71-0.75] for the reference; P < .001) and fewer overtriaged children who did not require inpatient management in the conventional triage levels 1 to 3 (immediate to urgent). The decision curve analysis demonstrated a greater net benefit of machine learning models over ranges of clinical thresholds.Conclusions and relevanceMachine learning-based triage had better discrimination ability to predict clinical outcomes and disposition, with reduction in undertriaging critically ill children and overtriaging children who are less ill.

Project description:BackgroundDespite its continued use in many low-volume emergency departments (EDs), 3-level triage systems have not been extensively studied, especially on live triage cases. We have modified from the Australasian Triage Scale and developed a 3-level triage scale, and sought to evaluate its validity, reliability, and over- and under-triage rates in real patient encounters in our setting.MethodThis was a cross-sectional study in a single ED with 24,000 attendances per year. At triage, each patient was simultaneously assessed by a triage nurse, an adjudicator (the "criterion standard"), and a study nurse independently. Predictive validity was determined by comparing clinical outcomes, such as hospitalization, across triage levels. The discriminating performance of the triage tool in identifying patients requiring earlier medical attention was determined. Inter-observer reliability between the triage nurse and criterion standard, and across providers were determined using kappa statistics.ResultsIn total, 453 triage ratings of 151 triage cases, involving 17 ED triage nurses and 57 nurse pairs, were analysed. The proportion of hospital admission significantly increased with a higher triage rating. The performance of the scale in identifying patients requiring earlier medical attention was as follows: sensitivity, 68.2% (95% CI 45.1-86.1%); specificity, 99.2% (95% CI 95.8-100%); positive predictive value, 93.8% (95% CI 67.6-99.1%); and negative predictive value, 94.8% (95% CI 90.8-97.1%). The over-triage and under-triage rates were 0.7% and 4.6%, respectively. Agreement between the triage nurse and criterion standard was substantial (quadratic-weighted kappa = 0.76, 95% CI, 0.60-0.92, p < 0.001), so was the agreement across nurses (quadratic-weighted kappa = 0.81, 95% CI 0.65-0.97, p < 0.001).ConclusionsThe 3-level triage system appears to have good validity and reasonable reliability in a low-volume ED setting. Further studies comparing 3-level and prevailing 5-level triage scales in live triage encounters and different ED settings are warranted.

Project description:BackgroundStudies on the reliability of the MTS and its predictive power for hospitalisation and mortality in the older population have demonstrated mixed results. The objective is to evaluate the performance of the Manchester Triage System (MTS) in older patients (?65?years) by assessing the predictive ability of the MTS for emergency department resource utilisation, emergency department length of stay (ED-LOS), hospitalisation, and in-hospital mortality rate. The secondary goal was to evaluate the performance of the MTS in older surgical versus medical patients.MethodsA retrospective cohort study was conducted of all emergency department visits by patients ?65?years between 01 and 09-2011 and 31-08-2012. Performance of the MTS was assessed by comparing the association of the MTS with emergency department resource utilisation, ED-LOS, hospital admission, and in-hospital mortality in older patients and the reference group (18-64?years), and by estimating the area under the receiver operating characteristics curves.ResultsData on 7108 emergency department visits by older patients and 13,767 emergency department visits by patients aged 18-64?years were included. In both patient groups, a higher emergency department resource utilisation was associated with a higher MTS urgency. The AUC for the MTS and hospitalisation was 0.74 (95%CI 0.73-0.75) in older patients and 0.76 (95%CI 0.76-0.77) in patients aged 18-64?years. Comparison of the predictive ability of the MTS for in-hospital mortality in older patients with patients aged 18-64?years revealed an AUC of 0.71 (95%CI 0.68-0.74) versus 0.79 (95%CI 0.72-0.85). The majority of older patients (54.8%) were evaluated by a medical specialty and 45.2% by a surgical specialty. The predictive ability of the MTS for hospitalisation and in-hospital mortality was higher in older surgical patients than in medical patients (AUC 0.74, 95%CI 0.72-0.76 and 0.74, 95%CI 0.68-0.81 versus 0.69, 95%CI 0.67-0.71 and 0.66, 95%CI 0.62-0.69).ConclusionThe performance of the MTS appeared inferior in older patients than younger patients, illustrated by a worse predictive ability of the MTS for in-hospital mortality in older patients. The MTS demonstrated a better performance in older surgical patients than older medical patients regarding hospitalisation and in-hospital mortality.

Project description:ImportanceEmergency department (ED) triage substantially affects how long patients wait for care but triage scoring relies on few objective criteria. Prior studies suggest that Black and Hispanic patients receive unequal triage scores, paralleled by disparities in the depth of physician evaluations.ObjectivesTo examine whether racial disparities in triage scores and physician evaluations are present across a multicenter network of academic and community hospitals and evaluate whether patients who do not speak English face similar disparities.Design, setting, and participantsThis was a cross-sectional, multicenter study examining adults presenting between February 28, 2019, and January 1, 2023, across the Mass General Brigham Integrated Health Care System, encompassing 7 EDs: 2 urban academic hospitals and 5 community hospitals. Analysis included all patients presenting with 1 of 5 common chief symptoms.ExposuresEmergency department nurse-led triage and physician evaluation.Main outcomes and measuresAverage Triage Emergency Severity Index [ESI] score and average visit work relative value units [wRVUs] were compared across symptoms and between individual minority racial and ethnic groups and White patients.ResultsThere were 249 829 visits (149 861 female [60%], American Indian or Alaska Native 0.2%, Asian 3.3%, Black 11.8%, Hispanic 18.8%, Native Hawaiian or Other Pacific Islander <0.1%, White 60.8%, and patients identifying as Other race or ethnicity 5.1%). Median age was 48 (IQR, 29-66) years. White patients had more acute ESI scores than Hispanic or Other patients across all symptoms (eg, chest pain: Hispanic, 2.68 [95% CI, 2.67-2.69]; White, 2.55 [95% CI, 2.55-2.56]; Other, 2.66 [95% CI, 2.64-2.68]; P < .001) and Black patients across most symptoms (nausea/vomiting: Black, 2.97 [95% CI, 2.96-2.99]; White: 2.90 [95% CI, 2.89-2.91]; P < .001). These differences were reversed for wRVUs (chest pain: Black, 4.32 [95% CI, 4.25-4.39]; Hispanic, 4.13 [95% CI, 4.08-4.18]; White 3.55 [95% CI, 3.52-3.58]; Other 3.96 [95% CI, 3.84-4.08]; P < .001). Similar patterns were seen for patients whose primary language was not English.Conclusions and relevanceIn this cross-sectional study, patients who identified as Black, Hispanic, and Other race and ethnicity were assigned less acute ESI scores than their White peers despite having received more involved physician workups, suggesting some degree of mistriage. Clinical decision support systems might reduce these disparities but would require careful calibration to avoid replicating bias.

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.

Dataset Information

Comparison of emergency department trauma triage performance of clinicians and clinical prediction models: a cohort study in India.

Publications

Comparison of emergency department trauma triage performance of clinicians and clinical prediction models: a cohort study in India.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets