Dataset Information

Comparison of emergency department trauma triage performance of clinicians and clinical prediction models: a cohort study in India.

ABSTRACT: OBJECTIVE:The aim of this study was to evaluate and compare the abilities of clinicians and clinical prediction models to accurately triage emergency department (ED) trauma patients. We compared the decisions made by clinicians with the Revised Trauma Score (RTS), the Glasgow Coma Scale, Age and Systolic Blood Pressure (GAP) score, the Kampala Trauma Score (KTS) and the Gerdin et al model. DESIGN:Prospective cohort study. SETTING:Three hospitals in urban India. PARTICIPANTS:In total, 7697 adult patients who presented to participating hospitals with a history of trauma were approached for enrolment. The final study sample included 5155 patients. The majority (4023, 78.0%) were male. MAIN OUTCOME MEASURE:The patient outcome was mortality within 30 days of arrival at the participating hospital. A grid search was used to identify model cut-off values. Clinicians and categorised models were evaluated and compared using the area under the receiver operating characteristics curve (AUROCC) and net reclassification improvement in non-survivors (NRI+) and survivors (NRI-) separately. RESULTS:The differences in AUROCC between each categorised model and the clinicians were 0.016 (95% CI -0.014 to 0.045) for RTS, 0.019 (95% CI -0.007 to 0.058) for GAP, 0.054 (95% CI 0.033 to 0.077) for KTS and -0.007 (95% CI -0.035 to 0.03) for Gerdin et al. The NRI+ for each model were -0.235 (-0.37 to -0.116), 0.17 (-0.042 to 0.405), 0.55 (0.47 to 0.65) and 0.22 (0.11 to 0.717), respectively. The NRI- were 0.385 (0.348 to 0.4), -0.059 (-0.476 to -0.005), -0.162 (-0.18 to -0.146) and 0.039 (-0.229 to 0.06), respectively. CONCLUSION:The findings of this study suggest that there are no substantial differences in discrimination and net reclassification improvement between clinicians and all four clinical prediction models when using 30-day mortality as the outcome of ED trauma triage in adult patients. TRIAL REGISTRATION NUMBER:ClinicalTrials.gov Registry (NCT02838459).

SUBMITTER: Warnberg Gerdin L

PROVIDER: S-EPMC7044989 | biostudies-literature | 2020 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparison of emergency department trauma triage performance of clinicians and clinical prediction models: a cohort study in India.

Wärnberg Gerdin Ludvig L Khajanchi Monty M Kumar Vineet V Roy Nobhojit N Saha Makhan Lal ML Soni Kapil Dev KD Mishra Anurag A Kamble Jyoti J Borle Nitin N Verma Chandrika Prasad CP Gerdin Wärnberg Martin M

BMJ open 20200218 2

<h4>Objective</h4>The aim of this study was to evaluate and compare the abilities of clinicians and clinical prediction models to accurately triage emergency department (ED) trauma patients. We compared the decisions made by clinicians with the Revised Trauma Score (RTS), the Glasgow Coma Scale, Age and Systolic Blood Pressure (GAP) score, the Kampala Trauma Score (KTS) and the Gerdin <i>et al</i> model.<h4>Design</h4>Prospective cohort study.<h4>Setting</h4>Three hospitals in urban India.<h4>Pa ...[more]

PMID: 32075827

Similar Datasets

Project description:BackgroundDevelopment of emergency department (ED) triage systems that accurately differentiate and prioritize critically ill from stable patients remains challenging. We used machine learning models to predict clinical outcomes, and then compared their performance with that of a conventional approach-the Emergency Severity Index (ESI).MethodsUsing National Hospital and Ambulatory Medical Care Survey (NHAMCS) ED data, from 2007 through 2015, we identified all adult patients (aged ??18?years). In the randomly sampled training set (70%), using routinely available triage data as predictors (e.g., demographics, triage vital signs, chief complaints, comorbidities), we developed four machine learning models: Lasso regression, random forest, gradient boosted decision tree, and deep neural network. As the reference model, we constructed a logistic regression model using the five-level ESI data. The clinical outcomes were critical care (admission to intensive care unit or in-hospital death) and hospitalization (direct hospital admission or transfer). In the test set (the remaining 30%), we measured the predictive performance, including area under the receiver-operating-characteristics curve (AUC) and net benefit (decision curves) for each model.ResultsOf 135,470 eligible ED visits, 2.1% had critical care outcome and 16.2% had hospitalization outcome. In the critical care outcome prediction, all four machine learning models outperformed the reference model (e.g., AUC, 0.86 [95%CI 0.85-0.87] in the deep neural network vs 0.74 [95%CI 0.72-0.75] in the reference model), with less under-triaged patients in ESI triage levels 3 to 5 (urgent to non-urgent). Likewise, in the hospitalization outcome prediction, all machine learning models outperformed the reference model (e.g., AUC, 0.82 [95%CI 0.82-0.83] in the deep neural network vs 0.69 [95%CI 0.68-0.69] in the reference model) with less over-triages in ESI triage levels 1 to 3 (immediate to urgent). In the decision curve analysis, all machine learning models consistently achieved a greater net benefit-a larger number of appropriate triages considering a trade-off with over-triages-across the range of clinical thresholds.ConclusionsCompared to the conventional approach, the machine learning models demonstrated a superior performance to predict critical care and hospitalization outcomes. The application of modern machine learning models may enhance clinicians' triage decision making, thereby achieving better clinical care and optimal resource utilization.

Project description:BACKGROUND:Improved trauma management can reduce the time between injury and medical interventions, thus decreasing morbidity and mortality. Triage at the emergency department is essential to ensure prioritization and timely assessment of injured patients. The aim of the present study was to investigate how a lack of formal triage system impacts timely intervention and mortality in a sub-Saharan referral hospital. Further, the study attempts to assess potential benefits of triage towards efficient management of trauma patients in one middle income country. METHODS:A prospective descriptive study was conducted. Adult trauma patients admitted to the emergency department during an 8-month period at Moi Teaching and Referral Hospital in Eldoret, Kenya, were included. Mode of arrival and vital parameters were registered. Variables included in the analysis were Injury Severity Score, time before physician's assessment, length of hospital stay, and mortality. The patients were retrospectively categorized according to the Rapid Emergency Triage and Treatment System (RETTS) from patient records. RESULTS:A total of 571 patients were analyzed, with a mean Injury Severity Score of 12.2 (SD 7.7) with a mean length of stay of 11.6 (SD 18.3) days. The mortality rate was 1.8%. The results obtained in this study illustrate that trauma patients admitted to the emergency department at Eldoret are not assessed in a timely fashion, and the time frame recommendations postulated by RETTS are not adhered to. Assessment of patients according to the triage algorithm used revealed a significantly higher average Injury Severity Score in the red category than in the other color categories. CONCLUSION:The results from this study clearly illustrate a lack of correct prioritization of patients in relation to the need for timely assessment. This is further demonstrated by the retrospective triage classification of patients, which identified patients with high ISS as in urgent need of care. Since no significant difference in to time to assessment regardless of injury severity was observed, the need for a well-functioning triage system is apparent.

Project description:ImportanceWhile machine learning approaches may enhance prediction ability, little is known about their utility in emergency department (ED) triage.ObjectivesTo examine the performance of machine learning approaches to predict clinical outcomes and disposition in children in the ED and to compare their performance with conventional triage approaches.Design, setting, and participantsPrognostic study of ED data from the National Hospital Ambulatory Medical Care Survey from January 1, 2007, through December 31, 2015. A nationally representative sample of 52 037 children aged 18 years or younger who presented to the ED were included. Data analysis was performed in August 2018.Main outcomes and measuresThe outcomes were critical care (admission to an intensive care unit and/or in-hospital death) and hospitalization (direct hospital admission or transfer). In the training set (70% random sample), using routinely available triage data as predictors (eg, demographic characteristics and vital signs), we derived 4 machine learning-based models: lasso regression, random forest, gradient-boosted decision tree, and deep neural network. In the test set (the remaining 30% of the sample), we measured the models' prediction performance by computing C statistics, prospective prediction results, and decision curves. These machine learning models were built for each outcome and compared with the reference model using the conventional triage classification information.ResultsOf 52 037 eligible ED visits by children (median [interquartile range] age, 6 [2-14] years; 24 929 [48.0%] female), 163 (0.3%) had the critical care outcome and 2352 (4.5%) had the hospitalization outcome. For the critical care prediction, all machine learning approaches had higher discriminative ability compared with the reference model, although the difference was not statistically significant (eg, C statistics of 0.85 [95% CI, 0.78-0.92] for the deep neural network vs 0.78 [95% CI, 0.71-0.85] for the reference; P = .16), and lower number of undertriaged critically ill children in the conventional triage levels 3 to 5 (urgent to nonurgent). For the hospitalization prediction, all machine learning approaches had significantly higher discrimination ability (eg, C statistic, 0.80 [95% CI, 0.78-0.81] for the deep neural network vs 0.73 [95% CI, 0.71-0.75] for the reference; P < .001) and fewer overtriaged children who did not require inpatient management in the conventional triage levels 1 to 3 (immediate to urgent). The decision curve analysis demonstrated a greater net benefit of machine learning models over ranges of clinical thresholds.Conclusions and relevanceMachine learning-based triage had better discrimination ability to predict clinical outcomes and disposition, with reduction in undertriaging critically ill children and overtriaging children who are less ill.

Project description:BackgroundDespite its continued use in many low-volume emergency departments (EDs), 3-level triage systems have not been extensively studied, especially on live triage cases. We have modified from the Australasian Triage Scale and developed a 3-level triage scale, and sought to evaluate its validity, reliability, and over- and under-triage rates in real patient encounters in our setting.MethodThis was a cross-sectional study in a single ED with 24,000 attendances per year. At triage, each patient was simultaneously assessed by a triage nurse, an adjudicator (the "criterion standard"), and a study nurse independently. Predictive validity was determined by comparing clinical outcomes, such as hospitalization, across triage levels. The discriminating performance of the triage tool in identifying patients requiring earlier medical attention was determined. Inter-observer reliability between the triage nurse and criterion standard, and across providers were determined using kappa statistics.ResultsIn total, 453 triage ratings of 151 triage cases, involving 17 ED triage nurses and 57 nurse pairs, were analysed. The proportion of hospital admission significantly increased with a higher triage rating. The performance of the scale in identifying patients requiring earlier medical attention was as follows: sensitivity, 68.2% (95% CI 45.1-86.1%); specificity, 99.2% (95% CI 95.8-100%); positive predictive value, 93.8% (95% CI 67.6-99.1%); and negative predictive value, 94.8% (95% CI 90.8-97.1%). The over-triage and under-triage rates were 0.7% and 4.6%, respectively. Agreement between the triage nurse and criterion standard was substantial (quadratic-weighted kappa = 0.76, 95% CI, 0.60-0.92, p < 0.001), so was the agreement across nurses (quadratic-weighted kappa = 0.81, 95% CI 0.65-0.97, p < 0.001).ConclusionsThe 3-level triage system appears to have good validity and reasonable reliability in a low-volume ED setting. Further studies comparing 3-level and prevailing 5-level triage scales in live triage encounters and different ED settings are warranted.

Project description:BackgroundStudies on the reliability of the MTS and its predictive power for hospitalisation and mortality in the older population have demonstrated mixed results. The objective is to evaluate the performance of the Manchester Triage System (MTS) in older patients (?65?years) by assessing the predictive ability of the MTS for emergency department resource utilisation, emergency department length of stay (ED-LOS), hospitalisation, and in-hospital mortality rate. The secondary goal was to evaluate the performance of the MTS in older surgical versus medical patients.MethodsA retrospective cohort study was conducted of all emergency department visits by patients ?65?years between 01 and 09-2011 and 31-08-2012. Performance of the MTS was assessed by comparing the association of the MTS with emergency department resource utilisation, ED-LOS, hospital admission, and in-hospital mortality in older patients and the reference group (18-64?years), and by estimating the area under the receiver operating characteristics curves.ResultsData on 7108 emergency department visits by older patients and 13,767 emergency department visits by patients aged 18-64?years were included. In both patient groups, a higher emergency department resource utilisation was associated with a higher MTS urgency. The AUC for the MTS and hospitalisation was 0.74 (95%CI 0.73-0.75) in older patients and 0.76 (95%CI 0.76-0.77) in patients aged 18-64?years. Comparison of the predictive ability of the MTS for in-hospital mortality in older patients with patients aged 18-64?years revealed an AUC of 0.71 (95%CI 0.68-0.74) versus 0.79 (95%CI 0.72-0.85). The majority of older patients (54.8%) were evaluated by a medical specialty and 45.2% by a surgical specialty. The predictive ability of the MTS for hospitalisation and in-hospital mortality was higher in older surgical patients than in medical patients (AUC 0.74, 95%CI 0.72-0.76 and 0.74, 95%CI 0.68-0.81 versus 0.69, 95%CI 0.67-0.71 and 0.66, 95%CI 0.62-0.69).ConclusionThe performance of the MTS appeared inferior in older patients than younger patients, illustrated by a worse predictive ability of the MTS for in-hospital mortality in older patients. The MTS demonstrated a better performance in older surgical patients than older medical patients regarding hospitalisation and in-hospital mortality.

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.

Project description:BackgroundEmergency department (ED) crowding is associated with negative health outcomes, patient dissatisfaction, and longer length of stay (LOS). The addition of advanced diagnostic imaging (ADI), namely CT, ultrasound (U/S), and MRI to ED encounter work up is a predictor of longer length of stay. Earlier and improved prediction of patients' need for advanced imaging may improve overall ED efficiency. The aim of the study was to detect the association between ADI utilization and the structured and unstructured information immediately available during ED triage, and to develop and validate models to predict utilization of ADI during an ED encounter.MethodsWe used the United States National Hospital Ambulatory Medical Care Survey data from 2009 to 2014 to examine which sociodemographic and clinical factors immediately available at ED triage were associated with the utilization of CT, U/S, MRI, and multiple ADI during a patient's ED stay. We used natural language processing (NLP) topic modeling to incorporate free-text reason for visit data available at time of ED triage in addition to other structured patient data to predict the use of ADI using multivariable logistic regression models.ResultsAmong the 139,150 adult ED visits from a national probability sample of hospitals across the U.S, 21.9% resulted in ADI use, including 16.8% who had a CT, 3.6% who had an ultrasound, 0.4% who had an MRI, and 1.2% of the population who had multiple types of ADI. The c-statistic of the predictive models was greater than or equal to 0.78 for all imaging outcomes, and the addition of text-based reason for visit information improved the accuracy of all predictive models.ConclusionsPatient information immediately available during ED triage can accurately predict the eventual use of advanced diagnostic imaging during an ED visit. Such models have the potential to be incorporated into the ED triage workflow in order to more rapidly identify patients who may require advanced imaging during their ED stay and assist with medical decision-making.

Project description:BackgroundThe emergency department (ED) triage system to classify and prioritize patients from high risk to less urgent continues to be a challenge.ObjectiveThis study, comprising 80,433 patients, aims to develop a machine learning algorithm prediction model of critical care outcomes for adult patients using information collected during ED triage and compare the performance with that of the baseline model using the Korean Triage and Acuity Scale (KTAS).MethodsTo predict the need for critical care, we used 13 predictors from triage information: age, gender, mode of ED arrival, the time interval between onset and ED arrival, reason of ED visit, chief complaints, systolic blood pressure, diastolic blood pressure, pulse rate, respiratory rate, body temperature, oxygen saturation, and level of consciousness. The baseline model with KTAS was developed using logistic regression, and the machine learning model with 13 variables was generated using extreme gradient boosting (XGB) and deep neural network (DNN) algorithms. The discrimination was measured by the area under the receiver operating characteristic (AUROC) curve. The ability of calibration with Hosmer-Lemeshow test and reclassification with net reclassification index were evaluated. The calibration plot and partial dependence plot were used in the analysis.ResultsThe AUROC of the model with the full set of variables (0.833-0.861) was better than that of the baseline model (0.796). The XGB model of AUROC 0.861 (95% CI 0.848-0.874) showed a higher discriminative performance than the DNN model of 0.833 (95% CI 0.819-0.848). The XGB and DNN models proved better reclassification than the baseline model with a positive net reclassification index. The XGB models were well-calibrated (Hosmer-Lemeshow test; P>.05); however, the DNN showed poor calibration power (Hosmer-Lemeshow test; P<.001). We further interpreted the nonlinear association between variables and critical care prediction.ConclusionsOur study demonstrated that the performance of the XGB model using initial information at ED triage for predicting patients in need of critical care outperformed the conventional model with KTAS.

Dataset Information

Comparison of emergency department trauma triage performance of clinicians and clinical prediction models: a cohort study in India.

Publications

Comparison of emergency department trauma triage performance of clinicians and clinical prediction models: a cohort study in India.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets