Dataset Information

Multimodal learning for emergency department triage implementation: experiences from Papua New Guinea during the COVID-19 pandemic.

ABSTRACT:

Background

Triage implementation in resource-limited emergency departments (EDs) has traditionally relied on intensive in-person training. This study sought to evaluate the impact of a novel digital-based learning strategy focused on the Interagency Integrated Triage Tool, a three-tier triage instrument recommended by the World Health Organization.

Methods

A mixed methods study utilising pre-post intervention methods was conducted in two EDs in Papua New Guinea. The primary outcome was the mean change in knowledge before and after completion of a voluntary, multimodal training program, primarily delivered through a digital learning platform accessible via smartphone. Secondary outcomes included the change in confidence to perform selected clinical tasks, and acceptability of the learning methods.

Findings

Among 136 eligible ED staff, 91 (66.9%) completed the digital learning program. The mean knowledge score on the post-training exam was 87.5% (SD 10.4), a mean increase of 12.9% (95% CI 10.7-15.1%, p < 0.0001) from the pre-training exam. There were statistically significant improvements in confidence for 13 of 15 clinical tasks, including undertaking a triage assessment and identifying an unwell patient.In an evaluation survey, 100% of 30 respondents agreed or strongly agreed the online learning platform was easy to access, use and navigate, and that the digital teaching methods were appropriate for their learning needs. In qualitative feedback, respondents reported that limited internet access and a lack of dedicated training time were barriers to participation.

Interpretation

The use of digital learning to support triage implementation in resource-limited EDs is feasible and effective when accompanied by in-person mentoring. Adequate internet access is an essential pre-requisite.

Funding

Development of the Kumul Helt Skul learning platform was undertaken as part of the Clinical Support Program (Phase II), facilitated by Johnstaff International Development on behalf of the Australian Government Department of Foreign Affairs and Trade through the PNG-Australia Partnership. RM is supported by a National Health and Medical Research Council Postgraduate Scholarship and a Monash Graduate Excellence Scholarship, while PC is supported by a Medical Research Future Fund Practitioner Fellowship. Funders had no role in study design, results analysis or manuscript preparation.

SUBMITTER: Mitchell R

PROVIDER: S-EPMC9901330 | biostudies-literature | 2023 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Multimodal learning for emergency department triage implementation: experiences from Papua New Guinea during the COVID-19 pandemic.

Mitchell Rob R Bornstein Sarah S Piamnok Donna D Sebby Wilma W Kingston Carl C Tefatu Rayleen R Kendino Mangu M Josaiah Betty B Pole Jasper J Kuk Sylvia S Körver Sarah S Miller Jean-Philippe JP Cole Travis T Erbs Andrew A O'Reilly Gerard G Cameron Peter P Sengiromo Duncan D Banks Colin C

The Lancet regional health. Western Pacific 20230206

<h4>Background</h4>Triage implementation in resource-limited emergency departments (EDs) has traditionally relied on intensive in-person training. This study sought to evaluate the impact of a novel digital-based learning strategy focused on the Interagency Integrated Triage Tool, a three-tier triage instrument recommended by the World Health Organization.<h4>Methods</h4>A mixed methods study utilising pre-post intervention methods was conducted in two EDs in Papua New Guinea. The primary outcom ...[more]

PMID: 36776620

Similar Datasets

Project description:BackgroundThe Interagency Integrated Triage Tool (IITT) is a three-tier triage system designed for resource-limited emergency care (EC) settings. This study sought to assess the validity and reliability of a pilot version of the tool in an urban emergency department (ED) in Papua New Guinea.MethodsA pragmatic observational study was conducted at Gerehu General Hospital in Port Moresby, commencing eight weeks after IITT implementation. All ED patients presenting within the subsequent two-month period were included. Triage assessments were performed by a variety of ED clinicians, including community health workers, nurses and doctors. The primary outcome was sensitivity for the detection of time-critical illness, defined by ten pre-specified diagnoses. The association between triage category and ED outcomes was examined using Cramer's V correlation coefficient. Reliability was assessed by inter-rater agreement between a local and an experienced, external triage officer.FindingsAmong 4512 presentations during the study period, 58 (1.3%) were classified as category one (emergency), 967 (21.6%) as category two (priority) and 3478 (77.1%) as category three (non-urgent). The tool's sensitivity for detecting the pre-specified set of time-sensitive conditions was 70.8% (95%CI 58.2-81.4%), with negative predictive values of 97.3% (95%CI 96.7 - 97.8%) for admission/transfer and 99.9% (95%CI 99.7 - 100.0%) for death. The admission/transfer rate was 44.8% (26/58) among emergency patients, 22.9% (223/976) among priority patients and 2.7% (94/3478) among non-urgent patients (Cramer's V=0.351, p=0.00). Four of 58 (6.9%) emergency patients, 19/976 (2.0%) priority patients and 3/3478 (0.1%) non-urgent patients died in the ED (Cramer's V=0.14, p=0.00). The under-triage rate was 2.7% (94/3477) and the over-triage rate 48.2% (28/58), both within pre-specified limits of acceptability. On average, it took staff 3 minutes 34 seconds (SD 1:06) to determine and document a triage category. Among 70 observed assessments, weighted κ was 0.84 (excellent agreement).InterpretationThe pilot version of the IITT demonstrated acceptable performance characteristics, and validation in other EC settings is warranted.FundingThis project was funded through a Friendship Grant from the Australian Government Department of Foreign Affairs and Trade and an International Development Fund Grant from the Australasian College for Emergency Medicine Foundation.

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.

Project description:BackgroundDevelopment of emergency department (ED) triage systems that accurately differentiate and prioritize critically ill from stable patients remains challenging. We used machine learning models to predict clinical outcomes, and then compared their performance with that of a conventional approach-the Emergency Severity Index (ESI).MethodsUsing National Hospital and Ambulatory Medical Care Survey (NHAMCS) ED data, from 2007 through 2015, we identified all adult patients (aged ≥ 18 years). In the randomly sampled training set (70%), using routinely available triage data as predictors (e.g., demographics, triage vital signs, chief complaints, comorbidities), we developed four machine learning models: Lasso regression, random forest, gradient boosted decision tree, and deep neural network. As the reference model, we constructed a logistic regression model using the five-level ESI data. The clinical outcomes were critical care (admission to intensive care unit or in-hospital death) and hospitalization (direct hospital admission or transfer). In the test set (the remaining 30%), we measured the predictive performance, including area under the receiver-operating-characteristics curve (AUC) and net benefit (decision curves) for each model.ResultsOf 135,470 eligible ED visits, 2.1% had critical care outcome and 16.2% had hospitalization outcome. In the critical care outcome prediction, all four machine learning models outperformed the reference model (e.g., AUC, 0.86 [95%CI 0.85-0.87] in the deep neural network vs 0.74 [95%CI 0.72-0.75] in the reference model), with less under-triaged patients in ESI triage levels 3 to 5 (urgent to non-urgent). Likewise, in the hospitalization outcome prediction, all machine learning models outperformed the reference model (e.g., AUC, 0.82 [95%CI 0.82-0.83] in the deep neural network vs 0.69 [95%CI 0.68-0.69] in the reference model) with less over-triages in ESI triage levels 1 to 3 (immediate to urgent). In the decision curve analysis, all machine learning models consistently achieved a greater net benefit-a larger number of appropriate triages considering a trade-off with over-triages-across the range of clinical thresholds.ConclusionsCompared to the conventional approach, the machine learning models demonstrated a superior performance to predict critical care and hospitalization outcomes. The application of modern machine learning models may enhance clinicians' triage decision making, thereby achieving better clinical care and optimal resource utilization.

Project description:ImportanceWhile machine learning approaches may enhance prediction ability, little is known about their utility in emergency department (ED) triage.ObjectivesTo examine the performance of machine learning approaches to predict clinical outcomes and disposition in children in the ED and to compare their performance with conventional triage approaches.Design, setting, and participantsPrognostic study of ED data from the National Hospital Ambulatory Medical Care Survey from January 1, 2007, through December 31, 2015. A nationally representative sample of 52 037 children aged 18 years or younger who presented to the ED were included. Data analysis was performed in August 2018.Main outcomes and measuresThe outcomes were critical care (admission to an intensive care unit and/or in-hospital death) and hospitalization (direct hospital admission or transfer). In the training set (70% random sample), using routinely available triage data as predictors (eg, demographic characteristics and vital signs), we derived 4 machine learning-based models: lasso regression, random forest, gradient-boosted decision tree, and deep neural network. In the test set (the remaining 30% of the sample), we measured the models' prediction performance by computing C statistics, prospective prediction results, and decision curves. These machine learning models were built for each outcome and compared with the reference model using the conventional triage classification information.ResultsOf 52 037 eligible ED visits by children (median [interquartile range] age, 6 [2-14] years; 24 929 [48.0%] female), 163 (0.3%) had the critical care outcome and 2352 (4.5%) had the hospitalization outcome. For the critical care prediction, all machine learning approaches had higher discriminative ability compared with the reference model, although the difference was not statistically significant (eg, C statistics of 0.85 [95% CI, 0.78-0.92] for the deep neural network vs 0.78 [95% CI, 0.71-0.85] for the reference; P = .16), and lower number of undertriaged critically ill children in the conventional triage levels 3 to 5 (urgent to nonurgent). For the hospitalization prediction, all machine learning approaches had significantly higher discrimination ability (eg, C statistic, 0.80 [95% CI, 0.78-0.81] for the deep neural network vs 0.73 [95% CI, 0.71-0.75] for the reference; P < .001) and fewer overtriaged children who did not require inpatient management in the conventional triage levels 1 to 3 (immediate to urgent). The decision curve analysis demonstrated a greater net benefit of machine learning models over ranges of clinical thresholds.Conclusions and relevanceMachine learning-based triage had better discrimination ability to predict clinical outcomes and disposition, with reduction in undertriaging critically ill children and overtriaging children who are less ill.

Project description:Study objectiveTriage quickly identifies critically ill patients, facilitating timely interventions. Many emergency departments (EDs) use emergency severity index (ESI) or abnormal vital sign triggers to guide triage. However, both use fixed thresholds, and false activations are costly. Prior approaches using machinelearning have relied on information that is often unavailable during the triage process. We examined whether deep-learning approaches could identify critically ill patients only using data immediately available at triage.MethodsWe conducted a retrospective, cross-sectional study at an urban tertiary care center, from January 1, 2012-January 1, 2020. De-identified triage information included structured (age, sex, initial vital signs) and textual (chief complaint) data, with critical illness (mortality or ICU admission within 24 hours) as the outcome. Four progressively complex deep-learning models were trained and applied to triage information from all patients. We compared the accuracy of the models against ESI as the standard diagnostic test, using area under the receiver-operator curve (AUC).ResultsA total of 445,925 patients were included, with 60,901 (13.7%) critically ill. Vital sign thresholds identified critically ill patients with AUC 0.521 (95% confidence interval [CI] = 0.519-0.522), and ESI <3 demonstrated AUC 0.672 (95% CI = 0.671-0.674), logistic regression classified patients with AUC 0.803 (95% CI = 0.802-0.804), 2-layer neural network with structured data with AUC 0.811 (95% CI = 0.807-0.815), gradient tree boosting with AUC 0.820 (95% CI = 0.818-0.821), and the neural network model with textual data with AUC 0.851 (95% CI = 0.849-0.852). All successive increases in AUC were statistically significant.ConclusionDeep-learning techniques represent a promising method of augmenting triage, even with limited information. Further research is needed to determine if improved predictions yield clinical and operational benefits.

Dataset Information

Multimodal learning for emergency department triage implementation: experiences from Papua New Guinea during the COVID-19 pandemic.

Background

Methods

Findings

Interpretation

Funding

Publications

Multimodal learning for emergency department triage implementation: experiences from Papua New Guinea during the COVID-19 pandemic.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets