Project description:BackgroundSymptom checkers are clinical decision support apps for patients, used by tens of millions of people annually. They are designed to provide diagnostic and triage advice and assist users in seeking the appropriate level of care. Little evidence is available regarding their diagnostic and triage accuracy with direct use by patients for urgent conditions.ObjectiveThe aim of this study is to determine the diagnostic and triage accuracy and usability of a symptom checker in use by patients presenting to an emergency department (ED).MethodsWe recruited a convenience sample of English-speaking patients presenting for care in an urban ED. Each consenting patient used a leading symptom checker from Ada Health before the ED evaluation. Diagnostic accuracy was evaluated by comparing the symptom checker's diagnoses and those of 3 independent emergency physicians viewing the patient-entered symptom data, with the final diagnoses from the ED evaluation. The Ada diagnoses and triage were also critiqued by the independent physicians. The patients completed a usability survey based on the Technology Acceptance Model.ResultsA total of 40 (80%) of the 50 participants approached completed the symptom checker assessment and usability survey. Their mean age was 39.3 (SD 15.9; range 18-76) years, and they were 65% (26/40) female, 68% (27/40) White, 48% (19/40) Hispanic or Latino, and 13% (5/40) Black or African American. Some cases had missing data or a lack of a clear ED diagnosis; 75% (30/40) were included in the analysis of diagnosis, and 93% (37/40) for triage. The sensitivity for at least one of the final ED diagnoses by Ada (based on its top 5 diagnoses) was 70% (95% CI 54%-86%), close to the mean sensitivity for the 3 physicians (on their top 3 diagnoses) of 68.9%. The physicians rated the Ada triage decisions as 62% (23/37) fully agree and 24% (9/37) safe but too cautious. It was rated as unsafe and too risky in 22% (8/37) of cases by at least one physician, in 14% (5/37) of cases by at least two physicians, and in 5% (2/37) of cases by all 3 physicians. Usability was rated highly; participants agreed or strongly agreed with the 7 Technology Acceptance Model usability questions with a mean score of 84.6%, although "satisfaction" and "enjoyment" were rated low.ConclusionsThis study provides preliminary evidence that a symptom checker can provide acceptable usability and diagnostic accuracy for patients with various urgent conditions. A total of 14% (5/37) of symptom checker triage recommendations were deemed unsafe and too risky by at least two physicians based on the symptoms recorded, similar to the results of studies on telephone and nurse triage. Larger studies are needed of diagnosis and triage performance with direct patient use in different clinical environments.

Project description:IntroductionCanadian patients presenting to the emergency department (ED) typically undergo a triage process where they are assessed by a specially trained nurse and assigned a Canadian Triage and Acuity Scale (CTAS) score, indicating their level of acuity and urgency of assessment. We sought to assess the ability of patients to self-triage themselves through use of one of two of our proprietary self-triage tools, and how this would compare with the standard nurse-driven triage process.MethodsWe enrolled a convenience sample of ambulatory ED patients aged 17 years or older who presented with chief complaints of chest pain, abdominal pain, breathing problems, or musculoskeletal pain. Participants completed one, or both, of an algorithm generated self-triage (AGST) survey, or visual acuity scale (VAS) based self-triage tool which subsequently generated a CTAS score. Our primary outcome was to assess the accuracy of these tools to the CTAS score generated through the nurse-driven triage process.ResultsA total of 223 patients were included in our analysis. Of these, 32 (14.3%) presented with chest pain, 25 (11.2%) with shortness of breath, 75 (33.6%) with abdominal pain, and 91 (40.8%) with musculoskeletal pain. Of the total number of patients, 142 (47.2%) completed the AGST tool, 159 (52.8%) completed the VAS tool and 78 (25.9%) completed both tools. When compared to the nurse-driven triage standard, both the AGST and VAS tools had poor levels of agreement for each of the four presenting complaints.ConclusionsSelf-triage through use of an AGST or VAS tool is inaccurate compared to the established standard of nurse-driven triage. Although existing literature exists which suggests that self-triage tools developed for specific subsets of complaints may be feasible, our results would suggest that adopting the self-triage approach on a broader scale for all-comers to the ED does not appear to be a viable option to enhance the current triage process. Further study is required to show if self-triage can be used in the ED to optimize the triage process.

Project description:ImportanceEmergency department (ED) triage substantially affects how long patients wait for care but triage scoring relies on few objective criteria. Prior studies suggest that Black and Hispanic patients receive unequal triage scores, paralleled by disparities in the depth of physician evaluations.ObjectivesTo examine whether racial disparities in triage scores and physician evaluations are present across a multicenter network of academic and community hospitals and evaluate whether patients who do not speak English face similar disparities.Design, setting, and participantsThis was a cross-sectional, multicenter study examining adults presenting between February 28, 2019, and January 1, 2023, across the Mass General Brigham Integrated Health Care System, encompassing 7 EDs: 2 urban academic hospitals and 5 community hospitals. Analysis included all patients presenting with 1 of 5 common chief symptoms.ExposuresEmergency department nurse-led triage and physician evaluation.Main outcomes and measuresAverage Triage Emergency Severity Index [ESI] score and average visit work relative value units [wRVUs] were compared across symptoms and between individual minority racial and ethnic groups and White patients.ResultsThere were 249 829 visits (149 861 female [60%], American Indian or Alaska Native 0.2%, Asian 3.3%, Black 11.8%, Hispanic 18.8%, Native Hawaiian or Other Pacific Islander <0.1%, White 60.8%, and patients identifying as Other race or ethnicity 5.1%). Median age was 48 (IQR, 29-66) years. White patients had more acute ESI scores than Hispanic or Other patients across all symptoms (eg, chest pain: Hispanic, 2.68 [95% CI, 2.67-2.69]; White, 2.55 [95% CI, 2.55-2.56]; Other, 2.66 [95% CI, 2.64-2.68]; P < .001) and Black patients across most symptoms (nausea/vomiting: Black, 2.97 [95% CI, 2.96-2.99]; White: 2.90 [95% CI, 2.89-2.91]; P < .001). These differences were reversed for wRVUs (chest pain: Black, 4.32 [95% CI, 4.25-4.39]; Hispanic, 4.13 [95% CI, 4.08-4.18]; White 3.55 [95% CI, 3.52-3.58]; Other 3.96 [95% CI, 3.84-4.08]; P < .001). Similar patterns were seen for patients whose primary language was not English.Conclusions and relevanceIn this cross-sectional study, patients who identified as Black, Hispanic, and Other race and ethnicity were assigned less acute ESI scores than their White peers despite having received more involved physician workups, suggesting some degree of mistriage. Clinical decision support systems might reduce these disparities but would require careful calibration to avoid replicating bias.

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.

Project description:BackgroundSyndromic surveillance in emergency departments (EDs) may be used to deliver early warnings of increases in disease activity, to provide situational awareness during events of public health significance, to supplement other information on trends in acute disease and injury, and to support the development and monitoring of prevention or response strategies. Changes in mental health related ED presentations may be relevant to these goals, provided they can be identified accurately and efficiently. This study aimed to measure the accuracy of using diagnostic codes in electronic ED presentation records to identify mental health-related visits.MethodsWe selected a random sample of 500 records from a total of 1,815,588 ED electronic presentation records from 59 NSW public hospitals during 2010. ED diagnoses were recorded using any of ICD-9, ICD-10 or SNOMED CT classifications. Three clinicians, blinded to the automatically generated syndromic grouping and each other's classification, reviewed the triage notes and classified each of the 500 visits as mental health-related or not. A "mental health problem presentation" for the purposes of this study was defined as any ED presentation where either a mental disorder or a mental health problem was the reason for the ED visit. The combined clinicians' assessment of the records was used as reference standard to measure the sensitivity, specificity, and positive and negative predictive values of the automatic classification of coded emergency department diagnoses. Agreement between the reference standard and the automated coded classification was estimated using the Kappa statistic.ResultsAgreement between clinician's classification and automated coded classification was substantial (Kappa = 0.73. 95% CI: 0.58 - 0.87). The automatic syndromic grouping of coded ED diagnoses for mental health-related visits was found to be moderately sensitive (68% 95% CI: 46%-84%) and highly specific at 99% (95% CI: 98%-99.7%) when compared with the reference standard in identifying mental health related ED visits. Positive predictive value was 81% (95% CI: 0.57 - 0.94) and negative predictive value was 98% (95% CI: 0.97-0.99).ConclusionsMental health presentations identified using diagnoses coded with various classifications in electronic ED presentation records offers sufficient accuracy for application in near real-time syndromic surveillance.

Project description:BackgroundDevelopment of emergency department (ED) triage systems that accurately differentiate and prioritize critically ill from stable patients remains challenging. We used machine learning models to predict clinical outcomes, and then compared their performance with that of a conventional approach-the Emergency Severity Index (ESI).MethodsUsing National Hospital and Ambulatory Medical Care Survey (NHAMCS) ED data, from 2007 through 2015, we identified all adult patients (aged ≥ 18 years). In the randomly sampled training set (70%), using routinely available triage data as predictors (e.g., demographics, triage vital signs, chief complaints, comorbidities), we developed four machine learning models: Lasso regression, random forest, gradient boosted decision tree, and deep neural network. As the reference model, we constructed a logistic regression model using the five-level ESI data. The clinical outcomes were critical care (admission to intensive care unit or in-hospital death) and hospitalization (direct hospital admission or transfer). In the test set (the remaining 30%), we measured the predictive performance, including area under the receiver-operating-characteristics curve (AUC) and net benefit (decision curves) for each model.ResultsOf 135,470 eligible ED visits, 2.1% had critical care outcome and 16.2% had hospitalization outcome. In the critical care outcome prediction, all four machine learning models outperformed the reference model (e.g., AUC, 0.86 [95%CI 0.85-0.87] in the deep neural network vs 0.74 [95%CI 0.72-0.75] in the reference model), with less under-triaged patients in ESI triage levels 3 to 5 (urgent to non-urgent). Likewise, in the hospitalization outcome prediction, all machine learning models outperformed the reference model (e.g., AUC, 0.82 [95%CI 0.82-0.83] in the deep neural network vs 0.69 [95%CI 0.68-0.69] in the reference model) with less over-triages in ESI triage levels 1 to 3 (immediate to urgent). In the decision curve analysis, all machine learning models consistently achieved a greater net benefit-a larger number of appropriate triages considering a trade-off with over-triages-across the range of clinical thresholds.ConclusionsCompared to the conventional approach, the machine learning models demonstrated a superior performance to predict critical care and hospitalization outcomes. The application of modern machine learning models may enhance clinicians' triage decision making, thereby achieving better clinical care and optimal resource utilization.

Project description:BackgroundEmergency department (ED) crowding is associated with negative health outcomes, patient dissatisfaction, and longer length of stay (LOS). The addition of advanced diagnostic imaging (ADI), namely CT, ultrasound (U/S), and MRI to ED encounter work up is a predictor of longer length of stay. Earlier and improved prediction of patients' need for advanced imaging may improve overall ED efficiency. The aim of the study was to detect the association between ADI utilization and the structured and unstructured information immediately available during ED triage, and to develop and validate models to predict utilization of ADI during an ED encounter.MethodsWe used the United States National Hospital Ambulatory Medical Care Survey data from 2009 to 2014 to examine which sociodemographic and clinical factors immediately available at ED triage were associated with the utilization of CT, U/S, MRI, and multiple ADI during a patient's ED stay. We used natural language processing (NLP) topic modeling to incorporate free-text reason for visit data available at time of ED triage in addition to other structured patient data to predict the use of ADI using multivariable logistic regression models.ResultsAmong the 139,150 adult ED visits from a national probability sample of hospitals across the U.S, 21.9% resulted in ADI use, including 16.8% who had a CT, 3.6% who had an ultrasound, 0.4% who had an MRI, and 1.2% of the population who had multiple types of ADI. The c-statistic of the predictive models was greater than or equal to 0.78 for all imaging outcomes, and the addition of text-based reason for visit information improved the accuracy of all predictive models.ConclusionsPatient information immediately available during ED triage can accurately predict the eventual use of advanced diagnostic imaging during an ED visit. Such models have the potential to be incorporated into the ED triage workflow in order to more rapidly identify patients who may require advanced imaging during their ED stay and assist with medical decision-making.

Dataset Information

Accuracy of Referral and Phone-Triage Diagnoses in an Eye Emergency Department.

Publications

Accuracy of Referral and Phone-Triage Diagnoses in an Eye Emergency Department.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets