Dataset Information

Using machine learning to predict blood culture outcomes in the emergency department: a single-centre, retrospective, observational study.

ABSTRACT:

Objectives

To develop predictive models for blood culture (BC) outcomes in an emergency department (ED) setting.

Design

Retrospective observational study.

Setting

ED of a large teaching hospital in the Netherlands between 1 September 2018 and 24 June 2020.

Participants

Adult patients from whom BCs were collected in the ED. Data of demographic information, vital signs, administered medications in the ED and laboratory and radiology results were extracted from the electronic health record, if available at the end of the ED visits.

Main outcome measures

The primary outcome was the performance of two models (logistic regression and gradient boosted trees) to predict bacteraemia in ED patients, defined as at least one true positive BC collected at the ED.

Results

In 4885 out of 51 399 ED visits (9.5%), BCs were collected. In 598/4885 (12.2%) visits, at least one of the BCs was true positive. Both a gradient boosted tree model and a logistic regression model showed good performance in predicting BC results with area under curve of the receiver operating characteristics of 0.77 (95% CI 0.73 to 0.82) and 0.78 (95% CI 0.73 to 0.82) in the test sets, respectively. In the gradient boosted tree model, the optimal threshold would predict 69% of BCs in the test set to be negative, with a negative predictive value of over 94%.

Conclusions

Both models can accurately identify patients with low risk of bacteraemia at the ED in this single-centre setting and may be useful to reduce unnecessary BCs and associated healthcare costs. Further studies are necessary for validation and to investigate the potential clinical benefits and possible risks after implementation.

SUBMITTER: Boerman AW

PROVIDER: S-EPMC8728456 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.

Project description:BACKGROUND:Little is known regarding paediatric medical emergency calls to Danish Emergency Medical Dispatch Centres (EMDC). This study aimed to investigate these calls, specifically the medical issues leading to them and the pre-hospital units dispatched to the paediatric emergencies. METHODS:We performed a retrospective, observational study on paediatric medical emergency calls managed by the EMDC in the Region of Southern Denmark in February 2016. We reviewed audio recordings of emergency calls and ambulance records to identify calls concerning patients ≤ 15 years. We examined EMDC dispatch records to establish how the medical issues leading to these calls were classified and which pre-hospital units were dispatched to the paediatric emergencies. We analysed the data using descriptive statistics. RESULTS:Of a total of 7052 emergency calls in February 2016, 485 (6.9%) concerned patients ≤ 15 years. We excluded 19 and analysed the remaining 466. The reported medical issues were commonly classified as: "seizures" (22.1%), "sick child" (18.9%) and "unclear problem" (12.9%). The overall most common pre-hospital response was immediate dispatch of an ambulance with sirens and lights with a supporting physician-manned mobile emergency care unit (56.4%). The classification of medical issues and the dispatched pre-hospital units varied with patient age. DISCUSSION:We believe our results might help focus the paediatric training received by emergency medical dispatch staff on commonly encountered medical issues, such as the symptoms and conditions pertaining to the symptom categories "seizures" and "sick child". Furthermore, the results could prove useful in hypothesis generation for future studies examining paediatric medical emergency calls. CONCLUSION:Almost 7% of all calls concerned patients ≤ 15 years. Medical issues pertaining to the symptom categories "seizures", "sick child" and "unclear problem" were common and the calls commonly resulted in urgent pre-hospital responses.

Project description:BackgroundUrinary tract infection (UTI) is a common emergency department (ED) diagnosis with reported high diagnostic error rates. Because a urine culture, part of the gold standard for diagnosis of UTI, is usually not available for 24-48 hours after an ED visit, diagnosis and treatment decisions are based on symptoms, physical findings, and other laboratory results, potentially leading to overutilization, antibiotic resistance, and delayed treatment. Previous research has demonstrated inadequate diagnostic performance for both individual laboratory tests and prediction tools.ObjectiveOur aim, was to train, validate, and compare machine-learning based predictive models for UTI in a large diverse set of ED patients.MethodsSingle-center, multi-site, retrospective cohort analysis of 80,387 adult ED visits with urine culture results and UTI symptoms. We developed models for UTI prediction with six machine learning algorithms using demographic information, vitals, laboratory results, medications, past medical history, chief complaint, and structured historical and physical exam findings. Models were developed with both the full set of 211 variables and a reduced set of 10 variables. UTI predictions were compared between models and to proxies of provider judgment (documentation of UTI diagnosis and antibiotic administration).ResultsThe machine learning models had an area under the curve ranging from 0.826-0.904, with extreme gradient boosting (XGBoost) the top performing algorithm for both full and reduced models. The XGBoost full and reduced models demonstrated greatly improved specificity when compared to the provider judgment proxy of UTI diagnosis OR antibiotic administration with specificity differences of 33.3 (31.3-34.3) and 29.6 (28.5-30.6), while also demonstrating superior sensitivity when compared to documentation of UTI diagnosis with sensitivity differences of 38.7 (38.1-39.4) and 33.2 (32.5-33.9). In the admission and discharge cohorts using the full XGboost model, approximately 1 in 4 patients (4109/15855) would be re-categorized from a false positive to a true negative and approximately 1 in 11 patients (1372/15855) would be re-categorized from a false negative to a true positive.ConclusionThe best performing machine learning algorithm, XGBoost, accurately diagnosed positive urine culture results, and outperformed previously developed models in the literature and several proxies for provider judgment. Future prospective validation is warranted.

Project description:BackgroundDevelopment of emergency department (ED) triage systems that accurately differentiate and prioritize critically ill from stable patients remains challenging. We used machine learning models to predict clinical outcomes, and then compared their performance with that of a conventional approach-the Emergency Severity Index (ESI).MethodsUsing National Hospital and Ambulatory Medical Care Survey (NHAMCS) ED data, from 2007 through 2015, we identified all adult patients (aged ≥ 18 years). In the randomly sampled training set (70%), using routinely available triage data as predictors (e.g., demographics, triage vital signs, chief complaints, comorbidities), we developed four machine learning models: Lasso regression, random forest, gradient boosted decision tree, and deep neural network. As the reference model, we constructed a logistic regression model using the five-level ESI data. The clinical outcomes were critical care (admission to intensive care unit or in-hospital death) and hospitalization (direct hospital admission or transfer). In the test set (the remaining 30%), we measured the predictive performance, including area under the receiver-operating-characteristics curve (AUC) and net benefit (decision curves) for each model.ResultsOf 135,470 eligible ED visits, 2.1% had critical care outcome and 16.2% had hospitalization outcome. In the critical care outcome prediction, all four machine learning models outperformed the reference model (e.g., AUC, 0.86 [95%CI 0.85-0.87] in the deep neural network vs 0.74 [95%CI 0.72-0.75] in the reference model), with less under-triaged patients in ESI triage levels 3 to 5 (urgent to non-urgent). Likewise, in the hospitalization outcome prediction, all machine learning models outperformed the reference model (e.g., AUC, 0.82 [95%CI 0.82-0.83] in the deep neural network vs 0.69 [95%CI 0.68-0.69] in the reference model) with less over-triages in ESI triage levels 1 to 3 (immediate to urgent). In the decision curve analysis, all machine learning models consistently achieved a greater net benefit-a larger number of appropriate triages considering a trade-off with over-triages-across the range of clinical thresholds.ConclusionsCompared to the conventional approach, the machine learning models demonstrated a superior performance to predict critical care and hospitalization outcomes. The application of modern machine learning models may enhance clinicians' triage decision making, thereby achieving better clinical care and optimal resource utilization.

Project description:IntroductionWithin a few months coronavirus disease 2019 (COVID-19) evolved into a pandemic causing millions of cases worldwide, but it remains challenging to diagnose the disease in a timely fashion in the emergency department (ED). In this study we aimed to construct machine-learning (ML) models to predict severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) infection based on the clinical features of patients visiting an ED during the early COVID-19 pandemic.MethodsWe retrospectively collected the data of all patients who received reverse transcriptase polymerase chain reaction (RT-PCR) testing for SARS-CoV-2 at the ED of Baylor Scott & White All Saints Medical Center, Fort Worth, from February 23-May 12, 2020. The variables collected included patient demographics, ED triage data, clinical symptoms, and past medical history. The primary outcome was the confirmed diagnosis of COVID-19 (or SARS-CoV-2 infection) by a positive RT-PCR test result for SARS-CoV-2, and was used as the label for ML tasks. We used univariate analyses for feature selection, and variables with P<0.1 were selected for model construction. Samples were split into training and testing cohorts on a 60:40 ratio chronologically. We tried various ML algorithms to construct the best predictive model, and we evaluated performances with the area under the receiver operating characteristic curve (AUC) in the testing cohort.ResultsA total of 580 ED patients were tested for SARS-CoV-2 during the study periods, and 98 (16.9%) were identified as having the SARS-CoV-2 infection based on the RT-PCR results. Univariate analyses selected 21 features for model construction. We assessed three ML methods for performance: of the three methods, random forest outperformed the others with the best AUC result (0.86), followed by gradient boosting (0.83) and extra trees classifier (0.82).ConclusionThis study shows that it is feasible to use ML models as an initial screening tool for identifying patients with SARS-CoV-2 infection. Further validation will be necessary to determine how effectively this prediction model can be used prospectively in clinical practice.