Dataset Information

Man vs. machine: Predicting hospital bed demand from an emergency department.

ABSTRACT: BACKGROUND:The recent literature reports promising results from using intelligent systems to support decision making in healthcare operations. Using these systems may lead to improved diagnostic and treatment protocols and to predict hospital bed demand. Predicting hospital bed demand in emergency department (ED) attendances could help resource allocation and reduce pressure on busy hospitals. However, there is still limited knowledge on whether intelligent systems can operate as fully autonomous, user-independent systems. OBJECTIVE:Compare the performance of a computer-based algorithm and humans in predicting hospital bed demand (admissions and discharges) based on the initial SOAP (Subjective, Objective, Assessment, Plan) records of the ED. METHODS:This was a retrospective cohort study that compared the performance of humans and machines in predicting hospital bed demand from an ED. It considered electronic medical records (EMR) of 9030 patients (230 used as a testing set, and hence evaluated both by humans and by an algorithm, and 8800 used as a training set exclusively by the algorithm) who visited the ED of a tertiary care and teaching public hospital located in Porto Alegre, Brazil between January and December 2014. The machine role was played by Support Vector Machine Classifier and the human prediction was performed by four ED physicians. Predictions were compared in terms of sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUROC). RESULTS:All graders achieved similar accuracies. The accuracy by AUROC for the testing set was 0.82 [95% confidence interval (CI) of 0.77-0.87], 0.80 (95% CI: 0.75-0.85), 0.76 (95% CI: 0.71-0.81) for novice physicians, machine, experienced physicians, respectively. Processing time per test EMR was 0.00812±0.0009 seconds. In contrast, novice physicians took on average 156.80 seconds per test EMR, while experienced physicians took on average 56.40 seconds per test EMR. CONCLUSIONS:Our data indicated that the system could predict patient admission or discharge states with 80% accuracy, which was similar the performance of novice and experienced physicians. These results suggested that the algorithm could operate as an autonomous and independent system to complete this task.

SUBMITTER: Lucini FR

PROVIDER: S-EPMC7451657 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Man vs. machine: Predicting hospital bed demand from an emergency department.

Lucini Filipe Rissieri FR Reis Mateus Augusto Dos MAD Silveira Giovani José Caetano da GJCD Fogliatto Flavio Sanson FS Anzanello Michel José MJ Andrioli Giordanna Guerra GG Nicolaidis Rafael R Beltrame Rafael Coimbra Ferreira RCF Neyeloff Jeruza Lavanholi JL Schaan Beatriz D'Agord BD

PloS one 20200827 8

<h4>Background</h4>The recent literature reports promising results from using intelligent systems to support decision making in healthcare operations. Using these systems may lead to improved diagnostic and treatment protocols and to predict hospital bed demand. Predicting hospital bed demand in emergency department (ED) attendances could help resource allocation and reduce pressure on busy hospitals. However, there is still limited knowledge on whether intelligent systems can operate as fully a ...[more]

PMID: 32853217

Similar Datasets

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.

Project description:BackgroundEmergency department (ED) visits show a high volatility over time. Therefore, EDs are likely to be crowded at peak-volume moments. ED crowding is a widely reported problem with negative consequences for patients as well as staff. Previous studies on the predictive value of weather variables on ED visits show conflicting results. Also, no such studies were performed in the Netherlands. Therefore, we evaluated prediction models for the number of ED visits in our large the Netherlands teaching hospital based on calendar and weather variables as potential predictors.MethodsData on all ED visits from June 2016 until December 31, 2019, were extracted. The 2016-2018 data were used as training set, the 2019 data as test set. Weather data were extracted from three publicly available datasets from the Royal Netherlands Meteorological Institute. Weather observations in proximity of the hospital were used to predict the weather in the hospital's catchment area by applying the inverse distance weighting interpolation method. The predictability of daily ED visits was examined by creating linear prediction models using stepwise selection; the mean absolute percentage error (MAPE) was used as measurement of fit.ResultsThe number of daily ED visits shows a positive time trend and a large impact of calendar events (higher on Mondays and Fridays, lower on Saturdays and Sundays, higher at special times such as carnival, lower in holidays falling on Monday through Saturday, and summer vacation). The weather itself was a better predictor than weather volatility, but only showed a small effect; the calendar-only prediction model had very similar coefficients to the calendar+weather model for the days of the week, time trend, and special time periods (both MAPE's were 8.7%).ConclusionsBecause of this similar performance, and the inaccuracy caused by weather forecasts, we decided the calendar-only model would be most useful in our hospital; it can probably be transferred for use in EDs of the same size and in a similar region. However, the variability in ED visits is considerable. Therefore, one should always anticipate potential unforeseen spikes and dips in ED visits that are not shown by the model.

Project description:BackgroundUrinary tract infection (UTI) is a common emergency department (ED) diagnosis with reported high diagnostic error rates. Because a urine culture, part of the gold standard for diagnosis of UTI, is usually not available for 24-48 hours after an ED visit, diagnosis and treatment decisions are based on symptoms, physical findings, and other laboratory results, potentially leading to overutilization, antibiotic resistance, and delayed treatment. Previous research has demonstrated inadequate diagnostic performance for both individual laboratory tests and prediction tools.ObjectiveOur aim, was to train, validate, and compare machine-learning based predictive models for UTI in a large diverse set of ED patients.MethodsSingle-center, multi-site, retrospective cohort analysis of 80,387 adult ED visits with urine culture results and UTI symptoms. We developed models for UTI prediction with six machine learning algorithms using demographic information, vitals, laboratory results, medications, past medical history, chief complaint, and structured historical and physical exam findings. Models were developed with both the full set of 211 variables and a reduced set of 10 variables. UTI predictions were compared between models and to proxies of provider judgment (documentation of UTI diagnosis and antibiotic administration).ResultsThe machine learning models had an area under the curve ranging from 0.826-0.904, with extreme gradient boosting (XGBoost) the top performing algorithm for both full and reduced models. The XGBoost full and reduced models demonstrated greatly improved specificity when compared to the provider judgment proxy of UTI diagnosis OR antibiotic administration with specificity differences of 33.3 (31.3-34.3) and 29.6 (28.5-30.6), while also demonstrating superior sensitivity when compared to documentation of UTI diagnosis with sensitivity differences of 38.7 (38.1-39.4) and 33.2 (32.5-33.9). In the admission and discharge cohorts using the full XGboost model, approximately 1 in 4 patients (4109/15855) would be re-categorized from a false positive to a true negative and approximately 1 in 11 patients (1372/15855) would be re-categorized from a false negative to a true positive.ConclusionThe best performing machine learning algorithm, XGBoost, accurately diagnosed positive urine culture results, and outperformed previously developed models in the literature and several proxies for provider judgment. Future prospective validation is warranted.

Project description:IntroductionAn increasing shortage of donor blood is expected, considering the demographic change in Germany. Due to the short shelf life and varying daily fluctuations in consumption, the storage of platelet concentrates (PCs) becomes challenging. This emphasizes the need for reliable prediction of needed PCs for the blood bank inventories. Therefore, the objective of this study was to evaluate multimodal data from multiple source systems within a hospital to predict the number of platelet transfusions in 3 days on a per-patient level.MethodsData were collected from 25,190 (42% female and 58% male) patients between 2017 and 2021. For each patient, the number of received PCs, platelet count blood tests, drugs causing thrombocytopenia, acute platelet diseases, procedures, age, gender, and the period of a patient's hospital stay were collected. Two models were trained on samples using a sliding window of 7 days as input and a day 3 target. The model predicts whether a patient will be transfused 3 days in the future. The model was trained with an excessive hyperparameter search using patient-level repeated 5-fold cross-validation to optimize the average macro F2-score.ResultsThe trained models were tested on 5,022 unique patients. The best-performing model has a specificity of 0.99, a sensitivity of 0.37, an area under the precision-recall curve score of 0.45, an MCC score of 0.43, and an F1-score of 0.43. However, the model does not generalize well for cases when the need for a platelet transfusion is recognized.ConclusionA patient AI-based platelet forecast could improve logistics management and reduce blood product waste. In this study, we build the first model to predict patient individual platelet demand. To the best of our knowledge, we are the first to introduce this approach. Our model predicts the need for platelet units for 3 days in the future. While sensitivity underperforms, specificity performs reliably. The model may be of clinical use as a pretest for potential patients needing a platelet transfusion within the next 3 days. As sensitivity needs to be improved, further studies should introduce deep learning and wider patient characterization to the methodological multimodal, multisource data approach. Furthermore, a hospital-wide consumption of PCs could be derived from individual predictions.

Project description:ImportanceHospital readmissions are associated with patient harm and expense. Ways to prevent hospital readmissions have focused on identifying patients at greatest risk using prediction scores.ObjectiveTo identify the type of score that best predicts hospital readmissions.Design, setting, and participantsThis prognostic study included 14?062 consecutive adult hospital patients with 16?649 discharges from a tertiary care center, suburban community hospital, and urban critical access hospital in Maryland from September 1, 2016, through December 31, 2016. Patients not included as eligible discharges by the Centers for Medicare & Medicaid Services or the Chesapeake Regional Information System for Our Patients were excluded. A machine learning rank score, the Baltimore score (B score) developed using a machine learning technique, for each individual hospital using data from the 2 years before September 1, 2016, was compared with standard readmission risk assessment scores to predict 30-day unplanned readmissions.Main outcomes and measuresThe 30-day readmission rate evaluated using various readmission scores: B score, HOSPITAL score, modified LACE score, and Maxim/RightCare score.ResultsOf the 10?732 patients (5605 [52.2%] male; mean [SD] age, 54.56 [22.42] years) deemed to be eligible for the study, 1422 were readmitted. The area under the receiver operating characteristic curve (AUROC) for individual rules was 0.63 (95% CI, 0.61-0.65) for the HOSPITAL score, which was significantly lower than the 0.66 for modified LACE score (95% CI, 0.64-0.68; P?<?.001). The B score machine learning score was significantly better than all other scores; 48 hours after admission, the AUROC of the B score was 0.72 (95% CI, 0.70-0.73), which increased to 0.78 (95% CI, 0.77-0.79) at discharge (all P?<?.001). At the hospital using Maxim/RightCare score, the AUROC was 0.63 (95% CI, 0.59-0.69) for HOSPITAL, 0.64 (95% CI, 0.61-0.68) for Maxim/RightCare, and 0.66 (95% CI, 0.62-0.69) for modified LACE score. The B score was 0.72 (95% CI, 0.69-0.75) 48 hours after admission and 0.81 (95% CI, 0.79-0.84) at discharge. In directly comparing the B score with the sensitivity at cutoff values for modified LACE, HOSPITAL, and Maxim/RightCare scores, the B score was able to identify the same number of readmitted patients while flagging 25.5% to 54.9% fewer patients.Conclusions and relevanceAmong 3 hospitals in different settings, an automated machine learning score better predicted readmissions than commonly used readmission scores. More efficiently targeting patients at higher risk of readmission may be the first step toward potentially preventing readmissions.

Project description:ObjectivesTo predict 72-h and 9-day emergency department (ED) return by using gradient boosting on an expansive set of clinical variables from the electronic health record.MethodsThis retrospective study included all adult discharges from a level 1 trauma center ED and a community hospital ED covering the period of March 2013 to July 2017. A total of 1500 variables were extracted for each visit, and samples split randomly into training, validation, and test sets (80%, 10%, and 10%). Gradient boosting models were fit on 3 selections of the data: administrative data (demographics, prior hospital usage, and comorbidity categories), data available at triage, and the full set of data available at discharge. A logistic regression (LR) model built on administrative data was used for baseline comparison. Finally, the top 20 most informative variables identified from the full gradient boosting models were used to build a reduced model for each outcome.ResultsA total of 330 631 discharges were available for analysis, with 29 058 discharges (8.8%) resulting in 72-h return and 52 748 discharges (16.0%) resulting in 9-day return to either ED. LR models using administrative data yielded test AUCs of 0.69 (95% confidence interval [CI] 0.68-0.70) and 0.71(95% CI 0.70-0.72), while gradient boosting models using administrative data yielded test AUCs of 0.73 (95% CI 0.72-0.74) and 0.74 (95% CI 0.73-0.74) for 72-h and 9-day return, respectively. Gradient boosting models using variables available at triage yielded test AUCs of 0.75 (95% CI 0.74-0.76) and 0.75 (95% CI 0.74-0.75), while those using the full set of variables yielded test AUCs of 0.76 (95% CI 0.75-0.77) and 0.75 (95% CI 0.75-0.76). Reduced models using the top 20 variables yielded test AUCs of 0.73 (95% CI 0.71-0.74) and 0.73 (95% CI 0.72-0.74).Discussion and conclusionGradient boosting models leveraging clinical data are superior to LR models built on administrative data at predicting 72-h and 9-day returns.

Project description:BackgroundA possible downstream effect of high in-hospital bed occupancy is that patients in the emergency department (ED) who would benefit from in-hospital care are denied admission. The present study aimed at evaluating this hypothesis through investigating associations between in-hospital bed occupancy at the time of presentation in the ED and the probability for unplanned 72-hour (72-h) revisits to the ED among patients discharged at index. A second outcome was unplanned 72-h revisits resulting in admission.MethodsAll visits to the ED of a 420-bed emergency hospital in southern Sweden between 1 January 2011 and 31 December 2012, which did not result in admission, death, or transfer to another hospital were included. Revisiting fractions were computed for in-hospital occupancy intervals <85%, 85% to 90%, 90% to 95%, 95% to 100%, 100% to 105%, and ?105%. Multivariate models were constructed in an attempt to take confounding factors from, e.g., presenting complaints, age, referral status, and triage priority into account.ResultsIncluded in the study are 81,878 visits. The fraction of unplanned 72-h revisits/unplanned 72-h revisits resulting in admission was 5.8%/1.4% overall, 6.2%/1.4% for occupancy <85%, 6.4%/1.5% for occupancy 85% to 90%, 5.8%/1.4% for occupancy 90% to 95%, 6.0%/1.6% for occupancy 95% to 100%, 5.4%/1.6% for occupancy 100% to 105%, and 4.9%/1.4% for occupancy ?105%. In the multivariate models, a trend to lower probability of unplanned 72-h revisits was observed at occupancy ?105% compared to occupancy <95% (OR 0.88, CI 0.76 to 1.01). No significant associations between in-hospital occupancy at index and the probability of making unplanned 72-h revisits resulting in admission were observed.ConclusionsThe lack of associations between in-hospital occupancy and unplanned 72-h revisits does not support the hypothesis that ED patients are inappropriately discharged when in-hospital beds are scarce. The results are reassuring as they indicate that physicians are able to make good decisions, also while resources are constrained.

Dataset Information

Man vs. machine: Predicting hospital bed demand from an emergency department.

Publications

Man vs. machine: Predicting hospital bed demand from an emergency department.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets