Dataset Information

Predicting hospital and emergency department utilization among community-dwelling older adults: Statistical and machine learning approaches.

ABSTRACT:

Objective

The objective of this study was to compare the performance of several commonly used machine learning methods to traditional statistical methods for predicting emergency department and hospital utilization among patients receiving publicly-funded home care services.

Study design and setting

We conducted a population-based retrospective cohort study of publicly-funded home care recipients in the Hamilton-Niagara-Haldimand-Brant region of southern Ontario, Canada between 2014 and 2016. Gradient boosted trees, neural networks, and random forests were tested against two variations of logistic regression for predicting three outcomes related to emergency department and hospital utilization within six months of a comprehensive home care clinical assessment. Models were trained on data from years 2014 and 2015 and tested on data from 2016. Performance was compared using logarithmic score, Brier score, AUC, and diagnostic accuracy measures.

Results

Gradient boosted trees achieved the best performance on all three outcomes. Gradient boosted trees provided small but statistically significant performance gains over both traditional methods on all three outcomes, while neural networks significantly outperformed logistic regression on two of three outcomes. However, sensitivity and specificity gains from using gradient boosted trees over logistic regression were only in the range of 1%-2% at several classification thresholds.

Conclusion

Gradient boosted trees and simple neural networks yielded small performance benefits over logistic regression for predicting emergency department and hospital utilization among patients receiving publicly-funded home care. However, the performance benefits were of negligible clinical importance.

SUBMITTER: Jones A

PROVIDER: S-EPMC6211724 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Predicting hospital and emergency department utilization among community-dwelling older adults: Statistical and machine learning approaches.

Jones Aaron A Costa Andrew P AP Pesevski Angelina A McNicholas Paul D PD

PloS one 20181101 11

<h4>Objective</h4>The objective of this study was to compare the performance of several commonly used machine learning methods to traditional statistical methods for predicting emergency department and hospital utilization among patients receiving publicly-funded home care services.<h4>Study design and setting</h4>We conducted a population-based retrospective cohort study of publicly-funded home care recipients in the Hamilton-Niagara-Haldimand-Brant region of southern Ontario, Canada between 20 ...[more]

PMID: 30383850

Similar Datasets

Project description:ObjectiveTo predict hospital admission at the time of ED triage using patient history in addition to information collected at triage.MethodsThis retrospective study included all adult ED visits between March 2014 and July 2017 from one academic and two community emergency rooms that resulted in either admission or discharge. A total of 972 variables were extracted per patient visit. Samples were randomly partitioned into training (80%), validation (10%), and test (10%) sets. We trained a series of nine binary classifiers using logistic regression (LR), gradient boosting (XGBoost), and deep neural networks (DNN) on three dataset types: one using only triage information, one using only patient history, and one using the full set of variables. Next, we tested the potential benefit of additional training samples by training models on increasing fractions of our data. Lastly, variables of importance were identified using information gain as a metric to create a low-dimensional model.ResultsA total of 560,486 patient visits were included in the study, with an overall admission risk of 29.7%. Models trained on triage information yielded a test AUC of 0.87 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.88) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on patient history yielded an AUC of 0.86 for LR (95% CI 0.86-0.87), 0.87 for XGBoost (95% CI 0.87-0.87) and 0.87 for DNN (95% CI 0.87-0.88). Models trained on the full set of variables yielded an AUC of 0.91 for LR (95% CI 0.91-0.91), 0.92 for XGBoost (95% CI 0.92-0.93) and 0.92 for DNN (95% CI 0.92-0.92). All algorithms reached maximum performance at 50% of the training set or less. A low-dimensional XGBoost model built on ESI level, outpatient medication counts, demographics, and hospital usage statistics yielded an AUC of 0.91 (95% CI 0.91-0.91).ConclusionMachine learning can robustly predict hospital admission using triage information and patient history. The addition of historical information improves predictive performance significantly compared to using triage information alone, highlighting the need to incorporate these variables into prediction models.

Project description:BACKGROUND:The recent literature reports promising results from using intelligent systems to support decision making in healthcare operations. Using these systems may lead to improved diagnostic and treatment protocols and to predict hospital bed demand. Predicting hospital bed demand in emergency department (ED) attendances could help resource allocation and reduce pressure on busy hospitals. However, there is still limited knowledge on whether intelligent systems can operate as fully autonomous, user-independent systems. OBJECTIVE:Compare the performance of a computer-based algorithm and humans in predicting hospital bed demand (admissions and discharges) based on the initial SOAP (Subjective, Objective, Assessment, Plan) records of the ED. METHODS:This was a retrospective cohort study that compared the performance of humans and machines in predicting hospital bed demand from an ED. It considered electronic medical records (EMR) of 9030 patients (230 used as a testing set, and hence evaluated both by humans and by an algorithm, and 8800 used as a training set exclusively by the algorithm) who visited the ED of a tertiary care and teaching public hospital located in Porto Alegre, Brazil between January and December 2014. The machine role was played by Support Vector Machine Classifier and the human prediction was performed by four ED physicians. Predictions were compared in terms of sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUROC). RESULTS:All graders achieved similar accuracies. The accuracy by AUROC for the testing set was 0.82 [95% confidence interval (CI) of 0.77-0.87], 0.80 (95% CI: 0.75-0.85), 0.76 (95% CI: 0.71-0.81) for novice physicians, machine, experienced physicians, respectively. Processing time per test EMR was 0.00812±0.0009 seconds. In contrast, novice physicians took on average 156.80 seconds per test EMR, while experienced physicians took on average 56.40 seconds per test EMR. CONCLUSIONS:Our data indicated that the system could predict patient admission or discharge states with 80% accuracy, which was similar the performance of novice and experienced physicians. These results suggested that the algorithm could operate as an autonomous and independent system to complete this task.

Project description:BACKGROUND:Whether types of hospitals with high readmission rates also have high overall postdischarge acute care utilization (including emergency department and observation care) is unknown. DESIGN:Cross-sectional analysis. SUBJECTS:Nonfederal United States acute care hospitals. MEASURES:Using methodology established by the Centers for Medicare & Medicaid Services, we calculated each hospital's "excess days in acute care" for fee-for-service (FFS) Medicare beneficiaries aged over 65 years discharged after hospitalization for acute myocardial infarction, heart failure (HF), or pneumonia, representing the mean difference between predicted and expected total days of acute care utilization in the 30 days following hospital discharge, per 100 discharges. We assessed the multivariable association of 8 hospital characteristics with excess days in acute care and the proportion of hospitals with each characteristic that were statistical outliers (95% credible interval estimate does not include 0). RESULTS:We included 2184 hospitals for acute myocardial infarction [228 (10.4%) better than expected, 549 (25.1%) worse than expected], 3720 hospitals for HF [484 (13.0%) better and 840 (22.6%) worse], and 4195 hospitals for pneumonia [673 (16.0%) better, 1005 (24.0%) worse]. Results for all conditions were similar. Worse than expected outliers for pneumonia included: 18.8% of safety net hospitals versus 26.1% of nonsafety net hospitals; 16.7% of public hospitals versus 33.1% of for-profit hospitals; 19.5% of nonteaching hospitals versus 52.2% of major teaching hospitals; 7.9% of rural hospitals versus 42.1% of large urban hospitals; 5.9% of hospitals with 24-<50 beds versus 58% of hospitals with >500 beds; and 29.0% of hospitals with nurse-to-bed ratios >1.0-1.5 versus 21.7% of hospitals with ratios >2.0. CONCLUSIONS:Including emergency department and observation stays in measures of postdischarge utilization produces similar results as measuring only readmissions in that major teaching, urban and for-profit hospitals still perform disproportionately poorly versus nonteaching or public hospitals. However, it enables identification of more outliers and a more granular assessment of the association of hospital factors and outcomes.

Project description:BackgroundEmergency department (ED) visits show a high volatility over time. Therefore, EDs are likely to be crowded at peak-volume moments. ED crowding is a widely reported problem with negative consequences for patients as well as staff. Previous studies on the predictive value of weather variables on ED visits show conflicting results. Also, no such studies were performed in the Netherlands. Therefore, we evaluated prediction models for the number of ED visits in our large the Netherlands teaching hospital based on calendar and weather variables as potential predictors.MethodsData on all ED visits from June 2016 until December 31, 2019, were extracted. The 2016-2018 data were used as training set, the 2019 data as test set. Weather data were extracted from three publicly available datasets from the Royal Netherlands Meteorological Institute. Weather observations in proximity of the hospital were used to predict the weather in the hospital's catchment area by applying the inverse distance weighting interpolation method. The predictability of daily ED visits was examined by creating linear prediction models using stepwise selection; the mean absolute percentage error (MAPE) was used as measurement of fit.ResultsThe number of daily ED visits shows a positive time trend and a large impact of calendar events (higher on Mondays and Fridays, lower on Saturdays and Sundays, higher at special times such as carnival, lower in holidays falling on Monday through Saturday, and summer vacation). The weather itself was a better predictor than weather volatility, but only showed a small effect; the calendar-only prediction model had very similar coefficients to the calendar+weather model for the days of the week, time trend, and special time periods (both MAPE's were 8.7%).ConclusionsBecause of this similar performance, and the inaccuracy caused by weather forecasts, we decided the calendar-only model would be most useful in our hospital; it can probably be transferred for use in EDs of the same size and in a similar region. However, the variability in ED visits is considerable. Therefore, one should always anticipate potential unforeseen spikes and dips in ED visits that are not shown by the model.

Project description:BackgroundUrinary tract infection (UTI) is a common emergency department (ED) diagnosis with reported high diagnostic error rates. Because a urine culture, part of the gold standard for diagnosis of UTI, is usually not available for 24-48 hours after an ED visit, diagnosis and treatment decisions are based on symptoms, physical findings, and other laboratory results, potentially leading to overutilization, antibiotic resistance, and delayed treatment. Previous research has demonstrated inadequate diagnostic performance for both individual laboratory tests and prediction tools.ObjectiveOur aim, was to train, validate, and compare machine-learning based predictive models for UTI in a large diverse set of ED patients.MethodsSingle-center, multi-site, retrospective cohort analysis of 80,387 adult ED visits with urine culture results and UTI symptoms. We developed models for UTI prediction with six machine learning algorithms using demographic information, vitals, laboratory results, medications, past medical history, chief complaint, and structured historical and physical exam findings. Models were developed with both the full set of 211 variables and a reduced set of 10 variables. UTI predictions were compared between models and to proxies of provider judgment (documentation of UTI diagnosis and antibiotic administration).ResultsThe machine learning models had an area under the curve ranging from 0.826-0.904, with extreme gradient boosting (XGBoost) the top performing algorithm for both full and reduced models. The XGBoost full and reduced models demonstrated greatly improved specificity when compared to the provider judgment proxy of UTI diagnosis OR antibiotic administration with specificity differences of 33.3 (31.3-34.3) and 29.6 (28.5-30.6), while also demonstrating superior sensitivity when compared to documentation of UTI diagnosis with sensitivity differences of 38.7 (38.1-39.4) and 33.2 (32.5-33.9). In the admission and discharge cohorts using the full XGboost model, approximately 1 in 4 patients (4109/15855) would be re-categorized from a false positive to a true negative and approximately 1 in 11 patients (1372/15855) would be re-categorized from a false negative to a true positive.ConclusionThe best performing machine learning algorithm, XGBoost, accurately diagnosed positive urine culture results, and outperformed previously developed models in the literature and several proxies for provider judgment. Future prospective validation is warranted.

Project description:Objectives: Using the Andersen's Behavioral Model of Health Services Use, we explored social, behavioral, and health factors that are associated with emergency department (ED) utilization among underserved African American (AA) older adults in one of the most economically disadvantaged urban areas in South Los Angeles, California. Methods: This cross-sectional study recruited a convenience sample of 609 non-institutionalized AA older adults (age ? 65 years) from South Los Angeles, California. Participants were interviewed for demographic factors, self-rated health, chronic medication conditions (CMCs), pain, depressive symptoms, access to care, and continuity of care. Outcomes included 1 or 2+ ED visits in the last 12 months. Polynomial regression was used for data analysis. Results: Almost 41% of participants were treated at an ED during the last 12 months. In all, 27% of participants attended an ED once and 14% two or more times. Half of those with 6+ chronic conditions reported being treated at an ED once; one quarter at least twice. Factors that predicted no ED visit were male gender (OR = 0.50, 95% CI = 0.29-0.85), higher continuity of medical care (OR = 1.55, 95% CI = 1.04-2.31), individuals with two CMCs or less (OR = 2.61 (1.03-6.59), second tertile of pain severity (OR = 2.80, 95% CI = 1.36-5.73). Factors that predicted only one ED visit were male gender (OR = 0.45, 95% CI = 0.25-0.82), higher continuity of medical care (OR = 1.39, 95% CI = 1.01-2.15) and second tertile of pain severity (OR = 2.42, 95% CI = 1.13-5.19). Conclusions: This study documented that a lack of continuity of care for individuals with multiple chronic conditions leads to a higher rate of ED presentations. The results are significant given that ED visits may contribute to health disparities among AA older adults. Future research should examine whether case management decreases ED utilization among underserved AA older adults with multiple chronic conditions and/or severe pain. To explore the generalizability of these findings, the study should be repeated in other settings.

Project description:BACKGROUND:Chronic spinal pain conditions affect millions of US adults and carry a high healthcare cost burden, both direct and indirect. Conservative interventions for spinal pain conditions, including chiropractic care, have been associated with lower healthcare costs and improvements in pain status in different clinical populations, including veterans. Little is currently known about predicting healthcare service utilization in the domain of conservative interventions for spinal pain conditions, including the frequency of use of chiropractic services. The purpose of this retrospective cohort study was to explore the use of supervised machine learning approaches to predicting one-year chiropractic service utilization by veterans receiving VA chiropractic care. METHODS:We included 19,946 veterans who entered the Musculoskeletal Diagnosis Cohort between October 1, 2003 and September 30, 2013 and utilized VA chiropractic services within one year of cohort entry. The primary outcome was one-year chiropractic service utilization following index chiropractic visit, split into quartiles represented by the following classes: 1 visit, 2 to 3 visits, 4 to 6 visits, and 7 or greater visits. We compared the performance of four multiclass classification algorithms (gradient boosted classifier, stochastic gradient descent classifier, support vector classifier, and artificial neural network) in predicting visit quartile using 158 sociodemographic and clinical features. RESULTS:The selected algorithms demonstrated poor prediction capabilities. Subset accuracy was 42.1% for the gradient boosted classifier, 38.6% for the stochastic gradient descent classifier, 41.4% for the support vector classifier, and 40.3% for the artificial neural network. The micro-averaged area under the precision-recall curve for each one-versus-rest classifier was 0.43 for the gradient boosted classifier, 0.38 for the stochastic gradient descent classifier, 0.43 for the support vector classifier, and 0.42 for the artificial neural network. Performance of each model yielded only a small positive shift in prediction probability (approximately 15%) compared to naïve classification. CONCLUSIONS:Using supervised machine learning to predict chiropractic service utilization remains challenging, with only a small shift in predictive probability over naïve classification and limited clinical utility. Future work should examine mechanisms to improve model performance.

Dataset Information

Predicting hospital and emergency department utilization among community-dwelling older adults: Statistical and machine learning approaches.

Objective

Study design and setting

Results

Conclusion

Publications

Predicting hospital and emergency department utilization among community-dwelling older adults: Statistical and machine learning approaches.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure