Dataset Information

A Trend-Based Early Warning Score Can Be Implemented in a Hospital Electronic Medical Record to Effectively Predict Inpatient Deterioration.

ABSTRACT:

Objectives

To determine whether a statistically derived, trend-based, deterioration index is superior to other early warning scores at predicting adverse events and whether it can be integrated into an electronic medical record to enable real-time alerts.

Design

Forty-three variables and their trends from cases and controls were used to develop a logistic model and deterioration index to predict patient deterioration greater than or equal to 1 hour prior to an adverse event.

Setting

Two large Australian teaching hospitals.

Patients

Cases were considered as patients who suffered adverse events (unexpected death, unplanned ICU transfer, urgent surgery, and rapid-response alert) between August 1, 2016, and April 1, 2019.

Interventions

The logistic model and deterioration index were tested on historical data and then integrated into an electronic medical record for a 6-month prospective "silent" validation.

Measurements and main results

Data were acquired from 258,732 admissions. There were 8,002 adverse events. The addition of vital sign and laboratory trend values to the logistic model increased the area under the curve from 0.84 to 0.89 and the sensitivity to predict an adverse event 1-48 hours prior from 0.35 to 0.41. A 48-hour simulation showed that the logistic model had a higher area under the curve than the Modified Early Warning Score and National Early Warning Score (0.87 vs 0.74 vs 0.71). During the silently run prospective trial, the sensitivity of the deterioration index to detect adverse event any time prior to the adverse event was 0.474, 0.369 1 hour prior, and 0.327 4 hours prior, with a specificity of 0.972.

Conclusions

A deterioration prediction model was developed using patient demographics, ward-based observations, laboratory values, and their trends. The model's outputs were converted to a deterioration index that was successfully integrated into a live hospital electronic medical record. The sensitivity and specificity of the tool to detect inpatient deterioration were superior to traditional early warning scores.

SUBMITTER: Bell D

PROVIDER: S-EPMC8439669 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:OBJECTIVES:Previous studies have looked at National Early Warning Score performance in predicting in-hospital deterioration and death, but data are lacking with respect to patient outcomes following implementation of National Early Warning Score. We sought to determine the effectiveness of National Early Warning Score implementation on predicting and preventing patient deterioration in a clinical setting. DESIGN:Retrospective cohort study. SETTING:Tertiary care academic facility and a community hospital. PATIENTS:Patients 18 years old or older hospitalized from March 1, 2014, to February 28, 2015, during preimplementation of National Early Warning Score to August 1, 2015, to July 31, 2016, after National Early Warning Score was implemented. INTERVENTIONS:Implementation of National Early Warning Score within the electronic health record and associated best practice alert. MEASUREMENTS AND MAIN RESULTS:In this study of 85,322 patients (42,402 patients pre-National Early Warning Score and 42,920 patients post-National Early Warning Score implementation), the primary outcome of rate of ICU transfer or death did not change after National Early Warning Score implementation, with adjusted hazard ratio of 0.94 (0.84-1.05) and 0.90 (0.77-1.05) at our academic and community hospital, respectively. In total, 175,357 best practice advisories fired during the study period, with the best practice advisory performing better at the community hospital than the academic at predicting an event within 12 hours 7.4% versus 2.2% of the time, respectively. Retraining National Early Warning Score with newly generated hospital-specific coefficients improved model performance. CONCLUSIONS:At both our academic and community hospital, National Early Warning Score had poor performance characteristics and was generally ignored by frontline nursing staff. As a result, National Early Warning Score implementation had no appreciable impact on defined clinical outcomes. Refitting of the model using site-specific data improved performance and supports validating predictive models on local data.

Project description:BackgroundThe rapid deterioration observed in the condition of some hospitalized patients can be attributed to either disease progression or imperfect triage and level of care assignment after their admission. An early warning system (EWS) to identify patients at high risk of subsequent intrahospital death can be an effective tool for ensuring patient safety and quality of care and reducing avoidable harm and costs.ObjectiveThe aim of this study was to prospectively validate a real-time EWS designed to predict patients at high risk of inpatient mortality during their hospital episodes.MethodsData were collected from the system-wide electronic medical record (EMR) of two acute Berkshire Health System hospitals, comprising 54,246 inpatient admissions from January 1, 2015, to September 30, 2017, of which 2.30% (1248/54,246) resulted in intrahospital deaths. Multiple machine learning methods (linear and nonlinear) were explored and compared. The tree-based random forest method was selected to develop the predictive application for the intrahospital mortality assessment. After constructing the model, we prospectively validated the algorithms as a real-time inpatient EWS for mortality.ResultsThe EWS algorithm scored patients' daily and long-term risk of inpatient mortality probability after admission and stratified them into distinct risk groups. In the prospective validation, the EWS prospectively attained a c-statistic of 0.884, where 99 encounters were captured in the highest risk group, 69% (68/99) of whom died during the episodes. It accurately predicted the possibility of death for the top 13.3% (34/255) of the patients at least 40.8 hours before death. Important clinical utilization features, together with coded diagnoses, vital signs, and laboratory test results were recognized as impactful predictors in the final EWS.ConclusionsIn this study, we prospectively demonstrated the capability of the newly-designed EWS to monitor and alert clinicians about patients at high risk of in-hospital death in real time, thereby providing opportunities for timely interventions. This real-time EWS is able to assist clinical decision making and enable more actionable and effective individualized care for patients' better health outcomes in target medical facilities.