Dataset Information

Development, Implementation, and Evaluation of an In-Hospital Optimized Early Warning Score for Patient Deterioration.

ABSTRACT: Background. Identification of patients at risk of deteriorating during their hospitalization is an important concern. However, many off-shelf scores have poor in-center performance. In this article, we report our experience developing, implementing, and evaluating an in-hospital score for deterioration. Methods. We abstracted 3 years of data (2014-2016) and identified patients on medical wards that died or were transferred to the intensive care unit. We developed a time-varying risk model and then implemented the model over a 10-week period to assess prospective predictive performance. We compared performance to our currently used tool, National Early Warning Score. In order to aid clinical decision making, we transformed the quantitative score into a three-level clinical decision support tool. Results. The developed risk score had an average area under the curve of 0.814 (95% confidence interval = 0.79-0.83) versus 0.740 (95% confidence interval = 0.72-0.76) for the National Early Warning Score. We found the proposed score was able to respond to acute clinical changes in patients' clinical status. Upon implementing the score, we were able to achieve the desired positive predictive value but needed to retune the thresholds to get the desired sensitivity. Discussion. This work illustrates the potential for academic medical centers to build, refine, and implement risk models that are targeted to their patient population and work flow.

SUBMITTER: O'Brien C

PROVIDER: S-EPMC6956604 | biostudies-literature | 2020 Jan-Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development, Implementation, and Evaluation of an In-Hospital Optimized Early Warning Score for Patient Deterioration.

O'Brien Cara C Goldstein Benjamin A BA Shen Yueqi Y Phelan Matthew M Lambert Curtis C Bedoya Armando D AD Steorts Rebecca C RC

MDM policy & practice 20200110 1

<b>Background.</b> Identification of patients at risk of deteriorating during their hospitalization is an important concern. However, many off-shelf scores have poor in-center performance. In this article, we report our experience developing, implementing, and evaluating an in-hospital score for deterioration. <b>Methods.</b> We abstracted 3 years of data (2014-2016) and identified patients on medical wards that died or were transferred to the intensive care unit. We developed a time-varying ris ...[more]

PMID: 31976373

Similar Datasets

Project description:ObjectivesPrevious studies have looked at National Early Warning Score performance in predicting in-hospital deterioration and death, but data are lacking with respect to patient outcomes following implementation of National Early Warning Score. We sought to determine the effectiveness of National Early Warning Score implementation on predicting and preventing patient deterioration in a clinical setting.DesignRetrospective cohort study.SettingTertiary care academic facility and a community hospital.PatientsPatients 18 years old or older hospitalized from March 1, 2014, to February 28, 2015, during preimplementation of National Early Warning Score to August 1, 2015, to July 31, 2016, after National Early Warning Score was implemented.InterventionsImplementation of National Early Warning Score within the electronic health record and associated best practice alert.Measurements and main resultsIn this study of 85,322 patients (42,402 patients pre-National Early Warning Score and 42,920 patients post-National Early Warning Score implementation), the primary outcome of rate of ICU transfer or death did not change after National Early Warning Score implementation, with adjusted hazard ratio of 0.94 (0.84-1.05) and 0.90 (0.77-1.05) at our academic and community hospital, respectively. In total, 175,357 best practice advisories fired during the study period, with the best practice advisory performing better at the community hospital than the academic at predicting an event within 12 hours 7.4% versus 2.2% of the time, respectively. Retraining National Early Warning Score with newly generated hospital-specific coefficients improved model performance.ConclusionsAt both our academic and community hospital, National Early Warning Score had poor performance characteristics and was generally ignored by frontline nursing staff. As a result, National Early Warning Score implementation had no appreciable impact on defined clinical outcomes. Refitting of the model using site-specific data improved performance and supports validating predictive models on local data.

Project description:ObjectivesTo determine whether a statistically derived, trend-based, deterioration index is superior to other early warning scores at predicting adverse events and whether it can be integrated into an electronic medical record to enable real-time alerts.DesignForty-three variables and their trends from cases and controls were used to develop a logistic model and deterioration index to predict patient deterioration greater than or equal to 1 hour prior to an adverse event.SettingTwo large Australian teaching hospitals.PatientsCases were considered as patients who suffered adverse events (unexpected death, unplanned ICU transfer, urgent surgery, and rapid-response alert) between August 1, 2016, and April 1, 2019.InterventionsThe logistic model and deterioration index were tested on historical data and then integrated into an electronic medical record for a 6-month prospective "silent" validation.Measurements and main resultsData were acquired from 258,732 admissions. There were 8,002 adverse events. The addition of vital sign and laboratory trend values to the logistic model increased the area under the curve from 0.84 to 0.89 and the sensitivity to predict an adverse event 1-48 hours prior from 0.35 to 0.41. A 48-hour simulation showed that the logistic model had a higher area under the curve than the Modified Early Warning Score and National Early Warning Score (0.87 vs 0.74 vs 0.71). During the silently run prospective trial, the sensitivity of the deterioration index to detect adverse event any time prior to the adverse event was 0.474, 0.369 1 hour prior, and 0.327 4 hours prior, with a specificity of 0.972.ConclusionsA deterioration prediction model was developed using patient demographics, ward-based observations, laboratory values, and their trends. The model's outputs were converted to a deterioration index that was successfully integrated into a live hospital electronic medical record. The sensitivity and specificity of the tool to detect inpatient deterioration were superior to traditional early warning scores.

Project description:BackgroundThe National Early Warning Score (NEWS2) is currently recommended in the UK for the risk stratification of COVID-19 patients, but little is known about its ability to detect severe cases. We aimed to evaluate NEWS2 for the prediction of severe COVID-19 outcome and identify and validate a set of blood and physiological parameters routinely collected at hospital admission to improve upon the use of NEWS2 alone for medium-term risk stratification.MethodsTraining cohorts comprised 1276 patients admitted to King's College Hospital National Health Service (NHS) Foundation Trust with COVID-19 disease from 1 March to 30 April 2020. External validation cohorts included 6237 patients from five UK NHS Trusts (Guy's and St Thomas' Hospitals, University Hospitals Southampton, University Hospitals Bristol and Weston NHS Foundation Trust, University College London Hospitals, University Hospitals Birmingham), one hospital in Norway (Oslo University Hospital), and two hospitals in Wuhan, China (Wuhan Sixth Hospital and Taikang Tongji Hospital). The outcome was severe COVID-19 disease (transfer to intensive care unit (ICU) or death) at 14 days after hospital admission. Age, physiological measures, blood biomarkers, sex, ethnicity, and comorbidities (hypertension, diabetes, cardiovascular, respiratory and kidney diseases) measured at hospital admission were considered in the models.ResultsA baseline model of 'NEWS2 + age' had poor-to-moderate discrimination for severe COVID-19 infection at 14 days (area under receiver operating characteristic curve (AUC) in training cohort = 0.700, 95% confidence interval (CI) 0.680, 0.722; Brier score = 0.192, 95% CI 0.186, 0.197). A supplemented model adding eight routinely collected blood and physiological parameters (supplemental oxygen flow rate, urea, age, oxygen saturation, C-reactive protein, estimated glomerular filtration rate, neutrophil count, neutrophil/lymphocyte ratio) improved discrimination (AUC = 0.735; 95% CI 0.715, 0.757), and these improvements were replicated across seven UK and non-UK sites. However, there was evidence of miscalibration with the model tending to underestimate risks in most sites.ConclusionsNEWS2 score had poor-to-moderate discrimination for medium-term COVID-19 outcome which raises questions about its use as a screening tool at hospital admission. Risk stratification was improved by including readily available blood and physiological parameters measured at hospital admission, but there was evidence of miscalibration in external sites. This highlights the need for a better understanding of the use of early warning scores for COVID.

Project description:BackgroundPediatric early warning (PEW) scores represent a "track-and-trigger system" that identifies clinical deterioration in a patient's condition in the hours preceding a sentinel event. Before implementation, nurses reported feeling unprepared to identify and advocate for acutely ill patients owing to a lack of skills, vocabulary, and agency. We implemented a Pediatric Early Warning Score for Resource-Limited Settings (PEWS-RL) with nurses in a rural district hospital in Rwanda. Although PEW scores can improve clinical outcomes, empowering nurses in resource-limited settings to discuss patient acuity with physicians is a critical first step. Our primary aims were to train nurses to obtain more accurate vital signs and assess their importance as early warning signs of clinical deterioration and use PEW scores to improve communication between nurses and physicians.ImplementationThe PEWS-RL tool implementation began with a training program that was created through discussions with nurses, physicians, and the medical director of the hospital. The program included lectures and application of learned skills through direct clinical mentorship of nurses, as well as training of physicians regarding PEWS-RL as a communication tool.EvaluationThe PEWS-RL protocol was evaluated based on pre- and post-tests to assess improvement in nurses' knowledge and skill, as well as skills assessments of accurate recognition of clinical deterioration. All 6 nurses passed skill testing with >80% accuracy. Nurses' feelings of empowerment to advocate for patients and to escalate care were assessed through pre- and post-training interviews. Nurses described increased confidence in calling for physician support.DiscussionImplementation of PEW scores increased nurses' technical skills and feelings of confidence and empowerment; however, the low-resource setting presented major challenges. Barriers to sustainable implementation include the rapid ward staff turnover as well as limited physician buy-in. Nevertheless, the PEWS-RL tool has the potential to empower nurses and improve patient outcomes if fully embraced by staff.

Project description:ObjectiveTo provide an overview and critical appraisal of early warning scores for adult hospital patients.DesignSystematic review.Data sourcesMedline, CINAHL, PsycInfo, and Embase until June 2019.Eligibility criteria for study selectionStudies describing the development or external validation of an early warning score for adult hospital inpatients.Results13 171 references were screened and 95 articles were included in the review. 11 studies were development only, 23 were development and external validation, and 61 were external validation only. Most early warning scores were developed for use in the United States (n=13/34, 38%) and the United Kingdom (n=10/34, 29%). Death was the most frequent prediction outcome for development studies (n=10/23, 44%) and validation studies (n=66/84, 79%), with different time horizons (the most frequent was 24 hours). The most common predictors were respiratory rate (n=30/34, 88%), heart rate (n=28/34, 83%), oxygen saturation, temperature, and systolic blood pressure (all n=24/34, 71%). Age (n=13/34, 38%) and sex (n=3/34, 9%) were less frequently included. Key details of the analysis populations were often not reported in development studies (n=12/29, 41%) or validation studies (n=33/84, 39%). Small sample sizes and insufficient numbers of event patients were common in model development and external validation studies. Missing data were often discarded, with just one study using multiple imputation. Only nine of the early warning scores that were developed were presented in sufficient detail to allow individualised risk prediction. Internal validation was carried out in 19 studies, but recommended approaches such as bootstrapping or cross validation were rarely used (n=4/19, 22%). Model performance was frequently assessed using discrimination (development n=18/22, 82%; validation n=69/84, 82%), while calibration was seldom assessed (validation n=13/84, 15%). All included studies were rated at high risk of bias.ConclusionsEarly warning scores are widely used prediction models that are often mandated in daily clinical practice to identify early clinical deterioration in hospital patients. However, many early warning scores in clinical use were found to have methodological weaknesses. Early warning scores might not perform as well as expected and therefore they could have a detrimental effect on patient care. Future work should focus on following recommended approaches for developing and evaluating early warning scores, and investigating the impact and safety of using these scores in clinical practice.Systematic review registrationPROSPERO CRD42017053324.

Dataset Information

Development, Implementation, and Evaluation of an In-Hospital Optimized Early Warning Score for Patient Deterioration.

Publications

Development, Implementation, and Evaluation of an In-Hospital Optimized Early Warning Score for Patient Deterioration.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets