Dataset Information

Comparison of early warning scores for sepsis early identification and prediction in the general ward setting.

ABSTRACT: The objective of this study was to directly compare the ability of commonly used early warning scores (EWS) for early identification and prediction of sepsis in the general ward setting. For general ward patients at a large, academic medical center between early-2012 and mid-2018, common EWS and patient acuity scoring systems were calculated from electronic health records (EHR) data for patients that both met and did not meet Sepsis-3 criteria. For identification of sepsis at index time, National Early Warning Score 2 (NEWS 2) had the highest performance (area under the receiver operating characteristic curve: 0.803 [95% confidence interval [CI]: 0.795-0.811], area under the precision recall curves: 0.130 [95% CI: 0.121-0.140]) followed NEWS, Modified Early Warning Score, and quick Sequential Organ Failure Assessment (qSOFA). Using validated thresholds, NEWS 2 also had the highest recall (0.758 [95% CI: 0.736-0.778]) but qSOFA had the highest specificity (0.950 [95% CI: 0.948-0.952]), positive predictive value (0.184 [95% CI: 0.169-0.198]), and F1 score (0.236 [95% CI: 0.220-0.253]). While NEWS 2 outperformed all other compared EWS and patient acuity scores, due to the low prevalence of sepsis, all scoring systems were prone to false positives (low positive predictive value without drastic sacrifices in sensitivity), thus leaving room for more computationally advanced approaches.

SUBMITTER: Yu SC

PROVIDER: S-EPMC8607822 | biostudies-literature | 2021 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparison of early warning scores for sepsis early identification and prediction in the general ward setting.

Yu Sean C SC Shivakumar Nirmala N Betthauser Kevin K Gupta Aditi A Lai Albert M AM Kollef Marin H MH Payne Philip R O PRO Michelson Andrew P AP

JAMIA open 20210701 3

The objective of this study was to directly compare the ability of commonly used early warning scores (EWS) for early identification and prediction of sepsis in the general ward setting. For general ward patients at a large, academic medical center between early-2012 and mid-2018, common EWS and patient acuity scoring systems were calculated from electronic health records (EHR) data for patients that both met and did not meet Sepsis-3 criteria. For identification of sepsis at index time, Nationa ...[more]

PMID: 34820600

Similar Datasets

Project description:BackgroundEarly warning scores are designed to identify hospitalized patients who are at high risk of clinical deterioration. Although many general scores have been developed for the medical-surgical wards, specific scores have also been developed for obstetric patients due to differences in normal vital sign ranges and potential complications in this unique population. The comparative performance of general and obstetric early warning scores for predicting deterioration and infection on the maternal wards is not known.MethodsThis was an observational cohort study at the University of Chicago that included patients hospitalized on obstetric wards from November 2008 to December 2018. Obstetric scores (modified early obstetric warning system (MEOWS), maternal early warning criteria (MEWC), and maternal early warning trigger (MEWT)), paper-based general scores (Modified Early Warning Score (MEWS) and National Early Warning Score (NEWS), and a general score developed using machine learning (electronic Cardiac Arrest Risk Triage (eCART) score) were compared using the area under the receiver operating characteristic score (AUC) for predicting ward to intensive care unit (ICU) transfer and/or death and new infection.ResultsA total of 19,611 patients were included, with 43 (0.2%) experiencing deterioration (ICU transfer and/or death) and 88 (0.4%) experiencing an infection. eCART had the highest discrimination for deterioration (p < 0.05 for all comparisons), with an AUC of 0.86, followed by MEOWS (0.74), NEWS (0.72), MEWC (0.71), MEWS (0.70), and MEWT (0.65). MEWC, MEWT, and MEOWS had higher accuracy than MEWS and NEWS but lower accuracy than eCART at specific cut-off thresholds. For predicting infection, eCART (AUC 0.77) had the highest discrimination.ConclusionsWithin the limitations of our retrospective study, eCART had the highest accuracy for predicting deterioration and infection in our ante- and postpartum patient population. Maternal early warning scores were more accurate than MEWS and NEWS. While institutional choice of an early warning system is complex, our results have important implications for the risk stratification of maternal ward patients, especially since the low prevalence of events means that small improvements in accuracy can lead to large decreases in false alarms.

Project description:ImportanceEarly warning decision support tools to identify clinical deterioration in the hospital are widely used, but there is little information on their comparative performance.ObjectiveTo compare 3 proprietary artificial intelligence (AI) early warning scores and 3 publicly available simple aggregated weighted scores.Design, setting, and participantsThis retrospective cohort study was performed at 7 hospitals in the Yale New Haven Health System. All consecutive adult medical-surgical ward hospital encounters between March 9, 2019, and November 9, 2023, were included.ExposuresSimultaneous Epic Deterioration Index (EDI), Rothman Index (RI), eCARTv5 (eCART), Modified Early Warning Score (MEWS), National Early Warning Score (NEWS), and NEWS2 scores.Main outcomes and measuresClinical deterioration, defined as a transfer from ward to intensive care unit or death within 24 hours of an observation.ResultsOf the 362 926 patient encounters (median patient age, 64 [IQR, 47-77] years; 200 642 [55.3%] female), 16 693 (4.6%) experienced a clinical deterioration event. eCART had the highest area under the receiver operating characteristic curve at 0.895 (95% CI, 0.891-0.900), followed by NEWS2 at 0.831 (95% CI, 0.826-0.836), NEWS at 0.829 (95% CI, 0.824-0.835), RI at 0.828 (95% CI, 0.823-0.834), EDI at 0.808 (95% CI, 0.802-0.812), and MEWS at 0.757 (95% CI, 0.750-0.764). After matching scores at the moderate-risk sensitivity level for a NEWS score of 5, overall positive predictive values (PPVs) ranged from a low of 6.3% (95% CI, 6.1%-6.4%) for an EDI score of 41 to a high of 17.3% (95% CI, 16.9%-17.8%) for an eCART score of 94. Matching scores at the high-risk specificity of a NEWS score of 7 yielded overall PPVs ranging from a low of 14.5% (95% CI, 14.0%-15.2%) for an EDI score of 54 to a high of 23.3% (95% CI, 22.7%-24.2%) for an eCART score of 97. The moderate-risk thresholds provided a median of at least 20 hours of lead time for all the scores. Median lead time at the high-risk threshold was 11 (IQR, 0-69) hours for eCART, 8 (IQR, 0-63) hours for NEWS, 6 (IQR, 0-62) hours for NEWS2, 5 (IQR, 0-56) hours for MEWS, 1 (IQR, 0-39) hour for EDI, and 0 (IQR, 0-42) hours for RI.Conclusions and relevanceIn this cohort study of inpatient encounters, eCART outperformed the other AI and non-AI scores, identifying more deteriorating patients with fewer false alarms and sufficient time to intervene. NEWS, a non-AI, publicly available early warning score, significantly outperformed EDI. Given the wide variation in accuracy, additional transparency and oversight of early warning tools may be warranted.

Project description:IntroductionProviding optimal critical care in developing countries is limited by lack of recognition of critical illness and lack of essential resources. The Modified Early Warning Score (MEWS), based on physiological parameters, is validated in adult medical and surgical patients as a predictor of mortality. The objective of this study performed in Uganda was to determine the prevalence of critical illness on the wards as defined by the MEWS, to evaluate the MEWS as a predictor of death, and to describe additional risk factors for mortality.MethodsWe conducted a prospective observational study at Mulago National Referral Teaching Hospital in Uganda. We included medical and surgical ward patients over 18 years old, excluding patients discharged the day of enrolment, obstetrical patients, and patients who self-discharged prior to study completion. Over a 72-hour study period, we collected demographic and vital signs, and calculated MEWS; at 7-days we measured outcomes. Patients discharged prior to 7 days were assumed to be alive at study completion. Descriptive and inferential statistical analyses were performed.ResultsOf 452 patients, the median age was 40.5 (IQR 29-54) years, 53.3% were male, 24.3% were HIV positive, and 45.1% had medical diagnoses. MEWS ranged from 0 to 9, with higher scores representing hemodynamic instability. The median MEWS was 2 [IQR 1-3] and the median length of hospital stay was 9 days [IQR 4-24]. In-hospital mortality at 7-days was 5.5%; 41.4% of patients were discharged and 53.1% remained on the ward. Mortality was independently associated with medical admission (OR: 7.17; 95% CI: 2.064-24.930; p = 0.002) and the MEWS ≥ 5 (OR: 5.82; 95% CI: 2.420-13.987; p<0.0001) in the multivariable analysis.ConclusionThere is a significant burden of critical illness at Mulago Hospital, Uganda. Implementation of the MEWS could provide a useful triage tool to identify patients at greatest risk of death. Future research should include refinement of MEWS for low-resource settings, and development of appropriate interventions for patients identified to be at high risk of death based on early warning scores.

Dataset Information

Comparison of early warning scores for sepsis early identification and prediction in the general ward setting.

Publications

Comparison of early warning scores for sepsis early identification and prediction in the general ward setting.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets