ABSTRACT: Abstract The objective of this study was to directly compare the ability of commonly used early warning scores (EWS) for early identification and prediction of sepsis in the general ward setting. For general ward patients at a large, academic medical center between early-2012 and mid-2018, common EWS and patient acuity scoring systems were calculated from electronic health records (EHR) data for patients that both met and did not meet Sepsis-3 criteria. For identification of sepsis at index time, National Early Warning Score 2 (NEWS 2) had the highest performance (area under the receiver operating characteristic curve: 0.803 [95% confidence interval [CI]: 0.795–0.811], area under the precision recall curves: 0.130 [95% CI: 0.121–0.140]) followed NEWS, Modified Early Warning Score, and quick Sequential Organ Failure Assessment (qSOFA). Using validated thresholds, NEWS 2 also had the highest recall (0.758 [95% CI: 0.736–0.778]) but qSOFA had the highest specificity (0.950 [95% CI: 0.948–0.952]), positive predictive value (0.184 [95% CI: 0.169–0.198]), and F1 score (0.236 [95% CI: 0.220–0.253]). While NEWS 2 outperformed all other compared EWS and patient acuity scores, due to the low prevalence of sepsis, all scoring systems were prone to false positives (low positive predictive value without drastic sacrifices in sensitivity), thus leaving room for more computationally advanced approaches. Lay Summary Sepsis is a syndrome caused by an infection resulting in organ dysfunction and high rates of death, is implicated in nearly half of all inpatient deaths, and is the costliest inpatient condition in the United States. Early recognition and treatment are critical to the management of septic patients. As a result, over time, researchers have developed numerous early warning scores that use clinical measurements such as vital signs and lab results to generate a value that is indicative of the severity of illness and is predictive of clinical deterioration. Increasingly, these scores have been used as screening tools for sepsis management. To understand the comparative performance of these early warning scores in the general ward setting, electronic health records data were used to calculate the scores. Of the compared scores, the National Early Warning Score (NEWS 2) outperformed the rest. However, partially due to the low prevalence of sepsis in the general ward, even NEWS 2 was prone to false positives, highlighting the potential for improvement using more advanced computational methods.
