Dataset Information

Ground truth labels challenge the validity of sepsis consensus definitions in critical illness.

ABSTRACT:

Background

Sepsis is the leading cause of death in the intensive care unit (ICU). Expediting its diagnosis, largely determined by clinical assessment, improves survival. Predictive and explanatory modelling of sepsis in the critically ill commonly bases both outcome definition and predictions on clinical criteria for consensus definitions of sepsis, leading to circularity. As a remedy, we collected ground truth labels for sepsis.

Methods

In the Ground Truth for Sepsis Questionnaire (GTSQ), senior attending physicians in the ICU documented daily their opinion on each patient's condition regarding sepsis as a five-category working diagnosis and nine related items. Working diagnosis groups were described and compared and their SOFA-scores analyzed with a generalized linear mixed model. Agreement and discriminatory performance measures for clinical criteria of sepsis and GTSQ labels as reference class were derived.

Results

We analyzed 7291 questionnaires and 761 complete encounters from the first survey year. Editing rates for all items were > 90%, and responses were consistent with current understanding of critical illness pathophysiology, including sepsis pathogenesis. Interrater agreement for presence and absence of sepsis was almost perfect but only slight for suspected infection. ICU mortality was 19.5% in encounters with SIRS as the "worst" working diagnosis compared to 5.9% with sepsis and 5.9% with severe sepsis without differences in admission and maximum SOFA. Compared to sepsis, proportions of GTSQs with SIRS plus acute organ dysfunction were equal and macrocirculatory abnormalities higher (p < 0.0001). SIRS proportionally ranked above sepsis in daily assessment of illness severity (p < 0.0001). Separate analyses of neurosurgical referrals revealed similar differences. Discriminatory performance of Sepsis-1/2 and Sepsis-3 compared to GTSQ labels was similar with sensitivities around 70% and specificities 92%. Essentially no difference between the prevalence of SIRS and SOFA ≥ 2 yielded sensitivities and specificities for detecting sepsis onset close to 55% and 83%, respectively.

Conclusions

GTSQ labels are a valid measure of sepsis in the ICU. They reveal suspicion of infection as an unclear clinical concept and refute an illness severity hierarchy in the SIRS-sepsis-severe sepsis spectrum. Ground truth challenges the accuracy of Sepsis-1/2 and Sepsis-3 in detecting sepsis onset. It is an indispensable intermediate step towards advancing diagnosis and therapy in the ICU and, potentially, other health care settings.

SUBMITTER: Lindner HA

PROVIDER: S-EPMC8760797 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IMPORTANCE:The Third International Consensus Definitions Task Force defined sepsis as "life-threatening organ dysfunction due to a dysregulated host response to infection." The performance of clinical criteria for this sepsis definition is unknown. OBJECTIVE:To evaluate the validity of clinical criteria to identify patients with suspected infection who are at risk of sepsis. DESIGN, SETTINGS, AND POPULATION:Among 1.3 million electronic health record encounters from January 1, 2010, to December 31, 2012, at 12 hospitals in southwestern Pennsylvania, we identified those with suspected infection in whom to compare criteria. Confirmatory analyses were performed in 4 data sets of 706,399 out-of-hospital and hospital encounters at 165 US and non-US hospitals ranging from January 1, 2008, until December 31, 2013. EXPOSURES:Sequential [Sepsis-related] Organ Failure Assessment (SOFA) score, systemic inflammatory response syndrome (SIRS) criteria, Logistic Organ Dysfunction System (LODS) score, and a new model derived using multivariable logistic regression in a split sample, the quick Sequential [Sepsis-related] Organ Failure Assessment (qSOFA) score (range, 0-3 points, with 1 point each for systolic hypotension [?100 mm Hg], tachypnea [?22/min], or altered mentation). MAIN OUTCOMES AND MEASURES:For construct validity, pairwise agreement was assessed. For predictive validity, the discrimination for outcomes (primary: in-hospital mortality; secondary: in-hospital mortality or intensive care unit [ICU] length of stay ?3 days) more common in sepsis than uncomplicated infection was determined. Results were expressed as the fold change in outcome over deciles of baseline risk of death and area under the receiver operating characteristic curve (AUROC). RESULTS:In the primary cohort, 148,907 encounters had suspected infection (n?=?74,453 derivation; n?=?74,454 validation), of whom 6347 (4%) died. Among ICU encounters in the validation cohort (n?=?7932 with suspected infection, of whom 1289 [16%] died), the predictive validity for in-hospital mortality was lower for SIRS (AUROC?=?0.64; 95% CI, 0.62-0.66) and qSOFA (AUROC?=?0.66; 95% CI, 0.64-0.68) vs SOFA (AUROC?=?0.74; 95% CI, 0.73-0.76; P?<?.001 for both) or LODS (AUROC?=?0.75; 95% CI, 0.73-0.76; P?<?.001 for both). Among non-ICU encounters in the validation cohort (n?=?66?522 with suspected infection, of whom 1886 [3%] died), qSOFA had predictive validity (AUROC?=?0.81; 95% CI, 0.80-0.82) that was greater than SOFA (AUROC?=?0.79; 95% CI, 0.78-0.80; P?<?.001) and SIRS (AUROC?=?0.76; 95% CI, 0.75-0.77; P?<?.001). Relative to qSOFA scores lower than 2, encounters with qSOFA scores of 2 or higher had a 3- to 14-fold increase in hospital mortality across baseline risk deciles. Findings were similar in external data sets and for the secondary outcome. CONCLUSIONS AND RELEVANCE:Among ICU encounters with suspected infection, the predictive validity for in-hospital mortality of SOFA was not significantly different than the more complex LODS but was statistically greater than SIRS and qSOFA, supporting its use in clinical criteria for sepsis. Among encounters with suspected infection outside of the ICU, the predictive validity for in-hospital mortality of qSOFA was statistically greater than SOFA and SIRS, supporting its use as a prompt to consider possible sepsis.

Project description:The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) present clinical criteria for the classification of patients with sepsis. We investigated incidence and long-term outcomes of patients diagnosed with these classifications, which are currently unknown.We did a retrospective analysis using data from 30?239 participants from the USA who were aged at least 45 years and enrolled in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) cohort. Patients were enrolled between Jan 25, 2003, and Oct 30, 2007, and we identified hospital admissions from Feb 5, 2003, to Dec 31, 2012, and applied three classifications: infection and systemic inflammatory response syndrome (SIRS) criteria, elevated sepsis-related organ failure assessment (SOFA) score from Sepsis-3, and elevated quick SOFA (qSOFA) score from Sepsis-3. We estimated incidence during the study period, in-hospital mortality, and 1-year mortality.Of 2593 first infection events, 1526 met SIRS criteria, 1080 met SOFA criteria, and 378 met qSOFA criteria. Incidence was 8·2 events (95% CI 7·8-8·7) per 1000 person-years for SIRS, 5·8 events (5·4-6·1) per 1000 person-years for SOFA, and 2·0 events (1·8-2·2) per 1000 person-years for qSOFA. In-hospital mortality was higher for patients with an elevated qSOFA score (67 [23%] of 295 patients died) than for those with an elevated SOFA score (125 [13%] of 960 patients died) or who met SIRS criteria (128 [9%] of 1392 patients died). Mortality at 1 year after discharge was also highest for patients with an elevated qSOFA score (29·4 deaths [95% CI 22·3-38·7] per 100 person-years) compared with those with an elevated SOFA score (22·6 deaths [19·2-26·6] per 100 person-years) or those who met SIRS criteria (14·7 deaths [12·5-17·2] per 100 person-years).SIRS, SOFA, and qSOFA classifications identified different incidences and mortality. Our findings support the use of the SOFA and qSOFA classifications to identify patients with infection who are at elevated risk of poor outcomes. These classifications could be used in future epidemiological assessments and studies of patients with infection.National Institute for Nursing Research, National Center for Research Resources, and National Institute of Neurological Disorders and Stroke.