Dataset Information

Validation of a Sequential Organ Failure Assessment Score using Electronic Health Record Data.

ABSTRACT: The sequential organ failure assessment (SOFA) score is a scoring system commonly used in critical care to assess severity of illness. Automated calculation of the SOFA score using existing electronic health record data would broaden its applicability. We performed a manual validation of an automated SOFA score previously developed at our institution. A retrospective analysis of a random subset of 300 patients from a previously published randomized trial of critically ill adults was performed, with manual validation of SOFA scores from the date of initial intensive care unit admission. Spearman's rank correlation coefficient, weighted Cohen's kappa, and Bland-Altman plots were used to assess agreement between manual and electronic versions of SOFA scores and between manual and electronic versions of their individual components. There was high agreement between manual and electronic SOFA scores (Spearman's rank correlation coefficient?=?0.90, 95% CI 0.87-0.93). Renal and respiratory components had lower agreement (weighted Cohen's kappa?=?0.63, 95% CI 0.53-0.73 for renal; weighted Cohen's kappa?=?0.77, 95% CI 0.70-0.84 for respiratory). The area under the receiver operating characteristic curve (AUC) for 30-day in-hospital mortality was 0.77 (95% CI 0.68-0.84) for manual SOFA scores and 0.75 (95% CI 0.66-0.83) for automated SOFA scores. Automatic calculation of SOFA scores from the electronic health record is feasible and correlates highly with manually calculated SOFA scores. Both have similar predictive value for 30-day in-hospital mortality.

SUBMITTER: Huerta LE

PROVIDER: S-EPMC6261278 | biostudies-literature | 2018 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validation of a Sequential Organ Failure Assessment Score using Electronic Health Record Data.

Huerta Luis E LE Wanderer Jonathan P JP Ehrenfeld Jesse M JM Freundlich Robert E RE Rice Todd W TW Semler Matthew W MW

Journal of medical systems 20180914 10

The sequential organ failure assessment (SOFA) score is a scoring system commonly used in critical care to assess severity of illness. Automated calculation of the SOFA score using existing electronic health record data would broaden its applicability. We performed a manual validation of an automated SOFA score previously developed at our institution. A retrospective analysis of a random subset of 300 patients from a previously published randomized trial of critically ill adults was performed, w ...[more]

PMID: 30218383

Similar Datasets

Project description:ObjectivesEarly identification of sepsis is critical to improving patient outcomes. Impact of the new sepsis definition (Sepsis-3) on timing of recognition in the emergency department has not been evaluated. Our study objective was to compare time to meeting systemic inflammatory response syndrome (Sepsis-2) criteria, Sequential Organ Failure Assessment (Sepsis-3) criteria, and quick Sequential Organ Failure Assessment criteria using electronic health record data.DesignRetrospective, observational study.SettingThe emergency department at the University of California, San Francisco.PatientsEmergency department encounters between June 2012 and December 2016 for patients greater than or equal to 18 years old with blood cultures ordered, IV antibiotic receipt, and identification with sepsis via systemic inflammatory response syndrome or Sequential Organ Failure Assessment within 72 hours of emergency department presentation.InterventionsNone.Measurements and main resultsWe analyzed timestamped electronic health record data from 16,612 encounters identified as sepsis by greater than or equal to 2 systemic inflammatory response syndrome criteria or a Sequential Organ Failure Assessment score greater than or equal to 2. The primary outcome was time from emergency department presentation to meeting greater than or equal to 2 systemic inflammatory response syndrome criteria, Sequential Organ Failure Assessment greater than or equal to 2, and/or greater than or equal to 2 quick Sequential Organ Failure Assessment criteria. There were 9,087 patients (54.7%) that met systemic inflammatory response syndrome-first a median of 26 minutes post-emergency department presentation (interquartile range, 0-109 min), with 83.1% meeting Sequential Organ Failure Assessment criteria a median of 118 minutes later (interquartile range, 44-401 min). There were 7,037 patients (42.3%) that met Sequential Organ Failure Assessment-first, a median of 113 minutes post-emergency department presentation (interquartile range, 60-251 min). Quick Sequential Organ Failure Assessment was met in 46.4% of patients a median of 351 minutes post-emergency department presentation (interquartile range, 67-1,165 min). Adjusted odds of in-hospital mortality were 39% greater in patients who met systemic inflammatory response syndrome-first compared with those who met Sequential Organ Failure Assessment-first (odds ratio, 1.39; 95% CI, 1.20-1.61).ConclusionsSystemic inflammatory response syndrome and Sequential Organ Failure Assessment initially identified distinct populations. Using systemic inflammatory response syndrome resulted in earlier electronic health record sepsis identification in greater than 50% of patients. Using Sequential Organ Failure Assessment alone may delay identification. Using systemic inflammatory response syndrome alone may lead to missed sepsis presenting as acute organ dysfunction. Thus, a combination of inflammatory (systemic inflammatory response syndrome) and organ dysfunction (Sequential Organ Failure Assessment) criteria may enhance timely electronic health record-based sepsis identification.

Project description:Prior research has hypothesized the Sequential Organ Failure Assessment (SOFA) score to be a poor predictor of mortality in mechanically ventilated patients with COVID-19. Yet, several U.S. states have proposed SOFA-based algorithms for ventilator triage during crisis standards of care. Using a large cohort of mechanically ventilated patients with COVID-19, we externally validated the predictive capacity of the preintubation SOFA score for mortality prediction with and without other commonly used algorithm elements. Multicenter, retrospective cohort study using electronic health record data. Eighty-six U.S. health systems. Patients with COVID-19 hospitalized between January 1, 2020, and February 14, 2021, and subsequently initiated on mechanical ventilation. None. Among 15,122 mechanically ventilated patients with COVID-19, SOFA score alone demonstrated poor discriminant accuracy for inhospital mortality in mechanically ventilated patients using the validation cohort (area under the receiver operating characteristic curve [AUC], 0.66; 95% CI, 0.65-0.67). Discriminant accuracy was even poorer using SOFA score categories (AUC, 0.54; 95% CI, 0.54-0.55). Age alone demonstrated greater discriminant accuracy for inhospital mortality than SOFA score (AUC, 0.71; 95% CI, 0.69-0.72). Discriminant accuracy for mortality improved upon addition of age to the continuous SOFA score (AUC, 0.74; 95% CI, 0.73-0.76) and categorized SOFA score (AUC, 0.72; 95% CI, 0.71-0.73) models, respectively. The addition of comorbidities did not substantially increase model discrimination. Of 36 U.S. states with crisis standards of care guidelines containing ventilator triage algorithms, 31 (86%) feature the SOFA score. Of these, 25 (81%) rely heavily on the SOFA score (12 exclusively propose SOFA; 13 place highest weight on SOFA or propose SOFA with one other variable). In a U.S. cohort of over 15,000 ventilated patients with COVID-19, the SOFA score displayed poor predictive accuracy for short-term mortality. Our findings warrant reappraisal of the SOFA score's implementation and weightage in existing ventilator triage pathways in current U.S. crisis standards of care guidelines.

Project description:The Sequential Organ Failure Assessment and other severity of illness scales rely on the Glasgow Coma Scale to measure acute neurologic dysfunction, but the Glasgow Coma Scale is unavailable or inconsistently applied in some institutions. The objective of this study was to assess the validity of a modified Sequential Organ Failure Assessment that uses the Richmond Agitation-Sedation Scale instead of Glasgow Coma Scale.Prospective cohort study.Medical and surgical ICUs within a large, tertiary care hospital.Critically ill medical/surgical ICU patients.We calculated daily Sequential Organ Failure Assessment scores by using electronic medical record-derived data. By using bedside nurse-recorded Glasgow Coma Scale and Richmond Agitation-Sedation Scale measures, we calculated neurologic Sequential Organ Failure Assessment scores using the original Glasgow Coma Scale-based approach and a novel Richmond Agitation-Sedation Scale-based approach, converting the 10-point Richmond Agitation-Sedation Scale to a 4-point neurologic Sequential Organ Failure Assessment score. We assessed construct validity of Richmond Agitation-Sedation Scale-based Sequential Organ Failure Assessment by analyzing correlations with established severity of illness constructs (Acute Physiology and Chronic Health Evaluation II and Glasgow Coma Scale-based Sequential Organ Failure Assessment) and predictive validity by using logistic regression to determine whether Richmond Agitation-Sedation Scale-based Sequential Organ Failure Assessment predicts ICU, hospital, and 1-year mortality. We assessed discriminative performance with c-statistics.Among 513 patients (5,199 patient-days), Richmond Agitation-Sedation Scale-based Sequential Organ Failure Assessment was strongly correlated with Acute Physiology and Chronic Health Evaluation II acute physiology score at enrollment (r = 0.583; 95% CI, 0.518-0.642) and daily Glasgow Coma Scale-based Sequential Organ Failure Assessment scores (r = 0.963; 95% CI, 0.956-0.968). Mean Richmond Agitation-Sedation Scale-based Sequential Organ Failure Assessment scores predicted ICU mortality (areas under the curve = 0.814)-as did mean Glasgow Coma Scale-based Sequential Organ Failure Assessment (0.799)-as well as hospital and 1-year mortality. Admission Sequential Organ Failure Assessment scores, whether using Richmond Agitation-Sedation Scale or Glasgow Coma Scale, were less accurate predictors of mortality; areas under the curves for ICU mortality for Richmond Agitation-Sedation Scale-based and Glasgow Coma Scale-based Sequential Organ Failure Assessment, for example, were 0.622 and 0.608, respectively.A modified Sequential Organ Failure Assessment score that uses bedside Richmond Agitation-Sedation Scale when Glasgow Coma Scale data are not available is a valid means of assessing daily severity of illness in the ICU and may be valuable for risk-adjustment and benchmarking purposes.

Project description:ImportanceThe Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) uses the Sequential Organ Failure Assessment (SOFA) score to grade organ dysfunction in adult patients with suspected infection. However, the SOFA score is not adjusted for age and therefore not suitable for children.ObjectivesTo adapt and validate a pediatric version of the SOFA score (pSOFA) in critically ill children and to evaluate the Sepsis-3 definitions in patients with confirmed or suspected infection.Design, setting, and participantsThis retrospective observational cohort study included all critically ill children 21 years or younger admitted to a 20-bed, multidisciplinary, tertiary pediatric intensive care unit between January 1, 2009 and August 1, 2016. Data on these children were obtained from an electronic health record database. The pSOFA score was developed by adapting the original SOFA score with age-adjusted cutoffs for the cardiovascular and renal systems and by expanding the respiratory criteria to include noninvasive surrogates of lung injury. Daily pSOFA scores were calculated from admission until day 28 of hospitalization, discharge, or death (whichever came first). Three additional pediatric organ dysfunction scores were calculated for comparison.ExposuresOrgan dysfunction measured by the pSOFA score, and sepsis and septic shock according to the Sepsis-3 definitions.Main outcomes and measuresThe primary outcome was in-hospital mortality. The daily pSOFA scores and additional pediatric organ dysfunction scores were compared. Performance was evaluated using the area under the curve. The pSOFA score was then used to assess the Sepsis-3 definitions in the subgroup of children with confirmed or suspected infection.ResultsIn all, 6303 patients with 8711 encounters met inclusion criteria. Each encounter was treated independently. Of the 8482 survivors of hospital encounters, 4644 (54.7%) were male and the median (interquartile range [IQR]) age was 69 (17-156) months. Among the 229 nonsurvivors, 127 (55.4%) were male with a median (IQR) age of 43 (8-144) months. In-hospital mortality was 2.6%. The maximum pSOFA score had excellent discrimination for in-hospital mortality, with an area under the curve of 0.94 (95% CI, 0.92-0.95). The pSOFA score had a similar or better performance than other pediatric organ dysfunction scores. According to the Sepsis-3 definitions, 1231 patients (14.1%) were classified as having sepsis and had a mortality rate of 12.1%, and 347 (4.0%) had septic shock and a mortality rate of 32.3%. Patients with sepsis were more likely to die than patients with confirmed or suspected infection but no sepsis (odds ratio, 18; 95% CI, 11-28). Of the 229 patients who died during their hospitalization, 149 (65.0%) had sepsis or septic shock during their course.Conclusions and relevanceThe pSOFA score was adapted and validated with age-adjusted variables in critically ill children. Using the pSOFA score, the Sepsis-3 definitions were assessed in children with confirmed or suspected infection. This study is the first assessment, to date, of the Sepsis-3 definitions in critically ill children. Use of these definitions in children is feasible and shows promising results.

Dataset Information

Validation of a Sequential Organ Failure Assessment Score using Electronic Health Record Data.

Publications

Validation of a Sequential Organ Failure Assessment Score using Electronic Health Record Data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets