Dataset Information

Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU.

ABSTRACT: OBJECTIVES:We validate a machine learning-based sepsis-prediction algorithm (InSight) for the detection and prediction of three sepsis-related gold standards, using only six vital signs. We evaluate robustness to missing data, customisation to site-specific data using transfer learning and generalisability to new settings. DESIGN:A machine-learning algorithm with gradient tree boosting. Features for prediction were created from combinations of six vital sign measurements and their changes over time. SETTING:A mixed-ward retrospective dataset from the University of California, San Francisco (UCSF) Medical Center (San Francisco, California, USA) as the primary source, an intensive care unit dataset from the Beth Israel Deaconess Medical Center (Boston, Massachusetts, USA) as a transfer-learning source and four additional institutions' datasets to evaluate generalisability. PARTICIPANTS:684?443 total encounters, with 90?353 encounters from June 2011 to March 2016 at UCSF. INTERVENTIONS:None. PRIMARY AND SECONDARY OUTCOME MEASURES:Area under the receiver operating characteristic (AUROC) curve for detection and prediction of sepsis, severe sepsis and septic shock. RESULTS:For detection of sepsis and severe sepsis, InSight achieves an AUROC curve of 0.92 (95% CI 0.90 to 0.93) and 0.87 (95% CI 0.86 to 0.88), respectively. Four hours before onset, InSight predicts septic shock with an AUROC of 0.96 (95% CI 0.94 to 0.98) and severe sepsis with an AUROC of 0.85 (95% CI 0.79 to 0.91). CONCLUSIONS:InSight outperforms existing sepsis scoring systems in identifying and predicting sepsis, severe sepsis and septic shock. This is the first sepsis screening system to exceed an AUROC of 0.90 using only vital sign inputs. InSight is robust to missing data, can be customised to novel hospital data using a small fraction of site data and retains strong discrimination across all institutions.

SUBMITTER: Mao Q

PROVIDER: S-EPMC5829820 | biostudies-other | 2018 Jan

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU.

Mao Qingqing Q Jay Melissa M Hoffman Jana L JL Calvert Jacob J Barton Christopher C Shimabukuro David D Shieh Lisa L Chettipally Uli U Fletcher Grant G Kerem Yaniv Y Zhou Yifan Y Das Ritankar R

BMJ open 20180126 1

<h4>Objectives</h4>We validate a machine learning-based sepsis-prediction algorithm (<i>InSight</i>) for the detection and prediction of three sepsis-related gold standards, using only six vital signs. We evaluate robustness to missing data, customisation to site-specific data using transfer learning and generalisability to new settings.<h4>Design</h4>A machine-learning algorithm with gradient tree boosting. Features for prediction were created from combinations of six vital sign measurements an ...[more]

PMID: 29374661

Similar Datasets

Project description:Accurate stratification of sepsis can effectively guide the triage of patient care and shared decision making in the emergency department (ED). However, previous research on sepsis identification models focused mainly on ICU patients, and discrepancies in model performance between the development and external validation datasets are rarely evaluated. The aim of our study was to develop and externally validate a machine learning model to stratify sepsis patients in the ED. We retrospectively collected clinical data from two geographically separate institutes that provided a different level of care at different time periods. The Sepsis-3 criteria were used as the reference standard in both datasets for identifying true sepsis cases. An eXtreme Gradient Boosting (XGBoost) algorithm was developed to stratify sepsis patients and the performance of the model was compared with traditional clinical sepsis tools; quick Sequential Organ Failure Assessment (qSOFA) and Systemic Inflammatory Response Syndrome (SIRS). There were 8296 patients (1752 (21%) being septic) in the development and 1744 patients (506 (29%) being septic) in the external validation datasets. The mortality of septic patients in the development and validation datasets was 13.5% and 17%, respectively. In the internal validation, XGBoost achieved an area under the receiver operating characteristic curve (AUROC) of 0.86, exceeding SIRS (0.68) and qSOFA (0.56). The performance of XGBoost deteriorated in the external validation (the AUROC of XGBoost, SIRS and qSOFA was 0.75, 0.57 and 0.66, respectively). Heterogeneity in patient characteristics, such as sepsis prevalence, severity, age, comorbidity and infection focus, could reduce model performance. Our model showed good discriminative capabilities for the identification of sepsis patients and outperformed the existing sepsis identification tools. Implementation of the ML model in the ED can facilitate timely sepsis identification and treatment. However, dataset discrepancies should be carefully evaluated before implementing the ML approach in clinical practice. This finding reinforces the necessity for future studies to perform external validation to ensure the generalisability of any developed ML approaches.

Project description:Study objectiveRecognition of pediatric sepsis is a key clinical challenge. We evaluate the performance of a sepsis recognition process including an electronic sepsis alert and bedside assessment in a pediatric emergency department (ED).MethodsThis was a cohort study with quality improvement intervention in a pediatric ED. Exposure was a positive electronic sepsis alert, defined as elevated pulse rate or hypotension, concern for infection, and at least one of the following: abnormal capillary refill, abnormal mental status, or high-risk condition. A positive electronic sepsis alert prompted team assessment or huddle to determine need for sepsis protocol. Clinicians could initiate team assessment or huddle according to clinical concern without positive electronic sepsis alert. Severe sepsis outcome defined as activation of the sepsis protocol in the ED or development of severe sepsis requiring ICU admission within 24 hours.ResultsThere were 182,509 ED visits during the study period, with 86,037 before electronic sepsis alert implementation and 96,472 afterward, and 1,112 (1.2%) positive electronic sepsis alerts. Overall, 326 patients (0.3%) were treated for severe sepsis within 24 hours. Test characteristics of the electronic sepsis alert alone to detect severe sepsis were sensitivity 86.2% (95% confidence interval [CI] 82.0% to 89.5%), specificity 99.1% (95% CI 99.0% to 99.2%), positive predictive value 25.4% (95% CI 22.8% to 28.0%), and negative predictive value 100% (95% CI 99.9% to 100%). Inclusion of the clinician screen identified 43 additional electronic sepsis alert-negative children, with severe sepsis sensitivity 99.4% (95% CI 97.8% to 99.8%) and specificity 99.1% (95% CI 99.1% to 99.2%). Electronic sepsis alert implementation increased ED sepsis detection from 83% to 96%.ConclusionElectronic sepsis alert for severe sepsis demonstrated good sensitivity and high specificity. Addition of clinician identification of electronic sepsis alert-negative patients further improved sensitivity. Implementation of the electronic sepsis alert was associated with improved recognition of severe sepsis.

Project description:PURPOSE:Sepsis is a common acute life-threatening condition that emergency physicians routinely face. Diagnostic options within the Emergency Department (ED) are limited due to lack of infrastructure, consequently limiting the use of invasive hemodynamic monitoring or imaging tests. The mortality rate due to sepsis can be assessed via multiple scoring systems, for example, mortality in emergency department sepsis (MEDS) score and sepsis patient evaluation in the emergency department (SPEED) score, both of which quantify the variation of mortality rates according to clinical findings, laboratory data, or therapeutic interventions. This study aims to improve the management processes of sepsis patients by comparing SPEED score and MEDS score for predicting the 28-day mortality in cases of emergency sepsis. METHODS:The study is a cross-sectional, prospective study including 61 sepsis patients in ED in Suez Canal University Hospital, Egypt, from August 2017 to June 2018. Patients were selected by two steps: (1) suspected septic patients presenting with at least one of the following abnormal clinical findings: (a) body temperature higher than 38 °C or lower than 36 °C, (b) heart rate higher than 90 beats/min, (c) hyperventilation evidenced by respiratory rate higher than 20 breaths/min or PaCO2 lower than 32 mmHg, and (d) white blood cell count higher than 12,000/?L or lower than 4000/?L; (2) confirmed septic patients with at least a 2-point increase from the baseline total sequential organ failure assessment (SOFA) score following infection. Other inclusion criteria included adult patients with an age ?18 years regardless of gender and those who had either systemic inflammatory response syndrome or suspected/confirmed infection. Patients were shortly follow-up for the 28-day mortality. Each patient was subject to SPEED score and MEDS score and then the results were compared to detect which of them was more effective in predicting outcome. The receiver operating characteristic curves were also done for MEDS and SPEED scores. RESULTS:Among the 61 patients, 41 died with the mortality rate of 67.2%. The mortality rate increased with a higher SPEED and MEDS scores. Both SPEED and MEDS scores revealed significant difference between the survivors and nonsurvivors (p = 0.004 and p < 0.001, respectively), indicating that both the two systems are effective in predicting the 28-day mortality of sepsis patients. Thereafter, the receiver operating characteristic curves were plotted, which showed that SPEED was better than the MEDS score when applied to the complete study population with an area under the curve being 0.87 (0.788-0.963) as compared with 0.75 (0.634-0.876) for MEDS. Logistic regression analysis revealed that the best fitting predictor of 28-day mortality for sepsis patients was the SPEED scoring system. For every one unit increase in SPEED score, the odds of 28-day mortality increased by 37%. CONCLUSION:SPEED score is more useful and accurate than MEDS score in predicting the 28-day mortality among sepsis patients. Therefore SPEED rather than MEDS should be more widely used in the ED for sepsis patients.

Project description:The Surviving Sepsis Campaign Guidelines 2021 recommends that adult patients with sepsis requiring intensive care should be admitted to the ICU within 6 hours of their emergency department (ED) visits. However, there is limited evidence on whether 6 hours is the best target time for compliance with the sepsis bundle. We aimed to investigate the association between time from ED visits to ICU admission (i.e., ED length of stay [ED-LOS]) and mortality and identify the optimal ED-LOS for patients with sepsis.DesignRetrospective cohort study.SettingThe Medical Information Mart for Intensive Care Emergency Department and Medical Information Mart for Intensive Care IV databases.PatientsAdult patients (≥ 18 yr old) who were transferred from the ED to the ICU and subsequently diagnosed with sepsis based on the Sepsis-3 criteria within 24 hours of ICU admission.InterventionsNone.Measurements and main resultsAmong 1,849 patients with sepsis, we found a disproportionally higher mortality rate in patients immediately admitted to the ICU (e.g., < 2 hr). When using ED-LOS as a continuous variable, ED-LOS was not significantly associated with 28-day mortality (adjusted odds ratio [OR] per hour increase, 1.04; 95% CI, 0.96-1.13; p = 0.3) after an adjustment for potential confounders (e.g., demographics, triage vital signs, and laboratory results) in the multivariable analysis. However, when we categorized all patients into time quartiles (ED-LOS: < 3.3 hr, 3.3-4.5 hr, 4.6-6.1 hr, and > 6.1 hr), patients in the higher time quartiles (e.g., 3.3-4.5 hr) had higher 28-day mortality compared with those in the lowest time quartile (< 3.3 hr) (e.g., adjusted OR for patients in the second time quartile [3.3-4.5 hr] 1.59; 95% CI, 1.03-2.46; p = 0.04).ConclusionsEarlier admission to the ICU (e.g., within 3.3 hr of ED visits) was associated with lower 28-day mortality in patients with sepsis. Our findings suggest patients with sepsis who require intensive care may benefit from a more immediate ICU admission than 6 hours.

Dataset Information

Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU.

Publications

Multicentre validation of a sepsis prediction algorithm using only vital sign data in the emergency department, general ward and ICU.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets