Dataset Information

Multicenter derivation and validation of an early warning score for acute respiratory failure or death in the hospital.

ABSTRACT: BACKGROUND:Acute respiratory failure occurs frequently in hospitalized patients and often starts before ICU admission. A risk stratification tool to predict mortality and risk for mechanical ventilation (MV) may allow for earlier evaluation and intervention. We developed and validated an automated electronic health record (EHR)-based model-Accurate Prediction of Prolonged Ventilation (APPROVE)-to identify patients at risk of death or respiratory failure requiring >=?48 h of MV. METHODS:This was an observational study of adults admitted to four hospitals in 2013 or a fifth hospital in 2017. Clinical data were extracted from the EHRs. The 2013 patients were randomly split 50:50 into a derivation/validation cohort. The qualifying event was death or intubation leading to MV >=?48 h. Random forest method was used in model derivation. APPROVE was calculated retrospectively whenever data were available in 2013, and prospectively every 4 h after hospital admission in 2017. The Modified Early Warning Score (MEWS) and National Early Warning Score (NEWS) were calculated at the same times as APPROVE. Clinicians were not alerted except for APPROVE in 2017cohort. RESULTS:There were 68,775 admissions in 2013 and 2258 in 2017. APPROVE had an area under the receiver operator curve of 0.87 (95% CI 0.85-0.88) in 2013 and 0.90 (95% CI 0.84-0.95) in 2017, which is significantly better than the MEWS and NEWS in 2013 but similar to the MEWS and NEWS in 2017. At a threshold of >?0.25, APPROVE had similar sensitivity and positive predictive value (PPV) (sensitivity 63% and PPV 21% in 2013 vs 64% and 16%, respectively, in 2017). Compared to APPROVE in 2013, at a threshold to achieve comparable PPV (19% at MEWS >?4 and 22% at NEWS >?6), the MEWS and NEWS had lower sensitivity (16% for MEWS and NEWS). Similarly in 2017, at a comparable sensitivity threshold (64% for APPROVE >?0.25 and 67% for MEWS and NEWS >?4), more patients who triggered an alert developed the event with APPROVE (PPV 16%) while achieving a lower false positive rate (FPR 5%) compared to the MEWS (PPV 7%, FPR 14%) and NEWS (PPV 4%, FPR 25%). CONCLUSIONS:An automated EHR model to identify patients at high risk of MV or death was validated retrospectively and prospectively, and was determined to be feasible for real-time risk identification. TRIAL REGISTRATION:ClinicalTrials.gov, NCT02488174 . Registered on 18 March 2015.

SUBMITTER: Dziadzko MA

PROVIDER: S-EPMC6206729 | biostudies-literature | 2018 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Multicenter derivation and validation of an early warning score for acute respiratory failure or death in the hospital.

Dziadzko Mikhail A MA Novotny Paul J PJ Sloan Jeff J Gajic Ognjen O Herasevich Vitaly V Mirhaji Parsa P Wu Yiyuan Y Gong Michelle Ng MN

Critical care (London, England) 20181030 1

<h4>Background</h4>Acute respiratory failure occurs frequently in hospitalized patients and often starts before ICU admission. A risk stratification tool to predict mortality and risk for mechanical ventilation (MV) may allow for earlier evaluation and intervention. We developed and validated an automated electronic health record (EHR)-based model-Accurate Prediction of Prolonged Ventilation (APPROVE)-to identify patients at risk of death or respiratory failure requiring >= 48 h of MV.<h4>Method ...[more]

PMID: 30373653

Similar Datasets

Project description:BackgroundEarly warning scores (EWS) have been developed as clinical prognostication tools to identify acutely deteriorating patients. In the past few years, there has been a proliferation of studies that describe the development and validation of novel machine learning-based EWS. Systematic reviews of published studies which focus on evaluating performance of both well-established and novel EWS have shown conflicting conclusions. A possible reason is the heterogeneity in validation methods applied. In this review, we aim to examine the methodologies and metrics used in studies which perform EWS validation.MethodsA systematic review of all eligible studies from the MEDLINE database and other sources, was performed. Studies were eligible if they performed validation on at least one EWS and reported associations between EWS scores and inpatient mortality, intensive care unit (ICU) transfers, or cardiac arrest (CA) of adults. Two reviewers independently did a full-text review and performed data abstraction by using standardized data-worksheet based on the TRIPOD (Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis) checklist. Meta-analysis was not performed due to heterogeneity.ResultsThe key differences in validation methodologies identified were (1) validation dataset used, (2) outcomes of interest, (3) case definition, time of EWS use and aggregation methods, and (4) handling of missing values. In terms of case definition, among the 48 eligible studies, 34 used the patient episode case definition while 12 used the observation set case definition, and 2 did the validation using both case definitions. Of those that used the patient episode case definition, 18 studies validated the EWS at a single point of time, mostly using the first recorded observation. The review also found more than 10 different performance metrics reported among the studies.ConclusionsMethodologies and performance metrics used in studies performing validation on EWS were heterogeneous hence making it difficult to interpret and compare EWS performance. Standardizing EWS validation methodology and reporting can potentially address this issue.

Project description:BackgroundReadmissions to hospital are common, costly and often preventable. An easy-to-use index to quantify the risk of readmission or death after discharge from hospital would help clinicians identify patients who might benefit from more intensive post-discharge care. We sought to derive and validate an index to predict the risk of death or unplanned readmission within 30 days after discharge from hospital to the community.MethodsIn a prospective cohort study, 48 patient-level and admission-level variables were collected for 4812 medical and surgical patients who were discharged to the community from 11 hospitals in Ontario. We used a split-sample design to derive and validate an index to predict the risk of death or nonelective readmission within 30 days after discharge. This index was externally validated using administrative data in a random selection of 1,000,000 Ontarians discharged from hospital between 2004 and 2008.ResultsOf the 4812 participating patients, 385 (8.0%) died or were readmitted on an unplanned basis within 30 days after discharge. Variables independently associated with this outcome (from which we derived the mnemonic "LACE") included length of stay ("L"); acuity of the admission ("A"); comorbidity of the patient (measured with the Charlson comorbidity index score) ("C"); and emergency department use (measured as the number of visits in the six months before admission) ("E"). Scores using the LACE index ranged from 0 (2.0% expected risk of death or urgent readmission within 30 days) to 19 (43.7% expected risk). The LACE index was discriminative (C statistic 0.684) and very accurate (Hosmer-Lemeshow goodness-of-fit statistic 14.1, p=0.59) at predicting outcome risk.InterpretationThe LACE index can be used to quantify risk of death or unplanned readmission within 30 days after discharge from hospital. This index can be used with both primary and administrative data. Further research is required to determine whether such quantification changes patient care or outcomes.

Project description:AbstractAs severe acute respiratory syndrome coronavirus 2 continues to spread, easy-to-use risk models that predict hospital mortality can assist in clinical decision making and triage. We aimed to develop a risk score model for in-hospital mortality in patients hospitalized with 2019 novel coronavirus (COVID-19) that was robust across hospitals and used clinical factors that are readily available and measured standardly across hospitals.In this retrospective observational study, we developed a risk score model using data collected by trained abstractors for patients in 20 diverse hospitals across the state of Michigan (Mi-COVID19) who were discharged between March 5, 2020 and August 14, 2020. Patients who tested positive for severe acute respiratory syndrome coronavirus 2 during hospitalization or were discharged with an ICD-10 code for COVID-19 (U07.1) were included. We employed an iterative forward selection approach to consider the inclusion of 145 potential risk factors available at hospital presentation. Model performance was externally validated with patients from 19 hospitals in the Mi-COVID19 registry not used in model development. We shared the model in an easy-to-use online application that allows the user to predict in-hospital mortality risk for a patient if they have any subset of the variables in the final model.Two thousand one hundred and ninety-three patients in the Mi-COVID19 registry met our inclusion criteria. The derivation and validation sets ultimately included 1690 and 398 patients, respectively, with mortality rates of 19.6% and 18.6%, respectively. The average age of participants in the study after exclusions was 64 years old, and the participants were 48% female, 49% Black, and 87% non-Hispanic. Our final model includes the patient's age, first recorded respiratory rate, first recorded pulse oximetry, highest creatinine level on day of presentation, and hospital's COVID-19 mortality rate. No other factors showed sufficient incremental model improvement to warrant inclusion. The area under the receiver operating characteristics curve for the derivation and validation sets were .796 (95% confidence interval, .767-.826) and .829 (95% confidence interval, .782-.876) respectively.We conclude that the risk of in-hospital mortality in COVID-19 patients can be reliably estimated using a few factors, which are standardly measured and available to physicians very early in a hospital encounter.

Project description:The aim of this study is to evaluate the usefulness of the pre-hospital National Early Warning Score (pNEWS) and the pre-hospital Modified Early Warning Score (pMEWS) for predicting admission and in-hospital mortality in elderly patients presenting to the emergency department (ED). We also compare the value of the pNEWS with that of the ED NEWS (eNEWS) and ED MEWS (eMEWS) for predicting admission and in-hospital mortality. This retrospective, single-centre observational study was carried out in the ED of Jikei University Kashiwa Hospital, in Chiba, Japan, from 1st April 2017 to 31st March 2018. All patients aged 65 years or older were included in this study. The pNEWS/eNEWS were derived from seven common physiological vital signs: respiratory rate, peripheral oxygen saturation, the presence of inhaled oxygen parameters, body temperature, systolic blood pressure, pulse rate and Alert, responds to Voice, responds to Pain, Unresponsive (AVPU) score, whereas the pMEWS/eMEWS were derived from six common physiological vital signs: respiratory rate, peripheral oxygen saturation, body temperature, systolic blood pressure, pulse rate and AVPU score. Discrimination was assessed by plotting the receiver operating characteristic (ROC) curve and calculating the area under the ROC curve (AUC). The median pNEWS, pMEWS, eNEWS and eMEWS were significantly higher at admission than at discharge (p < 0.001). The median pNEWS, pMEWS, eNEWS and eMEWS of non-survivors were significantly higher than those of the survivors (p < 0.001). The AUC for predicting admission was 0.559 for the pNEWS and 0.547 for the pMEWS. There was no significant difference between the AUCs of the pNEWS and the pMEWS for predicting admission (p = 0.102). The AUCs for predicting in-hospital mortality were 0.678 for the pNEWS and 0.652 for the pMEWS. There was no significant difference between the AUCs of the pNEWS and the pMEWS for predicting in-hospital mortality (p = 0.081). The AUC for predicting admission was 0.628 for the eNEWS and 0.591 for the eMEWS. The AUC of the eNEWS was significantly greater than that of the eMEWS for predicting admission (p < 0.001). The AUC for predicting in-hospital mortality was 0.789 for the eNEWS and 0.720 for the eMEWS. The AUC of the eNEWS was significantly greater than that of the eMEWS for predicting in-hospital mortality (p < 0.001). For admission and in-hospital mortality, the AUC of the eNEWS was significantly greater than that of the pNEWS (p < 0.001, p < 0.001), and the AUC of the eMEWS was significantly greater than that of the pMEWS (p < 0.01, p < 0.05). Our single-centre study has demonstrated the low utility of the pNEWS and the pMEWS as predictors of admission and in-hospital mortality in elderly patients, whereas the eNEWS and the eMEWS predicted admission and in-hospital mortality more accurately. Evidence from multicentre studies is needed before introducing pre-hospital versions of risk-scoring systems.

Dataset Information

Multicenter derivation and validation of an early warning score for acute respiratory failure or death in the hospital.

Publications

Multicenter derivation and validation of an early warning score for acute respiratory failure or death in the hospital.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets