Dataset Information

Development and external validation of a COVID-19 mortality risk prediction algorithm: a multicentre retrospective cohort study.

ABSTRACT:

Objective

This study aimed to develop and externally validate a COVID-19 mortality risk prediction algorithm.

Design

Retrospective cohort study.

Setting

Five designated tertiary hospitals for COVID-19 in Hubei province, China.

Participants

We routinely collected medical data of 1364 confirmed adult patients with COVID-19 between 8 January and 19 March 2020. Among them, 1088 patients from two designated hospitals in Wuhan were used to develop the prognostic model, and 276 patients from three hospitals outside Wuhan were used for external validation. All patients were followed up for a maximal of 60 days after the diagnosis of COVID-19.

Methods

The model discrimination was assessed by the area under the receiver operating characteristic curve (AUC) and Somers' D test, and calibration was examined by the calibration plot. Decision curve analysis was conducted.

Main outcome measures

The primary outcome was all-cause mortality within 60 days after the diagnosis of COVID-19.

Results

The full model included seven predictors of age, respiratory failure, white cell count, lymphocytes, platelets, D-dimer and lactate dehydrogenase. The simple model contained five indicators of age, respiratory failure, coronary heart disease, renal failure and heart failure. After cross-validation, the AUC statistics based on derivation cohort were 0.96 (95% CI, 0.96 to 0.97) for the full model and 0.92 (95% CI, 0.89 to 0.95) for the simple model. The AUC statistics based on the external validation cohort were 0.97 (95% CI, 0.96 to 0.98) for the full model and 0.88 (95% CI, 0.80 to 0.96) for the simple model. Good calibration accuracy of these two models was found in the derivation and validation cohort.

Conclusion

The prediction models showed good model performance in identifying patients with COVID-19 with a high risk of death in 60 days. It may be useful for acute risk classification.

Web calculator

We provided a freely accessible web calculator (https://www.whuyijia.com/).

SUBMITTER: Mei J

PROVIDER: S-EPMC7768618 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundPublic policy measures and clinical risk assessments relevant to COVID-19 need to be aided by risk prediction models that are rigorously developed and validated. We aimed to externally validate a risk prediction algorithm (QCovid) to estimate mortality outcomes from COVID-19 in adults in England.MethodsWe did a population-based cohort study using the UK Office for National Statistics Public Health Linked Data Asset, a cohort of individuals aged 19-100 years, based on the 2011 census and linked to Hospital Episode Statistics, the General Practice Extraction Service data for pandemic planning and research, and radiotherapy and systemic chemotherapy records. The primary outcome was time to COVID-19 death, defined as confirmed or suspected COVID-19 death as per death certification. Two periods were used: (1) Jan 24 to April 30, 2020, and (2) May 1 to July 28, 2020. We assessed the performance of the QCovid algorithms using measures of discrimination and calibration. Using predicted 90-day risk of COVID-19 death, we calculated r2 values, Brier scores, and measures of discrimination and calibration with corresponding 95% CIs over the two time periods.FindingsWe included 34 897 648 adults aged 19-100 years resident in England. 26 985 (0·08%) COVID-19 deaths occurred during the first period and 13 177 (0·04%) during the second. The algorithms had good discrimination and calibration in both periods. In the first period, they explained 77·1% (95% CI 76·9-77·4) of the variation in time to death in men and 76·3% (76·0-76·6) in women. The D statistic was 3·761 (3·732-3·789) for men and 3·671 (3·640-3·702) for women and Harrell's C was 0·935 (0·933-0·937) for men and 0·945 (0·943-0·947) for women. Similar results were obtained for the second time period. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths in the first period was 65·94% for men and 71·67% for women.InterpretationThe QCovid population-based risk algorithm performed well, showing high levels of discrimination for COVID-19 deaths in men and women for both time periods. QCovid has the potential to be dynamically updated as the pandemic evolves and, therefore, has potential use in guiding national policy.FundingUK National Institute for Health Research.

Project description:ObjectiveTo externally validate various prognostic models and scoring rules for predicting short term mortality in patients admitted to hospital for covid-19.DesignTwo stage individual participant data meta-analysis.SettingSecondary and tertiary care.Participants46 914 patients across 18 countries, admitted to a hospital with polymerase chain reaction confirmed covid-19 from November 2019 to April 2021.Data sourcesMultiple (clustered) cohorts in Brazil, Belgium, China, Czech Republic, Egypt, France, Iran, Israel, Italy, Mexico, Netherlands, Portugal, Russia, Saudi Arabia, Spain, Sweden, United Kingdom, and United States previously identified by a living systematic review of covid-19 prediction models published in The BMJ, and through PROSPERO, reference checking, and expert knowledge.Model selection and eligibility criteriaPrognostic models identified by the living systematic review and through contacting experts. A priori models were excluded that had a high risk of bias in the participant domain of PROBAST (prediction model study risk of bias assessment tool) or for which the applicability was deemed poor.MethodsEight prognostic models with diverse predictors were identified and validated. A two stage individual participant data meta-analysis was performed of the estimated model concordance (C) statistic, calibration slope, calibration-in-the-large, and observed to expected ratio (O:E) across the included clusters.Main outcome measures30 day mortality or in-hospital mortality.ResultsDatasets included 27 clusters from 18 different countries and contained data on 46 914patients. The pooled estimates ranged from 0.67 to 0.80 (C statistic), 0.22 to 1.22 (calibration slope), and 0.18 to 2.59 (O:E ratio) and were prone to substantial between study heterogeneity. The 4C Mortality Score by Knight et al (pooled C statistic 0.80, 95% confidence interval 0.75 to 0.84, 95% prediction interval 0.72 to 0.86) and clinical model by Wang et al (0.77, 0.73 to 0.80, 0.63 to 0.87) had the highest discriminative ability. On average, 29% fewer deaths were observed than predicted by the 4C Mortality Score (pooled O:E 0.71, 95% confidence interval 0.45 to 1.11, 95% prediction interval 0.21 to 2.39), 35% fewer than predicted by the Wang clinical model (0.65, 0.52 to 0.82, 0.23 to 1.89), and 4% fewer than predicted by Xie et al's model (0.96, 0.59 to 1.55, 0.21 to 4.28).ConclusionThe prognostic value of the included models varied greatly between the data sources. Although the Knight 4C Mortality Score and Wang clinical model appeared most promising, recalibration (intercept and slope updates) is needed before implementation in routine care.

Project description:AbstractAs severe acute respiratory syndrome coronavirus 2 continues to spread, easy-to-use risk models that predict hospital mortality can assist in clinical decision making and triage. We aimed to develop a risk score model for in-hospital mortality in patients hospitalized with 2019 novel coronavirus (COVID-19) that was robust across hospitals and used clinical factors that are readily available and measured standardly across hospitals.In this retrospective observational study, we developed a risk score model using data collected by trained abstractors for patients in 20 diverse hospitals across the state of Michigan (Mi-COVID19) who were discharged between March 5, 2020 and August 14, 2020. Patients who tested positive for severe acute respiratory syndrome coronavirus 2 during hospitalization or were discharged with an ICD-10 code for COVID-19 (U07.1) were included. We employed an iterative forward selection approach to consider the inclusion of 145 potential risk factors available at hospital presentation. Model performance was externally validated with patients from 19 hospitals in the Mi-COVID19 registry not used in model development. We shared the model in an easy-to-use online application that allows the user to predict in-hospital mortality risk for a patient if they have any subset of the variables in the final model.Two thousand one hundred and ninety-three patients in the Mi-COVID19 registry met our inclusion criteria. The derivation and validation sets ultimately included 1690 and 398 patients, respectively, with mortality rates of 19.6% and 18.6%, respectively. The average age of participants in the study after exclusions was 64 years old, and the participants were 48% female, 49% Black, and 87% non-Hispanic. Our final model includes the patient's age, first recorded respiratory rate, first recorded pulse oximetry, highest creatinine level on day of presentation, and hospital's COVID-19 mortality rate. No other factors showed sufficient incremental model improvement to warrant inclusion. The area under the receiver operating characteristics curve for the derivation and validation sets were .796 (95% confidence interval, .767-.826) and .829 (95% confidence interval, .782-.876) respectively.We conclude that the risk of in-hospital mortality in COVID-19 patients can be reliably estimated using a few factors, which are standardly measured and available to physicians very early in a hospital encounter.

Project description:AimsTo assess the prognostic accuracy of comorbidity-adjusted National Early Warning Score in suspected Coronavirus disease 2019 patients transferred from nursing homes by the Emergency Department.DesignMulticentre retrospective cohort study.MethodsPatients transferred by high-priority ambulances from nursing homes to Emergency Departments with suspected severe acute respiratory syndrome coronavirus 2 infection, from March 12 to July 31 2020, were considered. Included variables were: clinical covariates (respiratory rate, oxygen saturation, systolic blood pressure, heart rate, temperature, level of consciousness and supplemental oxygen use), the presence of comorbidities and confirmatory analytical diagnosis of severe acute respiratory syndrome coronavirus 2 infection. The primary outcome was a 2-day mortality rate. The discriminatory capability of the National Early Warning Score was assessed by the area under the receiver operating characteristic curve in two different cohorts, the validation and the revalidation, which were randomly selected from the main cohort.ResultsA total of 337 nursing homes, 10 advanced life support units, 51 basic life support units and 8 hospitals in Spain entailing 1,324 patients (median age 87 years) was involved in this study. Two-day mortality was 11.5% (152 cases), with a positivity rate of severe acute respiratory syndrome coronavirus 2 of 51.2%, 77.7% of hospitalization from whom 1% was of intensive care unit admission. The National Early Warning Score results for the revalidation cohort presented an AUC of 0.771, and of 0.885, 0.778 and 0.730 for the low-, medium- and high-level groups of comorbidities.ConclusionThe comorbidity-adjusted National Early Warning Score provides a good short-term prognostic criterion, information that can help in the decision-making process to guide the best strategy for each older adult, under the current pandemic.ImpactWhat problem did the study address? Under the current coronavirus disease 2019 pandemic, targeting older adults at high risk of deterioration in nursing homes remains challenging. What were the main findings? Comorbidity-adjusted National Early Warning Score helps to forecast the risk of clinical deterioration more accurately. Where and on whom will the research have impact? A high NEWS, with a low level of comorbidity is associated with optimal predictive performance, making these older adults likely to benefit from continued follow up and potentially hospital referral under the current coronavirus disease 2019 pandemic.