Dataset Information

Development and external validation of prognostic models for COVID-19 to support risk stratification in secondary care.

ABSTRACT:

Objectives

Existing UK prognostic models for patients admitted to the hospital with COVID-19 are limited by reliance on comorbidities, which are under-recorded in secondary care, and lack of imaging data among the candidate predictors. Our aims were to develop and externally validate novel prognostic models for adverse outcomes (death and intensive therapy unit (ITU) admission) in UK secondary care and externally validate the existing 4C score.

Design

Candidate predictors included demographic variables, symptoms, physiological measures, imaging and laboratory tests. Final models used logistic regression with stepwise selection.

Setting

Model development was performed in data from University Hospitals Birmingham (UHB). External validation was performed in the CovidCollab dataset.

Participants

Patients with COVID-19 admitted to UHB January-August 2020 were included.

Main outcome measures

Death and ITU admission within 28 days of admission.

Results

1040 patients with COVID-19 were included in the derivation cohort; 288 (28%) died and 183 (18%) were admitted to ITU within 28 days of admission. Area under the receiver operating characteristic curve (AUROC) for mortality was 0.791 (95% CI 0.761 to 0.822) in UHB and 0.767 (95% CI 0.754 to 0.780) in CovidCollab; AUROC for ITU admission was 0.906 (95% CI 0.883 to 0.929) in UHB and 0.811 (95% CI 0.795 to 0.828) in CovidCollab. Models showed good calibration. Addition of comorbidities to candidate predictors did not improve model performance. AUROC for the International Severe Acute Respiratory and Emerging Infection Consortium 4C score in the UHB dataset was 0.753 (95% CI 0.720 to 0.785).

Conclusions

The novel prognostic models showed good discrimination and calibration in derivation and external validation datasets, and performed at least as well as the existing 4C score using only routinely collected patient information. The models can be integrated into electronic medical records systems to calculate each individual patient's probability of death or ITU admission at the time of hospital admission. Implementation of the models and clinical utility should be evaluated.

SUBMITTER: Adderley NJ

PROVIDER: S-EPMC8764710 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Objectives backgroundTo externally validate clinical prediction models that aim to predict progression to invasive ventilation or death on the ICU in patients admitted with confirmed COVID-19 pneumonitis.DesignSingle-center retrospective external validation study.Data sourcesRoutinely collected healthcare data in the ICU electronic patient record. Curated data recorded for each ICU admission for the purposes of the U.K. Intensive Care National Audit and Research Centre (ICNARC).SettingThe ICU at Manchester Royal Infirmary, Manchester, United Kingdom.PatientsThree hundred forty-nine patients admitted to ICU with confirmed COVID-19 Pneumonitis, older than 18 years, from March 1, 2020, to February 28, 2022. Three hundred two met the inclusion criteria for at least one model. Fifty-five of the 349 patients were admitted before the widespread adoption of dexamethasone for the treatment of severe COVID-19 (pre-dexamethasone patients).OutcomesAbility to be externally validated, discriminate, and calibrate.MethodsArticles meeting the inclusion criteria were identified, and those that gave sufficient details on predictors used and methods to generate predictions were tested in our cohort of patients, which matched the original publications' inclusion/exclusion criteria and endpoint.ResultsThirteen clinical prediction articles were identified. There was insufficient information available to validate models in five of the articles; a further three contained predictors that were not routinely measured in our ICU cohort and were not validated; three had performance that was substantially lower than previously published (range C-statistic = 0.483-0.605 in pre-dexamethasone patients and C = 0.494-0.564 among all patients). One model retained its discriminative ability in our cohort compared with previously published results (C = 0.672 and 0.686), and one retained performance among pre-dexamethasone patients but was poor in all patients (C = 0.793 and 0.596). One model could be calibrated but with poor performance.ConclusionsOur findings, albeit from a single center, suggest that the published performance of COVID-19 prediction models may not be replicated when translated to other institutions. In light of this, we would encourage bedside intensivists to reflect on the role of clinical prediction models in their own clinical decision-making.

Project description:BackgroundThe rapid spread of coronavirus disease 2019 (COVID-19) revealed significant constraints in critical care capacity. In anticipation of subsequent waves, reliable prediction of disease severity is essential for critical care capacity management and may enable earlier targeted interventions to improve patient outcomes. The purpose of this study is to develop and externally validate a prognostic model/clinical tool for predicting COVID-19 critical disease at presentation to medical care.MethodsThis is a retrospective study of a prognostic model for the prediction of COVID-19 critical disease where critical disease was defined as ICU admission, ventilation, and/or death. The derivation cohort was used to develop a multivariable logistic regression model. Covariates included patient comorbidities, presenting vital signs, and laboratory values. Model performance was assessed on the validation cohort by concordance statistics. The model was developed with consecutive patients with COVID-19 who presented to University of California Irvine Medical Center in Orange County, California. External validation was performed with a random sample of patients with COVID-19 at Emory Healthcare in Atlanta, Georgia.ResultsOf a total 3208 patients tested in the derivation cohort, 9% (299/3028) were positive for COVID-19. Clinical data including past medical history and presenting laboratory values were available for 29% (87/299) of patients (median age, 48 years [range, 21-88 years]; 64% [36/55] male). The most common comorbidities included obesity (37%, 31/87), hypertension (37%, 32/87), and diabetes (24%, 24/87). Critical disease was present in 24% (21/87). After backward stepwise selection, the following factors were associated with greatest increased risk of critical disease: number of comorbidities, body mass index, respiratory rate, white blood cell count, % lymphocytes, serum creatinine, lactate dehydrogenase, high sensitivity troponin I, ferritin, procalcitonin, and C-reactive protein. Of a total of 40 patients in the validation cohort (median age, 60 years [range, 27-88 years]; 55% [22/40] male), critical disease was present in 65% (26/40). Model discrimination in the validation cohort was high (concordance statistic: 0.94, 95% confidence interval 0.87-1.01). A web-based tool was developed to enable clinicians to input patient data and view likelihood of critical disease.Conclusions and relevanceWe present a model which accurately predicted COVID-19 critical disease risk using comorbidities and presenting vital signs and laboratory values, on derivation and validation cohorts from two different institutions. If further validated on additional cohorts of patients, this model/clinical tool may provide useful prognostication of critical care needs.

Project description:BackgroundThis study aimed to develop prognostic models for predicting the need for invasive mechanical ventilation (IMV) in intensive care unit (ICU) patients with COVID-19 and compare their performance with the Respiratory rate-OXygenation (ROX) index.MethodsA retrospective cohort study was conducted using data collected between March 2020 and August 2021 at three hospitals in Rio de Janeiro, Brazil. ICU patients aged 18 years and older with a diagnosis of COVID-19 were screened. The exclusion criteria were patients who received IMV within the first 24 h of ICU admission, pregnancy, clinical decision for minimal end-of-life care and missing primary outcome data. Clinical and laboratory variables were collected. Multiple logistic regression analysis was performed to select predictor variables. Models were based on the lowest Akaike Information Criteria (AIC) and lowest AIC with significant p values. Assessment of predictive performance was done for discrimination and calibration. Areas under the curves (AUC)s were compared using DeLong's algorithm. Models were validated externally using an international database.ResultsOf 656 patients screened, 346 patients were included; 155 required IMV (44.8%), 191 did not (55.2%), and 207 patients were male (59.8%). According to the lowest AIC, arterial hypertension, diabetes mellitus, obesity, Sequential Organ Failure Assessment (SOFA) score, heart rate, respiratory rate, peripheral oxygen saturation (SpO2), temperature, respiratory effort signals, and leukocytes were identified as predictors of IMV at hospital admission. According to AIC with significant p values, SOFA score, SpO2, and respiratory effort signals were the best predictors of IMV; odds ratios (95% confidence interval): 1.46 (1.07-2.05), 0.81 (0.72-0.90), 9.13 (3.29-28.67), respectively. The ROX index at admission was lower in the IMV group than in the non-IMV group (7.3 [5.2-9.8] versus 9.6 [6.8-12.9], p < 0.001, respectively). In the external validation population, the area under the curve (AUC) of the ROX index was 0.683 (accuracy 63%), the AIC model showed an AUC of 0.703 (accuracy 69%), and the lowest AIC model with significant p values had an AUC of 0.725 (accuracy 79%).ConclusionsIn the development population of ICU patients with COVID-19, SOFA score, SpO2, and respiratory effort signals predicted the need for IMV better than the ROX index. In the external validation population, although the AUCs did not differ significantly, the accuracy was higher when using SOFA score, SpO2, and respiratory effort signals compared to the ROX index. This suggests that these variables may be more useful in predicting the need for IMV in ICU patients with COVID-19.ClinicaltrialsGov identifierNCT05663528.

Project description:BackgroundThe COVID-19 pandemic has a large impact worldwide and is known to particularly affect the older population. This paper outlines the protocol for external validation of prognostic models predicting mortality risk after presentation with COVID-19 in the older population. These prognostic models were originally developed in an adult population and will be validated in an older population (≥ 70 years of age) in three healthcare settings: the hospital setting, the primary care setting, and the nursing home setting.MethodsBased on a living systematic review of COVID-19 prediction models, we identified eight prognostic models predicting the risk of mortality in adults with a COVID-19 infection (five COVID-19 specific models: GAL-COVID-19 mortality, 4C Mortality Score, NEWS2 + model, Xie model, and Wang clinical model and three pre-existing prognostic scores: APACHE-II, CURB65, SOFA). These eight models will be validated in six different cohorts of the Dutch older population (three hospital cohorts, two primary care cohorts, and a nursing home cohort). All prognostic models will be validated in a hospital setting while the GAL-COVID-19 mortality model will be validated in hospital, primary care, and nursing home settings. The study will include individuals ≥ 70 years of age with a highly suspected or PCR-confirmed COVID-19 infection from March 2020 to December 2020 (and up to December 2021 in a sensitivity analysis). The predictive performance will be evaluated in terms of discrimination, calibration, and decision curves for each of the prognostic models in each cohort individually. For prognostic models with indications of miscalibration, an intercept update will be performed after which predictive performance will be re-evaluated.DiscussionInsight into the performance of existing prognostic models in one of the most vulnerable populations clarifies the extent to which tailoring of COVID-19 prognostic models is needed when models are applied to the older population. Such insight will be important for possible future waves of the COVID-19 pandemic or future pandemics.

Project description:BackgroundSeveral prognostic models for outcomes after chronic subdural hematoma (CSDH) treatment have been published in recent years. However, these models are not sufficiently validated for use in daily clinical practice. We aimed to assess the performance of existing prediction models for outcomes in patients diagnosed with CSDH.MethodsWe systematically searched relevant literature databases up to February 2021 to identify prognostic models for outcome prediction in patients diagnosed with CSDH. For the external validation of prognostic models, we used a retrospective database, containing data of 2384 patients from three Dutch regions. Prognostic models were included if they predicted either mortality, hematoma recurrence, functional outcome, or quality of life. Models were excluded when predictors were absent in our database or available for < 150 patients in our database. We assessed calibration, and discrimination (quantified by the concordance index C) of the included prognostic models in our retrospective database.ResultsWe identified 1680 original publications of which 1656 were excluded based on title or abstract, mostly because they did not concern CSDH or did not define a prognostic model. Out of 18 identified models, three could be externally validated in our retrospective database: a model for 30-day mortality in 1656 patients, a model for 2 months, and another for 3-month hematoma recurrence both in 1733 patients. The models overestimated the proportion of patients with these outcomes by 11% (15% predicted vs. 4% observed), 1% (10% vs. 9%), and 2% (11% vs. 9%), respectively. Their discriminative ability was poor to modest (C of 0.70 [0.63-0.77]; 0.46 [0.35-0.56]; 0.59 [0.51-0.66], respectively).ConclusionsNone of the examined models showed good predictive performance for outcomes after CSDH treatment in our dataset. This study confirms the difficulty in predicting outcomes after CSDH and emphasizes the heterogeneity of CSDH patients. The importance of developing high-quality models by using unified predictors and relevant outcome measures and appropriate modeling strategies is warranted.