Dataset Information

Regional performance variation in external validation of four prediction models for severity of COVID-19 at hospital admission: An observational multi-centre cohort study.

ABSTRACT:

Background

Prediction models should be externally validated to assess their performance before implementation. Several prediction models for coronavirus disease-19 (COVID-19) have been published. This observational cohort study aimed to validate published models of severity for hospitalized patients with COVID-19 using clinical and laboratory predictors.

Methods

Prediction models fitting relevant inclusion criteria were chosen for validation. The outcome was either mortality or a composite outcome of mortality and ICU admission (severe disease). 1295 patients admitted with symptoms of COVID-19 at Kings Cross Hospital (KCH) in London, United Kingdom, and 307 patients at Oslo University Hospital (OUH) in Oslo, Norway were included. The performance of the models was assessed in terms of discrimination and calibration.

Results

We identified two models for prediction of mortality (referred to as Xie and Zhang1) and two models for prediction of severe disease (Allenbach and Zhang2). The performance of the models was variable. For prediction of mortality Xie had good discrimination at OUH with an area under the receiver-operating characteristic (AUROC) 0.87 [95% confidence interval (CI) 0.79-0.95] and acceptable discrimination at KCH, AUROC 0.79 [0.76-0.82]. In prediction of severe disease, Allenbach had acceptable discrimination (OUH AUROC 0.81 [0.74-0.88] and KCH AUROC 0.72 [0.68-0.75]). The Zhang models had moderate to poor discrimination. Initial calibration was poor for all models but improved with recalibration.

Conclusions

The performance of the four prediction models was variable. The Xie model had the best discrimination for mortality, while the Allenbach model had acceptable results for prediction of severe disease.

SUBMITTER: Wickstrom KE

PROVIDER: S-EPMC8386866 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundIdentification of patients with novel coronavirus disease 2019 (COVID-19) requiring hospital admission or at high-risk of in-hospital mortality is essential to guide patient triage and to provide timely treatment for higher risk hospitalized patients.MethodsA retrospective multi-centre (8 hospital) cohort at Beaumont Health, Michigan, USA, reporting on COVID-19 patients diagnosed between 1 March and 1 April 2020 was used for score validation. The COVID-19 Risk of Complications Score was automatically computed by the EHR. Multivariate logistic regression models were built to predict hospital admission and in-hospital mortality using individual variables constituting the score. Validation was performed using both discrimination and calibration.ResultsCompared to Green scores, Yellow Scores (OR: 5.72) and Red Scores (OR: 19.1) had significantly higher odds of admission (both p < .0001). Similarly, Yellow Scores (OR: 4.73) and Red Scores (OR: 13.3) had significantly higher odds of in-hospital mortality than Green Scores (both p < .0001). The cross-validated C-Statistics for the external validation cohort showed good discrimination for both hospital admission (C = 0.79 (95% CI: 0.77-0.81)) and in-hospital mortality (C = 0.75 (95% CI: 0.71-0.78)).ConclusionsThe COVID-19 Risk of Complications Score predicts the need for hospital admission and in-hospital mortality patients with COVID-19. Key points: Can an electronic health record generated risk score predict the risk of hospital admission and in-hospital mortality in patients diagnosed with coronavirus disease 2019 (COVID-19)? In both validation cohorts of 2,025 and 1,290 COVID-19, the cross-validated C-Statistics showed good discrimination for both hospital admission (C = 0.79 (95% CI: 0.77-0.81)) and in-hospital mortality (C = 0.75 (95% CI: 0.71-0.78)), respectively. The COVID-19 Risk of Complications Score may help predict the need for hospital admission if a patient contracts SARS-CoV-2 infection and in-hospital mortality for a hospitalized patient with COVID-19.

Project description:ObjectiveTo develop and validate a prognostic model for in-hospital mortality after four days based on age, fever at admission and five haematological parameters routinely measured in hospitalized Covid-19 patients during the first four days after admission.MethodsHaematological parameters measured during the first 4 days after admission were subjected to a linear mixed model to obtain patient-specific intercepts and slopes for each parameter. A prediction model was built using logistic regression with variable selection and shrinkage factor estimation supported by bootstrapping. Model development was based on 481 survivors and 97 non-survivors, hospitalized before the occurrence of mutations. Internal validation was done by 10-fold cross-validation. The model was temporally-externally validated in 299 survivors and 42 non-survivors hospitalized when the Alpha variant (B.1.1.7) was prevalent.ResultsThe final model included age, fever on admission as well as the slope or intercept of lactate dehydrogenase, platelet count, C-reactive protein, and creatinine. Tenfold cross validation resulted in a mean area under the receiver operating characteristic curve (AUROC) of 0.92, a mean calibration slope of 1.0023 and a Brier score of 0.076. At temporal-external validation, application of the previously developed model showed an AUROC of 0.88, a calibration slope of 0.95 and a Brier score of 0.073. Regarding the relative importance of the variables, the (apparent) variation in mortality explained by the six variables deduced from the haematological parameters measured during the first four days is higher (explained variation 0.295) than that of age (0.210).ConclusionsThe presented model requires only variables routinely acquired in hospitals, which allows immediate and wide-spread use as a decision support for earlier discharge of low-risk patients to reduce the burden on the health care system.Clinical trial registrationAustrian Coronavirus Adaptive Clinical Trial (ACOVACT); ClinicalTrials.gov, identifier NCT04351724.

Project description:ObjectivesTo devise an assessment tool to aid discharge and admission decision-making in relation to children and young people in hospital urgent and emergency care facilities, and thereby improve the quality of care that patients receive, using a clinical prediction modelling approach.DesignObservational cohort study with internal and external validation of a predictive tool.SettingTwo general emergency departments (EDs) and an urgent care centre in the North of England.ParticipantsThe eligibility criteria were children and young people 0-16 years of age who attended one of the three hospital sites within one National Health Service (NHS) organisation. Children were excluded if they opted out of the study, were brought to the ED following their death in the community or arrived in cardiac arrest when the heart rate and respiratory rate would be unmeasurable.Main outcome measuresAdmission or discharge. A participant was defined as being admitted to hospital if they left the ED to enter the hospital for further assessment, (including being admitted to an observation and assessment unit or hospital ward), either on first presentation or with the same complaint within 7 days. Those who were not admitted were defined as having been discharged.ResultsThe study collected data on 36 365 participants. 15 328 participants were included in the final analysis cohort (21 045 observations) and 17 710 participants were included in the validation cohort (23 262 observations). There were 14 variables entered into the regression analysis. Of the 13 that remained in the final model, 10 were present in all 500 bootstraps. The resulting Paediatric Admission Guidance in the Emergency Department (PAGE) score demonstrated good internal validity. The C-index (area under the ROC) was 0.779 (95% CI 0.772 to 0.786).ConclusionsFor units without the immediate availability of paediatricians the PAGE score can assist staff to determine risk of admission. Cut-off values will need to be adjusted to local circumstance.Study protocolThe study protocol has been published in an open access journal: Riaz et al Refining and testing the diagnostic accuracy of an assessment tool (Pennine Acute Hospitals NHS Trust-Paediatric Observation Priority Score) to predict admission and discharge of children and young people who attend an ED: protocol for an observational study. BMC Pediatr 18, 303 (2018). https://doi.org/10.1186/s12887-018-1268-7.Trial registration numberThe protocol has been published and the study registered (NIHR RfPB Grant: PB-PG-0815-20034; ClinicalTrials.gov:213469).

Project description:ObjectivesCOVID-19 causes lung parenchymal and endothelial damage that lead to hypoxic acute respiratory failure (hARF). The influence of hARF severity on patients' outcomes is still poorly understood.DesignObservational, prospective, multicentre study.SettingThree academic hospitals in Milan (Italy) involving three respiratory high dependency units and three general wards.ParticipantsConsecutive adult hospitalised patients with a virologically confirmed diagnosis of COVID-19. Patients aged <18 years or unable to provide informed consent were excluded.InterventionsAnthropometrical, clinical characteristics and blood biomarkers were assessed within the first 24 hours from admission. hARF was graded as follows: severe (partial pressure of oxygen to fraction of inspired oxygen ratio (PaO2/FiO2) <100 mm Hg); moderate (PaO2/FiO2 101-200 mm Hg); mild (PaO2/FiO2 201-300 mm Hg) and normal (PaO2/FiO2 >300 mm Hg).Primary and secondary outcome measuresThe primary outcome was the assessment of clinical characteristics and in-hospital mortality based on the severity of respiratory failure. Secondary outcomes were intubation rate and application of continuous positive airway pressure during hospital stay.Results412 patients were enrolled (280 males, 68%). Median (IQR) age was 66 (55-76) years with a PaO2/FiO2 at admission of 262 (140-343) mm Hg. 50.2% had a cardiovascular disease. Prevalence of mild, moderate and severe hARF was 24.4%, 21.9% and 15.5%, respectively. In-hospital mortality proportionally increased with increasing impairment of gas exchange (p<0.001). The only independent risk factors for mortality were age ≥65 years (HR 3.41; 95% CI 2.00 to 5.78, p<0.0001), PaO2/FiO2 ratio ≤200 mm Hg (HR 3.57; 95% CI 2.20 to 5.77, p<0.0001) and respiratory failure at admission (HR 3.58; 95% CI 1.05 to 12.18, p=0.04).ConclusionsA moderate-to-severe impairment in PaO2/FiO2 was independently associated with a threefold increase in risk of in-hospital mortality. Severity of respiratory failure is useful to identify patients at higher risk of mortality.Trial registration numberNCT04307459.

Project description:ObjectivesHospitalisation is frequently used as a marker of disease severity in observational Emergency Department (ED) studies. The comparison of ED admission rates is complex in potentially being influenced by the characteristics of the region, ED, physician and patient. We aimed to study variation in ED admission rates of febrile children, to assess whether variation could be explained by disease severity and to identify patient groups with large variation, in order to use this to reduce unnecessary health care utilization that is often due to practice variation.DesignMOFICHE (Management and Outcome of Fever in children in Europe, part of the PERFORM study, www.perform2020.org), is a prospective cohort study using routinely collected data on febrile children regarding patient characteristics (age, referral, vital signs and clinical alarming signs), diagnostic tests, therapy, diagnosis and hospital admission.Setting and participantsData were collected on febrile children aged 0-18 years presenting to 12 European EDs (2017-2018).Main outcome measuresWe compared admission rates between EDs by using standardised admission rates after adjusting for patient characteristics and initiated tests at the ED, where standardised rates >1 demonstrate higher admission rates than expected and rates <1 indicate lower rates than expected based on the ED patient population.ResultsWe included 38,120 children. Of those, 9.695 (25.4%) were admitted to a general ward (range EDs 5.1-54.5%). Adjusted standardised admission rates ranged between 0.6 and 1.5. The largest variation was seen in short admission rates (0.1-5.0), PICU admission rates (0.2-2.2), upper respiratory tract infections (0.4-1.7) and fever without focus (0.5-2.7). Variation was small in sepsis/meningitis (0.9-1.1).ConclusionsLarge variation exists in admission rates of febrile children evaluated at European EDs, however, this variation is largely reduced after correcting for patient characteristics and therefore overall admission rates seem to adequately reflect disease severity or a potential for a severe disease course. However, for certain patient groups variation remains high even after adjusting for patient characteristics.

Project description:ObjectiveThe Diabetes Early Re-admission Risk Indicator (DERRI™) was previously developed and internally validated as a tool to predict the risk of all-cause re-admission within 30 days of discharge (30-day re-admission) of hospitalized patients with diabetes. In this study, the predictive performance of the DERRI™ with and without additional predictors was assessed in an external sample.MethodsWe conducted a retrospective cohort study of adult patients with diabetes discharged from two academic medical centers between January 1, 2000 and December 31, 2014. We applied the previously developed DERRI™, which includes admission laboratory results, sociodemographics, a diagnosis of certain comorbidities, and recent discharge information, and evaluated the effect of adding metabolic indicators on predictive performance using multivariable logistic regression. Total cholesterol and hemoglobin A1c (A1c) were selected based on clinical relevance and univariate association with 30-day re-admission.ResultsAmong 105,974 discharges, 19,032 (18.0%) were followed by 30-day re-admission for any cause. The DERRI™ had a C-statistic of 0.634 for 30-day re-admission. Total cholesterol was the lipid parameter most strongly associated with 30-day re-admission. The DERRI™ predictors A1c and total cholesterol were significantly associated with 30-day re-admission; however, their addition to the DERRI™ did not significantly change model performance (C-statistic, 0.643 [95% confidence interval, 0.638 to 0.647]; P = .92).ConclusionPerformance of the DERRI™ in this external cohort was modest but comparable to other re-admission prediction models. Addition of A1c and total cholesterol to the DERRI™ did not significantly improve performance. Although the DERRI™ may be useful to direct resources toward diabetes patients at higher risk, better prediction is needed.AbbreviationsA1c = hemoglobin A1c; CI = confidence interval; DERRI™ = Diabetes Early Re-admission Risk Indicator; GEE = generalized estimating equation; HDL-C = high-density-lipoprotein cholesterol; ICD-9-CM = International Classification of Diseases, Ninth Revision, Clinical Modification; LDL-C = low-density-lipoprotein cholesterol.