External validation of the 4C mortality score among COVID-19 patients admitted to hospital in Ontario, Canada: a retrospective study.
Ontology highlight
ABSTRACT:
SUBMITTER: Jones A
PROVIDER: S-EPMC8452633 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Prognostic models to predict the deterioration and mortality risk in COVID-19 patients are utterly needed to assist in informed decision making. Most of these models, however, are at high risk of bias, model overfitting, and unclear reporting. Here, we aimed to externally validate the modified (urea was omitted) 4C Deterioration Model and 4C Mortality Score in a cohort of Swiss COVID-19 patients and, second, to evaluate whether the inclusion of the neutrophil-to-lymphocyte ratio (NLR) improves the predictive performance of the models. We conducted a retrospective single-centre study with adult patients hospitalized with COVID-19. Both prediction models were updated by including the NLR. Model performance was assessed via the models' discriminatory performance (area under the curve, AUC), calibration (intercept and slope), and their performance overall (Brier score). For the validation of the 4C Deterioration Model and Mortality Score, 546 and 527 patients were included, respectively. In total, 133 (24.4%) patients met the definition of in-hospital deterioration. Discrimination of the 4C Deterioration Model was AUC = 0.78 (95% CI 0.73-0.82). A total of 55 (10.44%) patients died in hospital. Discrimination of the 4C Mortality Score was AUC = 0.85 (95% CI 0.79-0.89). There was no evidence for an incremental value of the NLR. Our data confirm the role of the modified 4C Deterioration Model and Mortality Score as reliable prediction tools for the risk of deterioration and mortality. There was no evidence that the inclusion of NLR improved model performance.
| S-EPMC9139628 | biostudies-literature
Project description:BackgroundIdentifying patients at risk for mortality from COVID-19 is crucial to triage, clinical decision-making, and the allocation of scarce hospital resources. The 4C Mortality Score effectively predicts COVID-19 mortality, but it has not been validated in a United States (U.S.) population. The purpose of this study is to determine whether the 4C Mortality Score accurately predicts COVID-19 mortality in an urban U.S. adult inpatient population.MethodsThis retrospective cohort study included adult patients admitted to a single-center, tertiary care hospital (Philadelphia, PA) with a positive SARS-CoV-2 PCR from 3/01/2020 to 6/06/2020. Variables were extracted through a combination of automated export and manual chart review. The outcome of interest was mortality during hospital admission or within 30 days of discharge.ResultsThis study included 426 patients; mean age was 64.4 years, 43.4% were female, and 54.5% self-identified as Black or African American. All-cause mortality was observed in 71 patients (16.7%). The area under the receiver operator characteristic curve of the 4C Mortality Score was 0.85 (95% confidence interval, 0.79-0.89).ConclusionsClinicians may use the 4C Mortality Score in an urban, majority Black, U.S. inpatient population. The derivation and validation cohorts were treated in the pre-vaccine era so the 4C Score may over-predict mortality in current patient populations. With stubbornly high inpatient mortality rates, however, the 4C Score remains one of the best tools available to date to inform thoughtful triage and treatment allocation.
| S-EPMC9054702 | biostudies-literature
Project description:ObjectivesEstimating mortality risk in hospitalised SARS-CoV-2+ patients may help with choosing level of care and discussions with patients. The Coronavirus Clinical Characterisation Consortium Mortality Score (4C Score) is a promising COVID-19 mortality risk model. We examined the association of risk factors with 30-day mortality in hospitalised, full-code SARS-CoV-2+ patients and investigated the discrimination and calibration of the 4C Score. This was a retrospective cohort study of SARS-CoV-2+ hospitalised patients within the RECOVER (REgistry of suspected COVID-19 in EmeRgency care) network.Setting99 emergency departments (EDs) across the USA.ParticipantsPatients ≥18 years old, positive for SARS-CoV-2 in the ED, and hospitalised.Primary outcomeDeath within 30 days of the index visit. We performed logistic regression analysis, reporting multivariable risk ratios (MVRRs) and calculated the area under the ROC curve (AUROC) and mean prediction error for the original 4C Score and after dropping the C reactive protein (CRP) component.ResultsOf 6802 hospitalised patients with COVID-19, 1149 (16.9%) died within 30 days. The 30-day mortality was increased with age 80+ years (MVRR=5.79, 95% CI 4.23 to 7.34); male sex (MVRR=1.17, 1.05 to 1.28); and nursing home/assisted living facility residence (MVRR=1.29, 1.1 to 1.48). The 4C Score had comparable discrimination in the RECOVER dataset compared with the original 4C validation dataset (AUROC: RECOVER 0.786 (95% CI 0.773 to 0.799), 4C validation 0.763 (95% CI 0.757 to 0.769). Score-specific mortalities in our sample were lower than in the 4C validation sample (mean prediction error 6.0%). Dropping the CRP component from the 4C Score did not substantially affect discrimination and 4C risk estimates were now close (mean prediction error 0.7%).ConclusionsWe independently validated 4C Score as predicting risk of 30-day mortality in hospitalised SARS-CoV-2+ patients. We recommend dropping the CRP component of the score and using our recalibrated mortality risk estimates.
| S-EPMC9023850 | biostudies-literature
Project description:Objective: To develop and validate a pragmatic risk score to predict mortality in patients admitted to hospital with coronavirus disease 2019 (covid-19).Design: Prospective observational cohort study.Setting: International Severe Acute Respiratory and emerging Infections Consortium (ISARIC) World Health Organization (WHO) Clinical Characterisation Protocol UK (CCP-UK) study (performed by the ISARIC Coronavirus Clinical Characterisation Consortium-ISARIC-4C) in 260 hospitals across England, Scotland, and Wales. Model training was performed on a cohort of patients recruited between 6 February and 20 May 2020, with validation conducted on a second cohort of patients recruited after model development between 21 May and 29 June 2020. PARTICIPANTS: Adults (age ?18 years) admitted to hospital with covid-19 at least four weeks before final data extraction.Main outcome measure: In-hospital mortality.Results: 35?463 patients were included in the derivation dataset (mortality rate 32.2%) and 22?361 in the validation dataset (mortality rate 30.1%). The final 4C Mortality Score included eight variables readily available at initial hospital assessment: age, sex, number of comorbidities, respiratory rate, peripheral oxygen saturation, level of consciousness, urea level, and C reactive protein (score range 0-21 points). The 4C Score showed high discrimination for mortality (derivation cohort: area under the receiver operating characteristic curve 0.79, 95% confidence interval 0.78 to 0.79; validation cohort: 0.77, 0.76 to 0.77) with excellent calibration (validation: calibration-in-the-large=0, slope=1.0). Patients with a score of at least 15 (n=4158, 19%) had a 62% mortality (positive predictive value 62%) compared with 1% mortality for those with a score of 3 or less (n=1650, 7%; negative predictive value 99%). Discriminatory performance was higher than 15 pre-existing risk stratification scores (area under the receiver operating characteristic curve range 0.61-0.76), with scores developed in other covid-19 cohorts often performing poorly (range 0.63-0.73).Conclusions: An easy-to-use risk stratification score has been developed and validated based on commonly available parameters at hospital presentation. The 4C Mortality Score outperformed existing scores, showed utility to directly inform clinical decision making, and can be used to stratify patients admitted to hospital with covid-19 into different management groups. The score should be further validated to determine its applicability in other populations.Study registration: ISRCTN66726260.
| S-EPMC7116472 | biostudies-literature
Project description:ObjectivesPredictive algorithms to inform risk management decisions are needed for patients with COVID-19, although the traditional risk scores have not been adequately assessed in Asian patients. We aimed to evaluate the performance of a COVID-19-specific prediction model, the 4C (Coronavirus Clinical Characterisation Consortium) Mortality Score, along with other conventional critical care risk models in Japanese nationwide registry data.DesignRetrospective cohort study.Setting and participantsHospitalised patients with COVID-19 and cardiovascular disease or coronary risk factors from January to May 2020 in 49 hospitals in Japan.Main outcome measuresTwo different types of outcomes, in-hospital mortality and a composite outcome, defined as the need for invasive mechanical ventilation and mortality.ResultsThe risk scores for 693 patients were tested by predicting in-hospital mortality for all patients and composite endpoint among those not intubated at baseline (n=659). The number of events was 108 (15.6%) for mortality and 178 (27.0%) for composite endpoints. After missing values were multiply imputed, the performance of the 4C Mortality Score was assessed and compared with three prediction models that have shown good discriminatory ability (RISE UP score, A-DROP score and the Rapid Emergency Medicine Score (REMS)). The area under the receiver operating characteristic curve (AUC) for the 4C Mortality Score was 0.84 (95% CI 0.80 to 0.88) for in-hospital mortality and 0.78 (95% CI 0.74 to 0.81) for the composite endpoint. It showed greater discriminatory ability compared with other scores, except for the RISE UP score, for predicting in-hospital mortality (AUC: 0.82, 95% CI 0.78 to 0.86). Similarly, the 4C Mortality Score showed a positive net reclassification improvement index over the A-DROP and REMS for mortality and over all three scores for the composite endpoint. The 4C Mortality Score model showed good calibration, regardless of outcome.ConclusionsThe 4C Mortality Score performed well in an independent external COVID-19 cohort and may enable appropriate disposition of patients and allocation of medical resources.Trial registration number UMIN000040598.
| S-EPMC8438580 | biostudies-literature
Project description:BackgroundIdentification of patients with novel coronavirus disease 2019 (COVID-19) requiring hospital admission or at high-risk of in-hospital mortality is essential to guide patient triage and to provide timely treatment for higher risk hospitalized patients.MethodsA retrospective multi-centre (8 hospital) cohort at Beaumont Health, Michigan, USA, reporting on COVID-19 patients diagnosed between 1 March and 1 April 2020 was used for score validation. The COVID-19 Risk of Complications Score was automatically computed by the EHR. Multivariate logistic regression models were built to predict hospital admission and in-hospital mortality using individual variables constituting the score. Validation was performed using both discrimination and calibration.ResultsCompared to Green scores, Yellow Scores (OR: 5.72) and Red Scores (OR: 19.1) had significantly higher odds of admission (both p < .0001). Similarly, Yellow Scores (OR: 4.73) and Red Scores (OR: 13.3) had significantly higher odds of in-hospital mortality than Green Scores (both p < .0001). The cross-validated C-Statistics for the external validation cohort showed good discrimination for both hospital admission (C = 0.79 (95% CI: 0.77-0.81)) and in-hospital mortality (C = 0.75 (95% CI: 0.71-0.78)).ConclusionsThe COVID-19 Risk of Complications Score predicts the need for hospital admission and in-hospital mortality patients with COVID-19. Key points: Can an electronic health record generated risk score predict the risk of hospital admission and in-hospital mortality in patients diagnosed with coronavirus disease 2019 (COVID-19)? In both validation cohorts of 2,025 and 1,290 COVID-19, the cross-validated C-Statistics showed good discrimination for both hospital admission (C = 0.79 (95% CI: 0.77-0.81)) and in-hospital mortality (C = 0.75 (95% CI: 0.71-0.78)), respectively. The COVID-19 Risk of Complications Score may help predict the need for hospital admission if a patient contracts SARS-CoV-2 infection and in-hospital mortality for a hospitalized patient with COVID-19.
| S-EPMC7877986 | biostudies-literature
Project description:BackgroundThe Hospital Frailty Risk Score (HFRS) has made it possible internationally to identify subgroups of patients with characteristics of frailty from routinely collected hospital data.ObjectiveTo externally validate the HFRS in France.DesignA retrospective analysis of the French medical information database.Setting743 hospitals in Metropolitan France.SubjectsAll patients aged 75 years or older hospitalised as an emergency in 2017 (n = 1,042,234).MethodsThe HFRS was calculated for each patient based on the index stay and hospitalisations over the preceding 2 years. Main outcome measures were 30-day in-patient mortality, length of stay (LOS) >10 days and 30-day readmissions. Mixed logistic regression models were used to investigate the association between outcomes and HFRS score.ResultsPatients with high HFRS risk were associated with increased risk of mortality and prolonged LOS (adjusted odds ratio [aOR] = 1.38 [1.35-1.42] and 3.27 [3.22-3.32], c-statistics = 0.676 and 0.684, respectively), while it appeared less predictive of readmissions (aOR = 1.00 [0.98-1.02], c-statistic = 0.600). Model calibration was excellent. Restricting the score to data prior to index admission reduced discrimination of HFRS substantially.ConclusionsHFRS can be used in France to determine risks of 30-day in-patient mortality and prolonged LOS, but not 30-day readmissions. Trial registration: Reference ID on clinicaltrials.gov: ID: NCT03905629.
| S-EPMC8753041 | biostudies-literature
Project description:ObjectiveTo examine the relationship between multimorbidity and mortality, and whether relationship varied by material deprivation/rural location and by age.MethodsRetrospective population-based cohort study conducted using 2013-14 data from previously created cohort of Ontario, Canada residents classified according to whether or not they had multimorbidity, defined as having 3+ of 17 chronic conditions. Adjusted rate ratios were calculated to compare mortality rates for those with and without multimorbidity, comparing rates by material deprivation/rural location, and by age group.ResultsThere were 13,581,191 people in the cohort ages 0 to 105 years; 15.2% had multimorbidity. Median length of observation was 365 days. Adjusted mortality rate ratios did not vary by material deprivation/rural location; overall adjusted mortality rate ratio was 2.41 (95% CI 2.37-2.45). Adjusted mortality rate ratios varied by age with ratios decreasing as age increased. Overall rate ratio was 14.7 (95% CI 14.48-14.91). Children (0-17 years) had highest ratio, 40.06 (95% CI 26.21-61.22). Youngest adult age group (18-24 years) had rate ratio of 9.96 (95% CI 7.18-13.84); oldest age group (80+ years) had rate ratio of 1.97 (95% CI 1.94-2.04).ConclusionCompared to people without multimorbidity, multimorbidity conferred higher risk of death in this study at all age groups. Risk was greater in early and middle adulthood than in older ages. Results reinforce the fact multimorbidity is not just a problem of aging, and multimorbidity leads not only to poorer health and higher health care utilization, but also to a higher risk of death at a younger age.
| S-EPMC7457707 | biostudies-literature
Project description:AbstractAs severe acute respiratory syndrome coronavirus 2 continues to spread, easy-to-use risk models that predict hospital mortality can assist in clinical decision making and triage. We aimed to develop a risk score model for in-hospital mortality in patients hospitalized with 2019 novel coronavirus (COVID-19) that was robust across hospitals and used clinical factors that are readily available and measured standardly across hospitals.In this retrospective observational study, we developed a risk score model using data collected by trained abstractors for patients in 20 diverse hospitals across the state of Michigan (Mi-COVID19) who were discharged between March 5, 2020 and August 14, 2020. Patients who tested positive for severe acute respiratory syndrome coronavirus 2 during hospitalization or were discharged with an ICD-10 code for COVID-19 (U07.1) were included. We employed an iterative forward selection approach to consider the inclusion of 145 potential risk factors available at hospital presentation. Model performance was externally validated with patients from 19 hospitals in the Mi-COVID19 registry not used in model development. We shared the model in an easy-to-use online application that allows the user to predict in-hospital mortality risk for a patient if they have any subset of the variables in the final model.Two thousand one hundred and ninety-three patients in the Mi-COVID19 registry met our inclusion criteria. The derivation and validation sets ultimately included 1690 and 398 patients, respectively, with mortality rates of 19.6% and 18.6%, respectively. The average age of participants in the study after exclusions was 64 years old, and the participants were 48% female, 49% Black, and 87% non-Hispanic. Our final model includes the patient's age, first recorded respiratory rate, first recorded pulse oximetry, highest creatinine level on day of presentation, and hospital's COVID-19 mortality rate. No other factors showed sufficient incremental model improvement to warrant inclusion. The area under the receiver operating characteristics curve for the derivation and validation sets were .796 (95% confidence interval, .767-.826) and .829 (95% confidence interval, .782-.876) respectively.We conclude that the risk of in-hospital mortality in COVID-19 patients can be reliably estimated using a few factors, which are standardly measured and available to physicians very early in a hospital encounter.
| S-EPMC8500634 | biostudies-literature
Project description:PURPOSE:To externally validate an intensive care unit (ICU) mortality prediction model that was created using the Ontario Critical Care Information System (CCIS), which includes the Multiple Organ Dysfunction Score (MODS). METHODS:We applied the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) recommendations to a prospective longitudinal cohort of patients discharged between 1 July 2015 and 31 December 31 2016 from 90 adult level-3 critical care units in Ontario. We used multivariable logistic regression with measures of discrimination, calibration-in-the-large, calibration slope, and flexible calibration plots to compare prediction model performance of the entire data set and for each ICU subtype. RESULTS:Among 121,201 CCIS records with ICU mortality of 11.3%, the C-statistic for the validation data set was 0.805. The C-statistic ranged from 0.775 to 0.846 among the ICU subtypes. After intercept recalibration to adjust the baseline risk, the mean predicted risk of death matched actual ICU mortality. The calibration slope was close to 1 with all CCIS data and ICU subtypes of cardiovascular and community hospitals with low ventilation rates. Calibration slopes significantly less than 1 were found for ICUs in teaching hospitals and community hospitals with high ventilation rates whereas coronary care units had a calibration slope significantly higher than 1. Calibration plots revealed over-prediction in high risk groups to a varying degree across all cohorts. CONCLUSIONS:A risk prediction model primarily based on the MODS shows reproducibility and transportability after intercept recalibration. Risk adjusting models that use existing and feasible data collection can support performance measurement at the individual ICU level.
| S-EPMC7223438 | biostudies-literature