Dataset Information

An external validation of the QCovid risk prediction algorithm for risk of mortality from COVID-19 in adults: a national validation cohort study in England.

ABSTRACT:

Background

Public policy measures and clinical risk assessments relevant to COVID-19 need to be aided by risk prediction models that are rigorously developed and validated. We aimed to externally validate a risk prediction algorithm (QCovid) to estimate mortality outcomes from COVID-19 in adults in England.

Methods

We did a population-based cohort study using the UK Office for National Statistics Public Health Linked Data Asset, a cohort of individuals aged 19-100 years, based on the 2011 census and linked to Hospital Episode Statistics, the General Practice Extraction Service data for pandemic planning and research, and radiotherapy and systemic chemotherapy records. The primary outcome was time to COVID-19 death, defined as confirmed or suspected COVID-19 death as per death certification. Two periods were used: (1) Jan 24 to April 30, 2020, and (2) May 1 to July 28, 2020. We assessed the performance of the QCovid algorithms using measures of discrimination and calibration. Using predicted 90-day risk of COVID-19 death, we calculated r² values, Brier scores, and measures of discrimination and calibration with corresponding 95% CIs over the two time periods.

Findings

We included 34 897 648 adults aged 19-100 years resident in England. 26 985 (0·08%) COVID-19 deaths occurred during the first period and 13 177 (0·04%) during the second. The algorithms had good discrimination and calibration in both periods. In the first period, they explained 77·1% (95% CI 76·9-77·4) of the variation in time to death in men and 76·3% (76·0-76·6) in women. The D statistic was 3·761 (3·732-3·789) for men and 3·671 (3·640-3·702) for women and Harrell's C was 0·935 (0·933-0·937) for men and 0·945 (0·943-0·947) for women. Similar results were obtained for the second time period. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths in the first period was 65·94% for men and 71·67% for women.

Interpretation

The QCovid population-based risk algorithm performed well, showing high levels of discrimination for COVID-19 deaths in men and women for both time periods. QCovid has the potential to be dynamically updated as the pandemic evolves and, therefore, has potential use in guiding national policy.

Funding

UK National Institute for Health Research.

SUBMITTER: Nafilyan V

PROVIDER: S-EPMC8148652 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectiveTo derive and validate a risk prediction algorithm to estimate hospital admission and mortality outcomes from coronavirus disease 2019 (covid-19) in adults.DesignPopulation based cohort study.Setting and participantsQResearch database, comprising 1205 general practices in England with linkage to covid-19 test results, Hospital Episode Statistics, and death registry data. 6.08 million adults aged 19-100 years were included in the derivation dataset and 2.17 million in the validation dataset. The derivation and first validation cohort period was 24 January 2020 to 30 April 2020. The second temporal validation cohort covered the period 1 May 2020 to 30 June 2020.Main outcome measuresThe primary outcome was time to death from covid-19, defined as death due to confirmed or suspected covid-19 as per the death certification or death occurring in a person with confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection in the period 24 January to 30 April 2020. The secondary outcome was time to hospital admission with confirmed SARS-CoV-2 infection. Models were fitted in the derivation cohort to derive risk equations using a range of predictor variables. Performance, including measures of discrimination and calibration, was evaluated in each validation time period.Results4384 deaths from covid-19 occurred in the derivation cohort during follow-up and 1722 in the first validation cohort period and 621 in the second validation cohort period. The final risk algorithms included age, ethnicity, deprivation, body mass index, and a range of comorbidities. The algorithm had good calibration in the first validation cohort. For deaths from covid-19 in men, it explained 73.1% (95% confidence interval 71.9% to 74.3%) of the variation in time to death (R2); the D statistic was 3.37 (95% confidence interval 3.27 to 3.47), and Harrell's C was 0.928 (0.919 to 0.938). Similar results were obtained for women, for both outcomes, and in both time periods. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths within 97 days was 75.7%. People in the top 20% of predicted risk of death accounted for 94% of all deaths from covid-19.ConclusionThe QCOVID population based risk algorithm performed well, showing very high levels of discrimination for deaths and hospital admissions due to covid-19. The absolute risks presented, however, will change over time in line with the prevailing SARS-C0V-2 infection rate and the extent of social distancing measures in place, so they should be interpreted with caution. The model can be recalibrated for different time periods, however, and has the potential to be dynamically updated as the pandemic evolves.

Project description:Prediction of lifetime cardiovascular disease (CVD) risk is recommended in many clinical guidelines, but lifetime risk models are rarely externally validated. The aim of this study was to externally validate the QRiskLifetime incident CVD risk prediction tool. Independent external validation of QRiskLifetime using Clinical Practice Research Datalink data, examining discrimination and calibration in the whole population and stratified by age, and reclassification compared to QRISK3. Since lifetime CVD risk is unobservable, performance was evaluated at 10-years' follow-up, and lifetime performance inferred in terms of performance for in the different age-groups from which lifetime predictions are derived. One million, two hundreds sixty thousand and three hundreds twenty nine women and 1,223,265 men were included in the analysis. Discrimination was excellent in the whole population (Harrell's-C = 0.844 in women, 0.808 in men), but moderate to poor stratified by age-group (Harrell's C in people aged 30-44 0.714 for both men and women, in people aged 75-84 0.578 in women and 0.556 in men). Ten-year CVD risk was under-predicted in the whole population, and in all age-groups except women aged 45-64, with worse under-prediction in older age-groups. Compared to those at highest QRISK3 estimated 10-year risk, those with highest lifetime risk were younger (mean age: women 50.5 vs. 71.3 years; men 46.3 vs. 63.8 years) and had lower systolic blood pressure and prevalence of treated hypertension, but had more family history of premature CVD, and were more commonly minority ethnic. Over 10-years, the estimated number needed to treat (NNT) with a statin to prevent one CVD event in people with QRISK3 ≥ 10% was 34 in women and 37 in men, compared to 99 and 100 for those at highest lifetime risk. QRiskLifetime underpredicts 10-year CVD risk in nearly all age-groups, so is likely to also underpredict lifetime risk. Treatment based on lifetime risk has considerably lower medium-term benefit than treatment based on 10-year risk.

Project description:ImportanceEstimating mortality risk in older adults with dementia is important for guiding decisions such as cancer screening, treatment of new and chronic medical conditions, and advance care planning.ObjectiveTo develop and externally validate a mortality prediction model in community-dwelling older adults with dementia.Design, setting, and participantsThis cohort study included community-dwelling participants (aged ≥65 years) in the Health and Retirement Study (HRS) from 1998 to 2016 (derivation cohort) and National Health and Aging Trends Study (NHATS) from 2011 to 2019 (validation cohort).ExposuresCandidate predictors included demographics, behavioral/health factors, functional measures (eg, activities of daily living [ADL] and instrumental activities of daily living [IADL]), and chronic conditions.Main outcomes and measuresThe primary outcome was time to all-cause death. We used Cox proportional hazards regression with backward selection and multiple imputation for model development. Model performance was assessed by discrimination (integrated area under the receiver operating characteristic curve [iAUC]) and calibration (plots of predicted and observed mortality).ResultsOf 4267 participants with probable dementia in HRS, the mean (SD) age was 82.2 (7.6) years, 2930 (survey-weighted 69.4%) were female, and 785 (survey-weighted 12.1%) identified as Black. Median (IQR) follow-up time was 3.9 (2.0-6.8) years, and 3466 (81.2%) participants died by end of follow-up. The final model included age, sex, body mass index, smoking status, ADL dependency count, IADL difficulty count, difficulty walking several blocks, participation in vigorous physical activity, and chronic conditions (cancer, heart disease, diabetes, lung disease). The optimism-corrected iAUC after bootstrap internal validation was 0.76 (95% CI, 0.75-0.76) with time-specific AUC of 0.73 (95% CI, 0.70-0.75) at 1 year, 0.75 (95% CI, 0.73-0.77) at 5 years, and 0.84 (95% CI, 0.82-0.85) at 10 years. On external validation in NHATS (n = 2404), AUC was 0.73 (95% CI, 0.70-0.76) at 1 year and 0.74 (95% CI, 0.71-0.76) at 5 years. Calibration plots suggested good calibration across the range of predicted risk from 1 to 10 years.Conclusions and relevanceWe developed and externally validated a mortality prediction model in community-dwelling older adults with dementia that showed good discrimination and calibration. The mortality risk estimates may help guide discussions regarding treatment decisions and advance care planning.

Project description:BackgroundYoung people with psychosis are at high risk of developing cardiometabolic disorders; however, there is no suitable cardiometabolic risk prediction algorithm for this group. We aimed to develop and externally validate a cardiometabolic risk prediction algorithm for young people with psychosis.MethodsWe developed the Psychosis Metabolic Risk Calculator (PsyMetRiC) to predict up to 6-year risk of incident metabolic syndrome in young people (aged 16-35 years) with psychosis from commonly recorded information at baseline. We developed two PsyMetRiC versions using the forced entry method: a full model (including age, sex, ethnicity, body-mass index, smoking status, prescription of a metabolically active antipsychotic medication, HDL concentration, and triglyceride concentration) and a partial model excluding biochemical results. PsyMetRiC was developed using data from two UK psychosis early intervention services (Jan 1, 2013, to Nov 4, 2020) and externally validated in another UK early intervention service (Jan 1, 2012, to June 3, 2020). A sensitivity analysis was done in UK birth cohort participants (aged 18 years) who were at risk of developing psychosis. Algorithm performance was assessed primarily via discrimination (C statistic) and calibration (calibration plots). We did a decision curve analysis and produced an online data-visualisation app.Findings651 patients were included in the development samples, 510 in the validation sample, and 505 in the sensitivity analysis sample. PsyMetRiC performed well at internal (full model: C 0·80, 95% CI 0·74-0·86; partial model: 0·79, 0·73-0·84) and external validation (full model: 0·75, 0·69-0·80; and partial model: 0·74, 0·67-0·79). Calibration of the full model was good, but there was evidence of slight miscalibration of the partial model. At a cutoff score of 0·18, in the full model PsyMetRiC improved net benefit by 7·95% (sensitivity 75%, 95% CI 66-82; specificity 74%, 71-78), equivalent to detecting an additional 47% of metabolic syndrome cases.InterpretationWe have developed an age-appropriate algorithm to predict the risk of incident metabolic syndrome, a precursor of cardiometabolic morbidity and mortality, in young people with psychosis. PsyMetRiC has the potential to become a valuable resource for early intervention service clinicians and could enable personalised, informed health-care decisions regarding choice of antipsychotic medication and lifestyle interventions.FundingNational Institute for Health Research and Wellcome Trust.