Correction: External validation of risk prediction models for incident colorectal cancer using UK Biobank.
Ontology highlight
ABSTRACT:
SUBMITTER: Usher-Smith JA
PROVIDER: S-EPMC7217758 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundThis study aimed to compare and externally validate risk scores developed to predict incident colorectal cancer (CRC) that include variables routinely available or easily obtainable via self-completed questionnaire.MethodsExternal validation of fourteen risk models from a previous systematic review in 373 112 men and women within the UK Biobank cohort with 5-year follow-up, no prior history of CRC and data for incidence of CRC through linkage to national cancer registries.ResultsThere were 1719 (0.46%) cases of incident CRC. The performance of the risk models varied substantially. In men, the QCancer10 model and models by Tao, Driver and Ma all had an area under the receiver operating characteristic curve (AUC) between 0.67 and 0.70. Discrimination was lower in women: the QCancer10, Wells, Tao, Guesmi and Ma models were the best performing with AUCs between 0.63 and 0.66. Assessment of calibration was possible for six models in men and women. All would require country-specific recalibration if estimates of absolute risks were to be given to individuals.ConclusionsSeveral risk models based on easily obtainable data have relatively good discrimination in a UK population. Modelling studies are now required to estimate the potential health benefits and cost-effectiveness of implementing stratified risk-based CRC screening.
| S-EPMC5846069 | biostudies-literature
Project description:The aim of this study was to compare and externally validate risk scores developed to predict incident colorectal cancer that include common genetic variants (SNPs), with or without established lifestyle/environmental (questionnaire-based/classical/phenotypic) risk factors. We externally validated 23 risk models from a previous systematic review in 443,888 participants ages 37 to 73 from the UK Biobank cohort who had 6-year prospective follow-up, no prior history of colorectal cancer, and data for incidence of colorectal cancer through linkage to national cancer registries. There were 2,679 (0.6%) cases of incident colorectal cancer. We assessed model discrimination using the area under the operating characteristic curve (AUC) and relative risk calibration. The AUC of models including only SNPs increased with the number of included SNPs and was similar in men and women: the model by Huyghe with 120 SNPs had the highest AUC of 0.62 [95% confidence interval (CI), 0.59-0.64] in women and 0.64 (95% CI, 0.61-0.66) in men. Adding phenotypic risk factors without age improved discrimination in men but not in women. Adding phenotypic risk factors and age increased discrimination in all cases (P < 0.05), with the best performing models including SNPs, phenotypic risk factors, and age having AUCs between 0.64 and 0.67 in women and 0.67 and 0.71 in men. Relative risk calibration varied substantially across the models. Among middle-aged people in the UK, existing polygenic risk scores discriminate moderately well between those who do and do not develop colorectal cancer over 6 years. Consideration should be given to exploring the feasibility of incorporating genetic and lifestyle/environmental information in any future stratified colorectal cancer screening program.
| S-EPMC7610623 | biostudies-literature
Project description:In China, colorectal cancer (CRC) incidence and mortality have been steadily increasing over the last decades. Risk models to predict incident CRC have been developed in various populations, but they have not been systematically externally validated in a Chinese population. This study aimed to assess the performance of risk scores in predicting CRC using the China Kadoorie Biobank (CKB), one of the largest and geographically diverse prospective cohort studies in China. Nine models were externally validated in 512,415 participants in CKB and included 2976 cases of CRC. Model discrimination was assessed, overall and by sex, age, site, and geographic location, using the area under the receiver operating characteristic curve (AUC). Model discrimination of these nine models was compared to a model using age alone. Calibration was assessed for five models, and they were re-calibrated in CKB. The three models with the highest discrimination (Ma (Cox model) AUC 0.70 [95% CI 0.69-0.71]; Aleksandrova 0.70 [0.69-0.71]; Hong 0.69 [0.67-0.71]) included the variables age, smoking, and alcohol. These models performed significantly better than using a model based on age alone (AUC of 0.65 [95% CI 0.64-0.66]). Model discrimination was generally higher in younger participants, males, urban environments, and for colon cancer. The two models (Guo and Chen) developed in Chinese populations did not perform better than the others. Among the 10% of participants with the highest risk, the three best performing models identified 24-26% of participants that went on to develop CRC. Several risk models based on easily obtainable demographic and modifiable lifestyle factor have good discrimination in a Chinese population. The three best performing models have a higher discrimination than using a model based on age alone.
| S-EPMC9454206 | biostudies-literature
Project description:Concordance measures are frequently used for assessing the discriminative ability of risk prediction models. The interpretation of estimated concordance at external validation is difficult if the case-mix differs from the model development setting. We aimed to develop a concordance measure that provides insight into the influence of case-mix heterogeneity and is robust to censoring of time-to-event data. We first derived a model-based concordance (mbc) measure that allows for quantification of the influence of case-mix heterogeneity on discriminative ability of proportional hazards and logistic regression models. This mbc can also be calculated including a regression slope that calibrates the predictions at external validation (c-mbc), hence assessing the influence of overall regression coefficient validity on discriminative ability. We derived variance formulas for both mbc and c-mbc. We compared the mbc and the c-mbc with commonly used concordance measures in a simulation study and in two external validation settings. The mbc was asymptotically equivalent to a previously proposed resampling-based case-mix corrected c-index. The c-mbc remained stable at the true value with increasing proportions of censoring, while Harrell's c-index and to a lesser extent Uno's concordance measure increased unfavorably. Variance estimates of mbc and c-mbc were well in agreement with the simulated empirical variances. We conclude that the mbc is an attractive closed-form measure that allows for a straightforward quantification of the expected change in a model's discriminative ability due to case-mix heterogeneity. The c-mbc also reflects regression coefficient validity and is a censoring-robust alternative for the c-index when the proportional hazards assumption holds. Copyright © 2016 John Wiley & Sons, Ltd.
| S-EPMC5550798 | biostudies-literature
Project description:BackgroundPeople with intermediate hyperglycemia (IH), including impaired fasting glucose and/or impaired glucose tolerance, are at higher risk of developing type 2 diabetes (T2D) than those with normoglycemia. We aimed to evaluate the performance of published T2D risk prediction models in Chinese people with IH to inform them about the choice of primary diabetes prevention measures.MethodsA systematic literature search was conducted to identify Asian-derived T2D risk prediction models, which were eligible if they were built on a prospective cohort of Asian adults without diabetes at baseline and utilized routinely-available variables to predict future risk of T2D. These Asian-derived and five prespecified non-Asian derived T2D risk prediction models were divided into BASIC (clinical variables only) and EXTENDED (plus laboratory variables) versions, with validation performed on them in three prospective Chinese IH cohorts: ACE (n = 3241), Luzhou (n = 1333), and TCLSIH (n = 1702). Model performance was assessed in terms of discrimination (C-statistic) and calibration (Hosmer-Lemeshow test).ResultsForty-four Asian and five non-Asian studies comprising 21 BASIC and 46 EXTENDED T2D risk prediction models for validation were identified. The majority were at high (n = 43, 87.8%) or unclear (n = 3, 6.1%) risk of bias, while only three studies (6.1%) were scored at low risk of bias. BASIC models showed poor-to-moderate discrimination with C-statistics 0.52-0.60, 0.50-0.59, and 0.50-0.64 in the ACE, Luzhou, and TCLSIH cohorts respectively. EXTENDED models showed poor-to-acceptable discrimination with C-statistics 0.54-0.73, 0.52-0.67, and 0.59-0.78 respectively. Fifteen BASIC and 40 EXTENDED models showed poor calibration (P < 0.05), overpredicting or underestimating the observed diabetes risk. Most recalibrated models showed improved calibration but modestly-to-severely overestimated diabetes risk in the three cohorts. The NAVIGATOR model showed the best discrimination in the three cohorts but had poor calibration (P < 0.05).ConclusionsIn Chinese people with IH, previously published BASIC models to predict T2D did not exhibit good discrimination or calibration. Several EXTENDED models performed better, but a robust Chinese T2D risk prediction tool in people with IH remains a major unmet need.
| S-EPMC9472437 | biostudies-literature
Project description:Colorectal cancer (CRC) risk prediction models could be used to risk-stratify the population to provide individually tailored screening provision. Using participants from the UK Biobank prospective cohort study, we evaluated whether the addition of a genetic risk score (GRS) could improve the performance of two previously validated models. Inclusion of the GRS did not appreciably improve discrimination of either model, and led to substantial miscalibration. Following recalibration the discrimination did not change, but good calibration for models incorporating the GRS was recovered. Comparing predictions between models with and without the GRS, 5% of participants or fewer changed their absolute risk by ±0.3% or more in either model. In summary, addition of a GRS did not meaningfully improve the performance of validated CRC-risk prediction models. At present, provision of genetic information is not useful for risk stratification for CRC.
| S-EPMC6203780 | biostudies-literature
Project description:The value of a developed prediction model depends on its performance outside the development sample. The key is therefore to externally validate the model on a different but related independent data. In this study, we propose a permutation method to assess heterogeneity in external validation for risk prediction models. The permutation p value measures the extent of homology between development and validation datasets. If p < 0.05, the model may not be directly transported to the external validation population without further revision or updating. Monte-Carlo simulations are conducted to evaluate the statistical properties of the proposed method, and two microarray breast cancer datasets are analyzed for demonstration. The permutation method is easy to implement and is recommended for routine use in external validation for risk prediction models.
| S-EPMC4301917 | biostudies-literature
Project description:BackgroundPatients with multiple conditions present a growing challenge for healthcare provision. Measures of multimorbidity may support clinical management, healthcare resource allocation and accounting for the health of participants in purpose-designed cohorts. The recently developed Cambridge Multimorbidity scores (CMS) have the potential to achieve these aims using primary care records, however, they have not yet been validated outside of their development cohort.MethodsThe CMS, developed in the Clinical Research Practice Dataset (CPRD), were validated in UK Biobank participants whose data is not available in CPRD (the cohort used for CMS development) with available primary care records (n = 111,898). This required mapping of the 37 pre-existing conditions used in the CMS to the coding frameworks used by UK Biobank data providers. We used calibration plots and measures of discrimination to validate the CMS for two of the three outcomes used in the development study (death and primary care consultation rate) and explored variation by age and sex. We also examined the predictive ability of the CMS for the outcome of cancer diagnosis. The results were compared to an unweighted count score of the 37 pre-existing conditions.ResultsFor all three outcomes considered, the CMS were poorly calibrated in UK Biobank. We observed a similar discriminative ability for the outcome of primary care consultation rate to that reported in the development study (C-index: 0.67 (95%CI:0.66-0.68) for both, 5-year follow-up); however, we report lower discrimination for the outcome of death than the development study (0.69 (0.68-0.70) and 0.89 (0.88-0.90) respectively). Discrimination for cancer diagnosis was adequate (0.64 (0.63-0.65)). The CMS performs favourably to the unweighted count score for death, but not for the outcomes of primary care consultation rate or cancer diagnosis.ConclusionsIn the UK Biobank, CMS discriminates reasonably for the outcomes of death, primary care consultation rate and cancer diagnosis and may be a valuable resource for clinicians, public health professionals and data scientists. However, recalibration will be required to make accurate predictions when cohort composition and risk levels differ substantially from the development cohort. The generated resources (including codelists for the conditions and code for CMS implementation in UK Biobank) are available online.
| S-EPMC10953059 | biostudies-literature
Project description:Background and purposeThe net benefit of carotid endarterectomy (CEA) is determined partly by the risk of procedural stroke or death. Current guidelines recommend CEA if 30-day risks are <6% for symptomatic stenosis and <3% for asymptomatic stenosis. We aimed to identify prediction models for procedural stroke or death after CEA and to externally validate these models in a large registry of patients from the United States.MethodsWe conducted a systematic search in MEDLINE and EMBASE for prediction models of procedural outcomes after CEA. We validated these models with data from patients who underwent CEA in the American College of Surgeons National Surgical Quality Improvement Program (2011-2017). We assessed discrimination using C statistics and calibration graphically. We determined the number of patients with predicted risks that exceeded recommended thresholds of procedural risks to perform CEA.ResultsAfter screening 788 reports, 15 studies describing 17 prediction models were included. Nine were developed in populations including both asymptomatic and symptomatic patients, 2 in symptomatic and 5 in asymptomatic populations. In the external validation cohort of 26 293 patients who underwent CEA, 702 (2.7%) developed a stroke or died within 30-days. C statistics varied between 0.52 and 0.64 using all patients, between 0.51 and 0.59 using symptomatic patients, and between 0.49 to 0.58 using asymptomatic patients. The Ontario Carotid Endarterectomy Registry model that included symptomatic status, diabetes, heart failure, and contralateral occlusion as predictors, had C statistic of 0.64 and the best concordance between predicted and observed risks. This model identified 4.5% of symptomatic and 2.1% of asymptomatic patients with procedural risks that exceeded recommended thresholds.ConclusionsOf the 17 externally validated prediction models, the Ontario Carotid Endarterectomy Registry risk model had most reliable predictions of procedural stroke or death after CEA and can inform patients about procedural hazards and help focus CEA toward patients who would benefit most from it.
| S-EPMC8712365 | biostudies-literature
Project description:[This corrects the article DOI: 10.1371/journal.pmed.1002955.].
| S-EPMC7380589 | biostudies-literature