Prediction performance and fairness heterogeneity in cardiovascular risk models.
Ontology highlight
ABSTRACT:
SUBMITTER: Kartoun U
PROVIDER: S-EPMC9307639 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Oral clefts are common birth defects that have a major impact on the affected individual, their family and society. World-wide, the incidence of oral clefts is 1/700 live births, making them the most common craniofacial birth defects. The successful prediction of oral clefts may help identify sub-population at high risk, and promote new diagnostic and therapeutic strategies. Nevertheless, developing a clinically useful oral clefts risk prediction model remains a great challenge. Compelling evidences suggest the etiologies of oral clefts are highly heterogeneous, and the development of a risk prediction model with consideration of phenotypic heterogeneity may potentially improve the accuracy of a risk prediction model. In this study, we applied a previously developed statistical method to investigate the risk prediction on sub-phenotypes of oral clefts. Our results suggested subtypes of cleft lip (CL) and palate have similar genetic etiologies (AUC = 0.572) with subtypes of CL only (AUC = 0.589), while the subtypes of cleft palate only (CPO) have heterogeneous underlying mechanisms (AUCs for soft CPO and hard CPO are 0.617 and 0.623, respectively). This highlighted the potential that the hard and soft forms of CPO have their own mechanisms despite sharing some of the genetic risk factors. Comparing with conventional methods for risk prediction modeling, our method considers phenotypic heterogeneity of a disease, which potentially improves the accuracy for predicting each sub-phenotype of oral clefts.
| S-EPMC4534829 | biostudies-literature
Project description:ImportanceExtant data on the performance of cardiovascular disease (CVD) risk score models in people living with HIV have not been synthesized.ObjectiveTo synthesize available data on the performance of the various CVD risk scores in people living with HIV.Data sourcesPubMed and Embase were searched from inception through January 31, 2021.Study selectionSelected studies (1) were chosen based on cohort design, (2) included adults with a diagnosis of HIV, (3) assessed CVD outcomes, and (4) had available data on a minimum of 1 CVD risk score.Data extraction and synthesisRelevant data related to study characteristics, CVD outcome, and risk prediction models were extracted in duplicate. Measures of calibration and discrimination are presented in tables and qualitatively summarized. Additionally, where possible, estimates of discrimination and calibration measures were combined and stratified by type of risk model.Main outcomes and measuresMeasures of calibration and discrimination.ResultsNine unique observational studies involving 75 304 people (weighted average age, 42 years; 59 490 male individuals [79%]) living with HIV were included. In the studies reporting these data, 86% were receiving antiretroviral therapy and had a weighted average CD4+ count of 449 cells/μL. Included in the study were current smokers (50%), patients with diabetes (5%), and patients with hypertension (25%). Ten risk prediction scores (6 in the general population and 4 in the HIV-specific population) were analyzed. Most risk scores had a moderate performance in discrimination (C statistic: 0.7-0.8), without a significant difference in performance between the risk scores of the general and HIV-specific populations. One of the HIV-specific risk models (Data Collection on Adverse Effects of Anti-HIV Drugs Cohort 2016) and 2 of the general population risk models (Framingham Risk Score [FRS] and Pooled Cohort Equation [PCE] 10 year) had the highest performance in discrimination. In general, models tended to underpredict CVD risk, except for FRS and PCE 10-year scores, which were better calibrated. There was substantial heterogeneity across the studies, with only a few studies contributing data for each risk score.Conclusions and relevanceResults of this systematic review and meta-analysis suggest that general population and HIV-specific CVD risk models had comparable, moderate discrimination ability in people living with HIV, with a general tendency to underpredict risk. These results reinforce the current recommendations provided by the American College of Cardiology/American Heart Association guidelines to consider HIV as a risk-enhancing factor when estimating CVD risk.
| S-EPMC9857084 | biostudies-literature
Project description:In light of the outbreak of COVID-19, analyzing and measuring human mobility has become increasingly important. A wide range of studies have explored spatiotemporal trends over time, examined associations with other variables, evaluated non-pharmacologic interventions (NPIs), and predicted or simulated COVID-19 spread using mobility data. Despite the benefits of publicly available mobility data, a key question remains unanswered: are models using mobility data performing equitably across demographic groups? We hypothesize that bias in the mobility data used to train the predictive models might lead to unfairly less accurate predictions for certain demographic groups. To test our hypothesis, we applied two mobility-based COVID infection prediction models at the county level in the United States using SafeGraph data, and correlated model performance with sociodemographic traits. Findings revealed that there is a systematic bias in models' performance toward certain demographic characteristics. Specifically, the models tend to favor large, highly educated, wealthy, young, and urban counties. We hypothesize that the mobility data currently used by many predictive models tends to capture less information about older, poorer, less educated and people from rural regions, which in turn negatively impacts the accuracy of the COVID-19 prediction in these areas. Ultimately, this study points to the need of improved data collection and sampling approaches that allow for an accurate representation of the mobility patterns across demographic groups.
| S-EPMC10584164 | biostudies-literature
Project description:BACKGROUND: Predicting cardiovascular risk is of great interest in renal transplant recipients since cardiovascular disease is the leading cause of mortality. OBJECTIVE: To conduct a systematic review to assess the validity of cardiovascular risk prediction models in this population. METHODS: Five databases were searched (MEDLINE, EMBASE, SCOPUS, CINAHL, and Web of Science) and cohort studies with at least one year of follow-up were included. Variables that described population characteristics, study design, and prognostic performance were extracted. The Quality in Prognostic Studies (QUIPS) tool was used to evaluate bias. RESULTS: Seven studies met the criteria for inclusion, of which, five investigated the Framingham risk score and three used a transplant-specific model. Sample sizes ranged from 344 to 23,575, and three studies lacked sufficient event rates to confidently reach conclusion. Four studies reported discrimination (as measured by c-statistic), which ranged from 0.701 to 0.75, while only one risk model was both internally and externally validated. CONCLUSION: The Framingham has underestimated cardiovascular events in renal transplant recipients, but these studies have not been robust. A risk prediction model has been externally validated at least on one occasion, but comprehensive validation in multiple cohorts and impact analysis are recommended before widespread clinical application is advocated.
| S-EPMC3996891 | biostudies-literature
Project description:ObjectiveTo validate the World Health Organization (WHO) non-laboratory-based cardiovascular disease risk prediction model in regions of China.MethodsWe performed an external validation of the WHO model for East Asia using the data set of China Kadoorie Biobank, an ongoing cohort study with 512 725 participants recruited from 10 regions of China from 2004-2008. We also recalculated the recalibration parameters for the WHO model in each region and evaluated the predictive performance of the model before and after recalibration. We assessed discrimination performance by Harrell's C index.FindingsWe included 412 225 participants aged 40-79 years. During a median follow-up of 11 years, 58 035 and 41 262 incident cardiovascular disease cases were recorded in women and men, respectively. Harrell's C of the WHO model was 0.682 in women and 0.700 in men but varied among regions. The WHO model underestimated the 10-year cardiovascular disease risk in most regions. After recalibration in each region, discrimination and calibration were both improved in the overall population. Harrell's C increased from 0.674 to 0.749 in women and from 0.698 to 0.753 in men. The ratios of predicted to observed cases before and after recalibration were 0.189 and 1.027 in women and 0.543 and 1.089 in men.ConclusionThe WHO model for East Asia yielded moderate discrimination for cardiovascular disease in the Chinese population and had limited prediction for cardiovascular disease risk in different regions in China. Recalibration for diverse regions greatly improved discrimination and calibration in the overall population.
| S-EPMC10042093 | biostudies-literature
Project description:Predicting the risk of cardiovascular disease is the key to primary prevention. Machine learning has attracted attention in analyzing increasingly large, complex healthcare data. We assessed discrimination and calibration of pre-existing cardiovascular risk prediction models and developed machine learning-based prediction algorithms. This study included 222,998 Korean adults aged 40-79 years, naïve to lipid-lowering therapy, had no history of cardiovascular disease. Pre-existing models showed moderate to good discrimination in predicting future cardiovascular events (C-statistics 0.70-0.80). Pooled cohort equation (PCE) specifically showed C-statistics of 0.738. Among other machine learning models such as logistic regression, treebag, random forest, and adaboost, the neural network model showed the greatest C-statistic (0.751), which was significantly higher than that for PCE. It also showed improved agreement between the predicted risk and observed outcomes (Hosmer-Lemeshow χ2 = 86.1, P < 0.001) than PCE for whites did (Hosmer-Lemeshow χ2 = 171.1, P < 0.001). Similar improvements were observed for Framingham risk score, systematic coronary risk evaluation, and QRISK3. This study demonstrated that machine learning-based algorithms could improve performance in cardiovascular risk prediction over contemporary cardiovascular risk models in statin-naïve healthy Korean adults without cardiovascular disease. The model can be easily adopted for risk assessment and clinical decision making.
| S-EPMC8076166 | biostudies-literature
Project description:The value of a developed prediction model depends on its performance outside the development sample. The key is therefore to externally validate the model on a different but related independent data. In this study, we propose a permutation method to assess heterogeneity in external validation for risk prediction models. The permutation p value measures the extent of homology between development and validation datasets. If p < 0.05, the model may not be directly transported to the external validation population without further revision or updating. Monte-Carlo simulations are conducted to evaluate the statistical properties of the proposed method, and two microarray breast cancer datasets are analyzed for demonstration. The permutation method is easy to implement and is recommended for routine use in external validation for risk prediction models.
| S-EPMC4301917 | biostudies-literature
Project description:ImportanceLong-term cumulative systolic blood pressure (SBP) is significantly associated with increased rates of atherosclerotic cardiovascular disease (ASCVD) development independent of single SBP levels. However, published ASCVD risk prediction algorithms only include currently measured SBP.ObjectiveTo determine whether including long-term (5- and 10-year) cumulative SBP in risk equations improves the predictive ability compared with single SBP measurements.Design, setting, and participantsAdults aged 45 to 65 years at the time of risk estimation with at least 20 years of follow-up (5 and 10 years prior to risk estimation and 10 years of event follow-up). The Lifetime Risk Pooling Project included data from the following cohorts: Coronary Artery Risk Development in Young Adults Study, Atherosclerosis Risk in Communities Study, and Framingham Heart Study (both the original and offspring).ExposuresTen-year ASCVD risk, calculated using the approach of the 2013 American College of Cardiology/American Heart Association 10-year ASCVD risk equations, first with current SBP and then substituting 5- and 10-year cumulative SBP levels.Main outcomes and measuresIncident ASCVD events that occurred over 10 years of follow-up, compared with the predicted risks, using the C statistic, net reclassification index at event rate, and the integrated discrimination index.ResultsThis study included 11 767 participants with a mean (SD) age of 59.1 (4.7) years at risk estimation. A total of 6873 participants (58%) were women, and 1499 (13%) were African American. In the 10 years of follow-up from risk estimation, 1887 participants (16%) had an ASCVD event. There were no significant improvements in the C statistic when including 5- or 10-year cumulative SBP. However, the addition of cumulative SBP resulted in significant improvements in the net reclassification index at event rate (10-year net reclassification index for men, 0.04 [95% CI, 0.02-0.06]; 10-year net reclassification index for women, 0.03 [95% CI, 0.01-0.06]) and the relative integrated discrimination index (10-year relative integrated discrimination index for men, 0.12; 10-year relative integrated discrimination index for women, 0.10).Conclusions and relevanceUsing long-term measures of cumulative blood pressure, instead of single measurements, can modestly improve the ability of cardiovascular disease risk prediction models to correctly classify individuals in terms of their risk for cardiovascular disease.
| S-EPMC6583053 | biostudies-literature
Project description:Objective To provide an overview of prediction models for risk of cardiovascular disease (CVD) in the general population.Design Systematic review.Data sources Medline and Embase until June 2013.Eligibility criteria for study selection Studies describing the development or external validation of a multivariable model for predicting CVD risk in the general population.Results 9965 references were screened, of which 212 articles were included in the review, describing the development of 363 prediction models and 473 external validations. Most models were developed in Europe (n=167, 46%), predicted risk of fatal or non-fatal coronary heart disease (n=118, 33%) over a 10 year period (n=209, 58%). The most common predictors were smoking (n=325, 90%) and age (n=321, 88%), and most models were sex specific (n=250, 69%). Substantial heterogeneity in predictor and outcome definitions was observed between models, and important clinical and methodological information were often missing. The prediction horizon was not specified for 49 models (13%), and for 92 (25%) crucial information was missing to enable the model to be used for individual risk prediction. Only 132 developed models (36%) were externally validated and only 70 (19%) by independent investigators. Model performance was heterogeneous and measures such as discrimination and calibration were reported for only 65% and 58% of the external validations, respectively.Conclusions There is an excess of models predicting incident CVD in the general population. The usefulness of most of the models remains unclear owing to methodological shortcomings, incomplete presentation, and lack of external validation and model impact studies. Rather than developing yet another similar CVD risk prediction model, in this era of large datasets, future research should focus on externally validating and comparing head-to-head promising CVD risk models that already exist, on tailoring or even combining these models to local settings, and investigating whether these models can be extended by addition of new predictors.
| S-EPMC4868251 | biostudies-literature
Project description:BackgroundAtrial fibrillation (AF) is associated with increased risks of stroke and heart failure. Electronic health record (EHR)-based AF risk prediction may facilitate efficient deployment of interventions to diagnose or prevent AF altogether.MethodsWe externally validated an electronic health record AF (EHR-AF) score in IBM Explorys Life Sciences, a multi-institutional dataset containing statistically deidentified EHR data for over 21 million individuals (Explorys Dataset). We included individuals with complete AF risk data, ≥2 office visits within 2 years, and no prevalent AF. We compared EHR-AF to existing scores including CHARGE-AF (Cohorts for Heart and Aging Research in Genomic Epidemiology Atrial Fibrillation), C2HEST (coronary artery disease or chronic obstructive pulmonary disease, hypertension, elderly, systolic heart failure, thyroid disease), and CHA2DS2-VASc. We assessed association between AF risk scores and 5-year incident AF, stroke, and heart failure using Cox proportional hazards modeling, 5-year AF discrimination using C indices, and calibration of predicted AF risk to observed AF incidence.ResultsOf 21 825 853 individuals in the Explorys Dataset, 4 508 180 comprised the analysis (age 62.5, 56.3% female). AF risk scores were strongly associated with 5-year incident AF (hazard ratio per SD increase 1.85 using CHA2DS2-VASc to 2.88 using EHR-AF), stroke (1.61 using C2HEST to 1.92 using CHARGE-AF), and heart failure (1.91 using CHA2DS2-VASc to 2.58 using EHR-AF). EHR-AF (C index, 0.808 [95% CI, 0.807-0.809]) demonstrated favorable AF discrimination compared to CHARGE-AF (0.806 [95% CI, 0.805-0.807]), C2HEST (0.683 [95% CI, 0.682-0.684]), and CHA2DS2-VASc (0.720 [95% CI, 0.719-0.722]). Of the scores, EHR-AF demonstrated the best calibration to incident AF (calibration slope, 1.002 [95% CI, 0.997-1.007]). In subgroup analyses, AF discrimination using EHR-AF was lower in individuals with stroke (C index, 0.696 [95% CI, 0.692-0.700]) and heart failure (0.621 [95% CI, 0.617-0.625]).ConclusionsEHR-AF demonstrates predictive accuracy for incident AF using readily ascertained EHR data. AF risk is associated with incident stroke and heart failure. Use of such risk scores may facilitate decision support and population health management efforts focused on minimizing AF-related morbidity.
| S-EPMC7856013 | biostudies-literature