Dataset Information

External validation of four dementia prediction models for use in the general community-dwelling population: a comparative analysis from the Rotterdam Study.

ABSTRACT: To systematically review the literature for dementia prediction models for use in the general population and externally validate their performance in a head-to-head comparison. We selected four prediction models for validation: CAIDE, BDSI, ANU-ADRI and DRS. From the Rotterdam Study, 6667 non-demented individuals aged 55 years and older were assessed between 1997 and 2001. Subsequently, participants were followed for dementia until 1 January, 2015. For each individual, we computed the risk of dementia using the reported scores from each prediction model. We used the C-statistic and calibration plots to assess the performance of each model to predict 10-year risk of all-cause dementia. For comparisons, we also evaluated discriminative accuracy using only the age component of these risk scores for each model separately. During 75,581 person-years of follow-up, 867 participants developed dementia. C-statistics for 10-year dementia risk prediction were 0.55 (95% CI 0.53-0.58) for CAIDE, 0.78 (0.76-0.81) for BDSI, 0.75 (0.74-0.77) for ANU-ADRI, and 0.81 (0.78-0.83) for DRS. Calibration plots showed that predicted risks were too extreme with underestimation at low risk and overestimation at high risk. Importantly, in all models age alone already showed nearly identical discriminative accuracy as the full model (C-statistics: 0.55 (0.53-0.58) for CAIDE, 0.81 (0.78-0.83) for BDSI, 0.77 (0.75-0.79) for ANU-ADRI, and 0.81 (0.78-0.83) for DRS). In this study, we found high variability in discriminative ability for predicting dementia in an elderly, community-dwelling population. All models showed similar discriminative ability when compared to prediction based on age alone. These findings highlight the urgent need for updated or new models to predict dementia risk in the general population.

SUBMITTER: Licher S

PROVIDER: S-EPMC6061119 | biostudies-literature | 2018 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

External validation of four dementia prediction models for use in the general community-dwelling population: a comparative analysis from the Rotterdam Study.

Licher Silvan S Yilmaz Pınar P Leening Maarten J G MJG Wolters Frank J FJ Vernooij Meike W MW Stephan Blossom C M BCM Ikram M Kamran MK Ikram M Arfan MA

European journal of epidemiology 20180508 7

To systematically review the literature for dementia prediction models for use in the general population and externally validate their performance in a head-to-head comparison. We selected four prediction models for validation: CAIDE, BDSI, ANU-ADRI and DRS. From the Rotterdam Study, 6667 non-demented individuals aged 55 years and older were assessed between 1997 and 2001. Subsequently, participants were followed for dementia until 1 January, 2015. For each individual, we computed the risk of de ...[more]

PMID: 29740780

Similar Datasets

Project description:ImportanceEstimating mortality risk in older adults with dementia is important for guiding decisions such as cancer screening, treatment of new and chronic medical conditions, and advance care planning.ObjectiveTo develop and externally validate a mortality prediction model in community-dwelling older adults with dementia.Design, setting, and participantsThis cohort study included community-dwelling participants (aged ≥65 years) in the Health and Retirement Study (HRS) from 1998 to 2016 (derivation cohort) and National Health and Aging Trends Study (NHATS) from 2011 to 2019 (validation cohort).ExposuresCandidate predictors included demographics, behavioral/health factors, functional measures (eg, activities of daily living [ADL] and instrumental activities of daily living [IADL]), and chronic conditions.Main outcomes and measuresThe primary outcome was time to all-cause death. We used Cox proportional hazards regression with backward selection and multiple imputation for model development. Model performance was assessed by discrimination (integrated area under the receiver operating characteristic curve [iAUC]) and calibration (plots of predicted and observed mortality).ResultsOf 4267 participants with probable dementia in HRS, the mean (SD) age was 82.2 (7.6) years, 2930 (survey-weighted 69.4%) were female, and 785 (survey-weighted 12.1%) identified as Black. Median (IQR) follow-up time was 3.9 (2.0-6.8) years, and 3466 (81.2%) participants died by end of follow-up. The final model included age, sex, body mass index, smoking status, ADL dependency count, IADL difficulty count, difficulty walking several blocks, participation in vigorous physical activity, and chronic conditions (cancer, heart disease, diabetes, lung disease). The optimism-corrected iAUC after bootstrap internal validation was 0.76 (95% CI, 0.75-0.76) with time-specific AUC of 0.73 (95% CI, 0.70-0.75) at 1 year, 0.75 (95% CI, 0.73-0.77) at 5 years, and 0.84 (95% CI, 0.82-0.85) at 10 years. On external validation in NHATS (n = 2404), AUC was 0.73 (95% CI, 0.70-0.76) at 1 year and 0.74 (95% CI, 0.71-0.76) at 5 years. Calibration plots suggested good calibration across the range of predicted risk from 1 to 10 years.Conclusions and relevanceWe developed and externally validated a mortality prediction model in community-dwelling older adults with dementia that showed good discrimination and calibration. The mortality risk estimates may help guide discussions regarding treatment decisions and advance care planning.

Project description:BackgroundMany dementia prediction models have been developed, but only few have been externally validated, which hinders clinical uptake and may pose a risk if models are applied to actual patients regardless. Externally validating an existing prediction model is a difficult task, where we mostly rely on the completeness of model reporting in a published article. In this study, we aim to externally validate existing dementia prediction models. To that end, we define model reporting criteria, review published studies, and externally validate three well reported models using routinely collected health data from administrative claims and electronic health records.MethodsWe identified dementia prediction models that were developed between 2011 and 2020 and assessed if they could be externally validated given a set of model criteria. In addition, we externally validated three of these models (Walters' Dementia Risk Score, Mehta's RxDx-Dementia Risk Index, and Nori's ADRD dementia prediction model) on a network of six observational health databases from the United States, United Kingdom, Germany and the Netherlands, including the original development databases of the models.ResultsWe reviewed 59 dementia prediction models. All models reported the prediction method, development database, and target and outcome definitions. Less frequently reported by these 59 prediction models were predictor definitions (52 models) including the time window in which a predictor is assessed (21 models), predictor coefficients (20 models), and the time-at-risk (42 models). The validation of the model by Walters (development c-statistic: 0.84) showed moderate transportability (0.67-0.76 c-statistic). The Mehta model (development c-statistic: 0.81) transported well to some of the external databases (0.69-0.79 c-statistic). The Nori model (development AUROC: 0.69) transported well (0.62-0.68 AUROC) but performed modestly overall. Recalibration showed improvements for the Walters and Nori models, while recalibration could not be assessed for the Mehta model due to unreported baseline hazard.ConclusionWe observed that reporting is mostly insufficient to fully externally validate published dementia prediction models, and therefore, it is uncertain how well these models would work in other clinical settings. We emphasize the importance of following established guidelines for reporting clinical prediction models. We recommend that reporting should be more explicit and have external validation in mind if the model is meant to be applied in different settings.

Project description:BackgroundEstablished clinical assessments for detecting dementia risk often require time, cost, and face-to-face meetings. We aimed to develop a Simplified Telephone Assessment for Dementia risk (STAD) (a new screening tool utilizing telephonic interviews to predict dementia risk) and examine the predictive validity of the STAD for the incidence of dementia.MethodsWe developed STAD based on a combination of literature review, statistical analysis, and expert opinion. We selected 12 binary questions on subjective cognitive complaints, depressive symptoms, and lifestyle activities. In the validation study, we used STAD for 4298 community-dwelling older adults and observed the incidence of dementia during the 24-month follow-up period. The total score of STAD ranging from 0 to 12 was calculated, and the cut-off point for dementia incidence was determined using the Youden index. The survival rate of dementia incidence according to the cut-off points was determined. Furthermore, we used a decision-tree model (classification and regression tree, CART) to enhance the predictive ability of STAD for dementia risk screening.ResultsThe cut-off point of STAD was set at 4/5. Participants scoring ≥ 5 points showed a significantly higher risk of dementia than those scoring ≤ 4 points, even after adjusting for covariates (hazard ratio [95% confidence interval], 2.67 [1.40-5.08]). A decision tree model using the CART algorithm was constructed using 12 nodes with three STAD items. It showed better performance for dementia prediction in terms of accuracy and specificity as compared to the logistic regression model, although its sensitivity was worse than the logistic regression model.ConclusionsWe developed a 12-item questionnaire, STAD, as a screening tool to predict dementia risk utilizing telephonic interviews and confirmed its predictive validity. Our findings might provide useful information for early screening of dementia risk and enable bridging between community and clinical settings. Additionally, STAD could be employed without face-to-face meetings in a short time; therefore, it may be a suitable screening tool for community-dwelling older adults who have negative attitudes toward clinical examination or are non-adherent to follow-up assessments in clinical trials.

Project description:Background and objectivesThe International IgA Nephropathy Network recently developed and externally validated two models to predict the risk of progression of IgA nephropathy: full models without and with race. This study sought to externally validate the International IgA Nephropathy Prediction Tool in a large, independent, and contemporary cohort in China.Design, setting, participants, & measurementsWe included 1373 patients with biopsy-confirmed primary IgA nephropathy from The First Affiliated Hospital of Zhengzhou University from January 2012 to May 2018 and calculated predicted risks for each patient. The outcomes of interest were a 50% decline in eGFR or kidney failure. We assessed the performance of both models using discrimination (concordance statistics and Kaplan-Meier curves between subgroups), calibration (calibration plots), reclassification (net reclassification improvement and integrated discrimination improvement), and clinical utility (decision curve analysis).ResultsThe median follow-up was 29 months (interquartile range, 21-43 months; range, 1-95 months), and 186 (14%) patients reached the kidney outcomes of interest. Both models showed excellent discrimination (concordance statistics >0.85 and well separated survival curves). Overall, the full model without race generally underestimated the risk of primary outcome, whereas the full model with race was well calibrated for predicting 5-year risk. Compared with the full model without race, the full model with race had significant improvement in reclassification, as assessed by the net reclassification improvement (0.49; 95% confidence interval, 0.41 to 0.59) and integrated discrimination improvement (0.06; 95% confidence interval, 0.04 to 0.08). Decision curve analysis showed that both full models had a higher net benefit than default strategies, and the model with race performed better.ConclusionsIn this study, both full models demonstrated remarkable discrimination, acceptable calibration, and satisfactory clinical utility. The relatively short follow-up time may have limited the validation of these models.

Dataset Information

External validation of four dementia prediction models for use in the general community-dwelling population: a comparative analysis from the Rotterdam Study.

Publications

External validation of four dementia prediction models for use in the general community-dwelling population: a comparative analysis from the Rotterdam Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets