Dataset Information

External validation and comparison of three prediction tools for risk of osteoporotic fractures using data from population based electronic health records: retrospective cohort study.

ABSTRACT: To directly compare the performance and externally validate the three most studied prediction tools for osteoporotic fractures-QFracture, FRAX, and Garvan-using data from electronic health records. Retrospective cohort study. Payer provider healthcare organisation in Israel. 1?054?815 members aged 50 to 90 years for comparison between tools and cohorts of different age ranges, corresponding to those in each tools' development study, for tool specific external validation. First diagnosis of a major osteoporotic fracture (for QFracture and FRAX tools) and hip fractures (for all three tools) recorded in electronic health records from 2010 to 2014. Observed fracture rates were compared to probabilities predicted retrospectively as of 2010. The observed five year hip fracture rate was 2.7% and the rate for major osteoporotic fractures was 7.7%. The areas under the receiver operating curve (AUC) for hip fracture prediction were 82.7% for QFracture, 81.5% for FRAX, and 77.8% for Garvan. For major osteoporotic fractures, AUCs were 71.2% for QFracture and 71.4% for FRAX. All the tools underestimated the fracture risk, but the average observed to predicted ratios and the calibration slopes of FRAX were closest to 1. Tool specific validation analyses yielded hip fracture prediction AUCs of 88.0% for QFracture (among those aged 30-100 years), 81.5% for FRAX (50-90 years), and 71.2% for Garvan (60-95 years). Both QFracture and FRAX had high discriminatory power for hip fracture prediction, with QFracture performing slightly better. This performance gap was more pronounced in previous studies, likely because of broader age inclusion criteria for QFracture validations. The simpler FRAX performed almost as well as QFracture for hip fracture prediction, and may have advantages if some of the input data required for QFracture are not available. However, both tools require calibration before implementation.

SUBMITTER: Dagan N

PROVIDER: S-EPMC5244817 | biostudies-other | 2017 Jan

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

External validation and comparison of three prediction tools for risk of osteoporotic fractures using data from population based electronic health records: retrospective cohort study.

Dagan Noa N Cohen-Stavi Chandra C Leventer-Roberts Maya M Balicer Ran D RD

BMJ (Clinical research ed.) 20170119

<h4>Objective</h4> To directly compare the performance and externally validate the three most studied prediction tools for osteoporotic fractures-QFracture, FRAX, and Garvan-using data from electronic health records.<h4>Design</h4> Retrospective cohort study.<h4>Setting</h4> Payer provider healthcare organisation in Israel.<h4>Participants</h4> 1 054 815 members aged 50 to 90 years for comparison between tools and cohorts of different age ranges, corresponding to those in each tools' development ...[more]

PMID: 28104610

Similar Datasets

Project description:BackgroundHip fracture is associated with immobility, morbidity, mortality, and high medical cost. Due to limited availability of dual-energy X-ray absorptiometry (DXA), hip fracture prediction models without using bone mineral density (BMD) data are essential. We aimed to develop and validate 10-year sex-specific hip fracture prediction models using electronic health records (EHR) without BMD.MethodsIn this retrospective, population-based cohort study, anonymized medical records were retrieved from the Clinical Data Analysis and Reporting System for public healthcare service users in Hong Kong aged ≥60 years as of 31 December 2005. A total of 161,051 individuals (91,926 female; 69,125 male) with complete follow-up from 1 January 2006 till the study end date on 31 December 2015 were included in the derivation cohort. The sex-stratified derivation cohort was randomly divided into 80% training and 20% internal testing datasets. An independent validation cohort comprised 3046 community-dwelling participants aged ≥60 years as of 31 December 2005 from the Hong Kong Osteoporosis Study, a prospective cohort which recruited participants between 1995 and 2010. With 395 potential predictors (age, diagnosis, and drug prescription records from EHR), 10-year sex-specific hip fracture prediction models were developed using stepwise selection by logistic regression (LR) and four machine learning (ML) algorithms (gradient boosting machine, random forest, eXtreme gradient boosting, and single-layer neural networks) in the training cohort. Model performance was evaluated in both internal and independent validation cohorts.FindingsIn female, the LR model had the highest AUC (0.815; 95% Confidence Interval [CI]: 0.805-0.825) and adequate calibration in internal validation. Reclassification metrics showed the LR model had better discrimination and classification performance than the ML algorithms. Similar performance was attained by the LR model in independent validation, with high AUC (0.841; 95% CI: 0.807-0.87) comparable to other ML algorithms. In internal validation for male, LR model had high AUC (0.818; 95% CI: 0.801-0.834) and it outperformed all ML models as indicated by reclassification metrics, with adequate calibration. In independent validation, the LR model had high AUC (0.898; 95% CI: 0.857-0.939) comparable to ML algorithms. Reclassification metrics demonstrated that LR model had the best discrimination performance.InterpretationEven without using BMD data, the 10-year hip fracture prediction models developed by conventional LR had better discrimination performance than the models developed by ML algorithms. Upon further validation in independent cohorts, the LR models could be integrated into the routine clinical workflow, aiding the identification of people at high risk for DXA scan.FundingHealth and Medical Research Fund, Health Bureau, Hong Kong SAR Government (reference: 17181381).

Project description:ObjectivesThis study sought to determine whether the risk of atrial fibrillation AF can be estimated accurately by using routinely ascertained features in the electronic health record (EHR) and whether AF risk is associated with stroke.BackgroundEarly diagnosis of AF and treatment with anticoagulation may prevent strokes.MethodsUsing a multi-institutional EHR, this study identified 412,085 individuals 45 to 95 years of age without prevalent AF between 2000 and 2014. A prediction model was derived and validated for 5-year AF risk by using split-sample validation and model performance was compared with other methods of AF risk assessment.ResultsWithin 5 years, 14,334 individuals developed AF. In the derivation sample (7,216 AF events of 206,042 total), the optimal risk model included sex, age, race, smoking, height, weight, diastolic blood pressure, hypertension, hyperlipidemia, heart failure, coronary heart disease, valvular disease, prior stroke, peripheral arterial disease, chronic kidney disease, hypothyroidism, and quadratic terms for height, weight, and age. In the validation sample (7,118 AF events of 206,043 total) the AF risk model demonstrated good discrimination (C-statistic: 0.777; 95% confidence interval [CI:] 0.771 to 0.783) and calibration (0.99; 95% CI: 0.96 to 1.01). Model discrimination and calibration were superior to CHARGE-AF (Cohorts for Heart and Aging Research in Genomic Epidemiology AF) (C-statistic: 0.753; 95% CI: 0.747 to 0.759; calibration slope: 0.72; 95% CI: 0.71 to 0.74), C2HEST (Coronary artery disease / chronic obstructive pulmonary disease; Hypertension; Elderly [age ≥75 years]; Systolic heart failure; Thyroid disease [hyperthyroidism]) (C-statistic: 0.754; 95% CI: 0.747 to 0.762; calibration slope: 0.44; 95% CI: 0.43 to 0.45), and CHA2DS2-VASc (Congestive heart failure, Hypertension, Age ≥75 years, Diabetes mellitus, Prior stroke, transient ischemic attack [TIA], or thromboembolism, Vascular disease, Age 65-74 years, Sex category [female]) scores (C-statistic: 0.702; 95% CI: 0.693 to 0.710; calibration slope: 0.37; 95% CI: 0.36 to 0.38). AF risk discriminated incident stroke (n = 4,814; C-statistic: 0.684; 95% CI: 0.677 to 0.692) and stroke within 90 days of incident AF (n = 327; C-statistic: 0.789; 95% CI: 0.764 to 0.814).ConclusionsA model developed from a real-world EHR database predicted AF accurately and stratified stroke risk. Incorporating AF prediction into EHRs may enable risk-guided screening for AF.

Project description:IntroductionAtrial fibrillation (AF) is a major public health issue and there is rationale for the early diagnosis of AF before the first complication occurs. Previous AF screening research is limited by low yields of new cases and strokes prevented in the screened populations. For AF screening to be clinically and cost-effective, the efficiency of identification of newly diagnosed AF needs to be improved and the intervention offered may have to extend beyond oral anticoagulation for stroke prophylaxis. Previous prediction models for incident AF have been limited by their data sources and methodologies.Methods and analysisWe will investigate the application of random forest and multivariable logistic regression to predict incident AF within a 6-month prediction horizon, that is, a time-window consistent with conducting investigation for AF. The Clinical Practice Research Datalink (CPRD)-GOLD dataset will be used for derivation, and the Clalit Health Services (CHS) dataset will be used for international external geographical validation. Analyses will include metrics of prediction performance and clinical utility. We will create Kaplan-Meier plots for individuals identified as higher and lower predicted risk of AF and derive the cumulative incidence rate for non-AF cardio-renal-metabolic diseases and death over the longer term to establish how predicted AF risk is associated with a range of new non-AF disease states.Ethics and disseminationPermission for CPRD-GOLD was obtained from CPRD (ref no: 19_076). The CPRD ethical approval committee approved the study. CHS Helsinki committee approval 21-0169 and data usage committee approval 901. The results will be submitted as a research paper for publication to a peer-reviewed journal and presented at peer-reviewed conferences.Trial registration numberA systematic review to guide the overall project was registered on PROSPERO (registration number CRD42021245093). The study was registered on ClinicalTrials.gov (NCT05837364).

Dataset Information

External validation and comparison of three prediction tools for risk of osteoporotic fractures using data from population based electronic health records: retrospective cohort study.

Publications

External validation and comparison of three prediction tools for risk of osteoporotic fractures using data from population based electronic health records: retrospective cohort study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets