Dataset Information

Performance of an electronic health record-based predictive model to identify patients with atrial fibrillation across countries.

ABSTRACT:

Background

Atrial fibrillation (AF) burden on patients and healthcare systems warrants innovative strategies for screening asymptomatic individuals.

Objective

We sought to externally validate a predictive model originally developed in a German population to detect unidentified incident AF utilising real-world primary healthcare databases from countries in Europe and Australia.

Methods

This retrospective cohort study used anonymized, longitudinal patient data from 5 country-level primary care databases, including Australia, Belgium, France, Germany, and the UK. The study eligibility included adult patients (≥45 years) with either an AF diagnosis (cases) or no diagnosis (controls) who had continuous enrolment in the respective database prior to the study period. Logistic regression was fitted to a binary response (yes/no) for AF diagnosis using pre-determined risk factors.

Results

AF patients were from Germany (n = 63,562), the UK (n = 42,652), France (n = 7,213), Australia (n = 2,753), and Belgium (n = 1,371). Cases were more likely to have hypertension or other cardiac conditions than controls in all validation datasets compared to the model development data. The area under the receiver operating characteristic (ROC) curve in the validation datasets ranged from 0.79 (Belgium) to 0.84 (Germany), comparable to the German study model, which had an area under the curve of 0.83. Most validation sets reported similar specificity at approximately 80% sensitivity, ranging from 67% (France) to 71% (United Kingdom). The positive predictive value (PPV) ranged from 2% (Belgium) to 16% (Germany), and the number needed to be screened was 50 in Belgium and 6 in Germany. The prevalence of AF varied widely between these datasets, which may be related to different coding practices. Low prevalence affected PPV, but not sensitivity, specificity, and ROC curves.

Conclusions

AF risk prediction algorithms offer targeted ways to identify patients using electronic health records, which could improve screening number and the cost-effectiveness of AF screening if implemented in clinical practice.

SUBMITTER: Mokgokong R

PROVIDER: S-EPMC9269467 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Importance:Atrial fibrillation (AF) is the most common sustained cardiac arrhythmia, and its early detection could lead to significant improvements in outcomes through the appropriate prescription of anticoagulation medication. Although a variety of methods exist for screening for AF, a targeted approach, which requires an efficient method for identifying patients at risk, would be preferred. Objective:To examine machine learning approaches applied to electronic health record data that have been harmonized to the Observational Medical Outcomes Partnership Common Data Model for identifying risk of AF. Design, Setting, and Participants:This diagnostic study used data from 2?252?219 individuals cared for in the UCHealth hospital system, which comprises 3 large hospitals in Colorado, from January 1, 2011, to October 1, 2018. Initial analysis was performed in December 2018; follow-up analysis was performed in July 2019. Exposures:All Observational Medical Outcomes Partnership Common Data Model-harmonized electronic health record features, including diagnoses, procedures, medications, age, and sex. Main Outcomes and Measures:Classification of incident AF in designated 6-month intervals, adjudicated retrospectively, based on area under the receiver operating characteristic curve and F1 statistic. Results:Of 2?252?219 individuals (1?225?533 [54.4%] women; mean [SD] age, 42.9 [22.3] years), 28?036 (1.2%) developed incident AF during a designated 6-month interval. The machine learning model that used the 200 most common electronic health record features, including age and sex, and random oversampling with a single-layer, fully connected neural network provided the optimal prediction of 6-month incident AF, with an area under the receiver operating characteristic curve of 0.800 and an F1 score of 0.110. This model performed only slightly better than a more basic logistic regression model composed of known clinical risk factors for AF, which had an area under the receiver operating characteristic curve of 0.794 and an F1 score of 0.079. Conclusions and Relevance:Machine learning approaches to electronic health record data offer a promising method for improving risk prediction for incident AF, but more work is needed to show improvement beyond standard risk factors.

Project description:BackgroundThe electronic medical record contains a wealth of information buried in free text. We created a natural language processing algorithm to identify patients with atrial fibrillation (AF) using text alone.Methods and resultsWe created 3 data sets from patients with at least one AF billing code from 2010 to 2017: a training set (n=886), an internal validation set from site no. 1 (n=285), and an external validation set from site no. 2 (n=276). A team of clinicians reviewed and adjudicated patients as AF present or absent, which served as the reference standard. We trained 54 algorithms to classify each patient, varying the model, number of features, number of stop words, and the method used to create the feature set. The algorithm with the highest F-score (the harmonic mean of sensitivity and positive predictive value) in the training set was applied to the validation sets. F-scores and area under the receiver operating characteristic curves were compared between site no. 1 and site no. 2 using bootstrapping. Adjudicated AF prevalence was 75.1% at site no. 1 and 86.2% at site no. 2. Among 54 algorithms, the best performing model was logistic regression, using 1000 features, 100 stop words, and term frequency-inverse document frequency method to create the feature set, with sensitivity 92.8%, specificity 93.9%, and an area under the receiver operating characteristic curve of 0.93 in the training set. The performance at site no. 1 was sensitivity 92.5%, specificity 88.7%, with an area under the receiver operating characteristic curve of 0.91. The performance at site no. 2 was sensitivity 89.5%, specificity 71.1%, with an area under the receiver operating characteristic curve of 0.80. The F-score was lower at site no. 2 compared with site no. 1 (92.5% [SD, 1.1%] versus 94.2% [SD, 1.1%]; P<0.001).ConclusionsWe developed a natural language processing algorithm to identify patients with AF using text alone, with >90% F-score at 2 separate sites. This approach allows better use of the clinical narrative and creates an opportunity for precise, high-throughput cohort identification.

Project description:ObjectivesThis study sought to determine whether the risk of atrial fibrillation AF can be estimated accurately by using routinely ascertained features in the electronic health record (EHR) and whether AF risk is associated with stroke.BackgroundEarly diagnosis of AF and treatment with anticoagulation may prevent strokes.MethodsUsing a multi-institutional EHR, this study identified 412,085 individuals 45 to 95 years of age without prevalent AF between 2000 and 2014. A prediction model was derived and validated for 5-year AF risk by using split-sample validation and model performance was compared with other methods of AF risk assessment.ResultsWithin 5 years, 14,334 individuals developed AF. In the derivation sample (7,216 AF events of 206,042 total), the optimal risk model included sex, age, race, smoking, height, weight, diastolic blood pressure, hypertension, hyperlipidemia, heart failure, coronary heart disease, valvular disease, prior stroke, peripheral arterial disease, chronic kidney disease, hypothyroidism, and quadratic terms for height, weight, and age. In the validation sample (7,118 AF events of 206,043 total) the AF risk model demonstrated good discrimination (C-statistic: 0.777; 95% confidence interval [CI:] 0.771 to 0.783) and calibration (0.99; 95% CI: 0.96 to 1.01). Model discrimination and calibration were superior to CHARGE-AF (Cohorts for Heart and Aging Research in Genomic Epidemiology AF) (C-statistic: 0.753; 95% CI: 0.747 to 0.759; calibration slope: 0.72; 95% CI: 0.71 to 0.74), C2HEST (Coronary artery disease / chronic obstructive pulmonary disease; Hypertension; Elderly [age ≥75 years]; Systolic heart failure; Thyroid disease [hyperthyroidism]) (C-statistic: 0.754; 95% CI: 0.747 to 0.762; calibration slope: 0.44; 95% CI: 0.43 to 0.45), and CHA2DS2-VASc (Congestive heart failure, Hypertension, Age ≥75 years, Diabetes mellitus, Prior stroke, transient ischemic attack [TIA], or thromboembolism, Vascular disease, Age 65-74 years, Sex category [female]) scores (C-statistic: 0.702; 95% CI: 0.693 to 0.710; calibration slope: 0.37; 95% CI: 0.36 to 0.38). AF risk discriminated incident stroke (n = 4,814; C-statistic: 0.684; 95% CI: 0.677 to 0.692) and stroke within 90 days of incident AF (n = 327; C-statistic: 0.789; 95% CI: 0.764 to 0.814).ConclusionsA model developed from a real-world EHR database predicted AF accurately and stratified stroke risk. Incorporating AF prediction into EHRs may enable risk-guided screening for AF.

Project description:BackgroundThe Meaningful Use (MU) program has promoted electronic health record adoption among US hospitals. Studies have shown that electronic health record adoption has been slower than desired in certain types of hospitals; but generally, the overall adoption rate has increased among hospitals. However, these studies have neither evaluated the adoption of advanced functionalities of electronic health records (beyond MU) nor forecasted electronic health record maturation over an extended period in a holistic fashion. Additional research is needed to prospectively assess US hospitals' electronic health record technology adoption and advancement patterns.ObjectiveThis study forecasts the maturation of electronic health record functionality adoption among US hospitals through 2035.MethodsThe Healthcare Information and Management Systems Society (HIMSS) Analytics' Electronic Medical Record Adoption Model (EMRAM) dataset was used to track historic uptakes of various electronic health record functionalities considered critical to improving health care quality and efficiency in hospitals. The Bass model was used to predict the technological diffusion rates for repeated electronic health record adoptions where upgrades undergo rapid technological improvements. The forecast used EMRAM data from 2006 to 2014 to estimate adoption levels to the year 2035.ResultsIn 2014, over 5400 hospitals completed HIMSS' annual EMRAM survey (86%+ of total US hospitals). In 2006, the majority of the US hospitals were in EMRAM Stages 0, 1, and 2. By 2014, most hospitals had achieved Stages 3, 4, and 5. The overall technology diffusion model (ie, the Bass model) reached an adjusted R-squared of .91. The final forecast depicted differing trends for each of the EMRAM stages. In 2006, the first year of observation, peaks of Stages 0 and 1 were shown as electronic health record adoption predates HIMSS' EMRAM. By 2007, Stage 2 reached its peak. Stage 3 reached its full height by 2011, while Stage 4 peaked by 2014. The first three stages created a graph that exhibits the expected "S-curve" for technology diffusion, with inflection point being the peak diffusion rate. This forecast indicates that Stage 5 should peak by 2019 and Stage 6 by 2026. Although this forecast extends to the year 2035, no peak was readily observed for Stage 7. Overall, most hospitals will achieve Stages 5, 6, or 7 of EMRAM by 2020; however, a considerable number of hospitals will not achieve Stage 7 by 2035.ConclusionsWe forecasted the adoption of electronic health record capabilities from a paper-based environment (Stage 0) to an environment where only electronic information is used to document and direct care delivery (Stage 7). According to our forecasts, the majority of hospitals will not reach Stage 7 until 2035, absent major policy changes or leaps in technological capabilities. These results indicate that US hospitals are decades away from fully implementing sophisticated decision support applications and interoperability functionalities in electronic health records as defined by EMRAM's Stage 7.