Dataset Information

Predictive Accuracy of Heart Failure-Specific Risk Equations in an Electronic Health Record-Based Cohort.

ABSTRACT:

Background

Guidelines recommend identification of individuals at risk for heart failure (HF). However, implementation of risk-based prevention strategies requires validation of HF-specific risk scores in diverse, real-world cohorts. Therefore, our objective was to assess the predictive accuracy of the Pooled Cohort Equations to Prevent HF within a primary prevention cohort derived from the electronic health record.

Methods

We retrospectively identified patients between the ages of 30 to 79 years in a multi-center integrated healthcare system, free of cardiovascular disease, with available data on HF risk factors, and at least 5 years of follow-up. We applied the Pooled Cohort Equations to Prevent HF tool to calculate sex and race-specific 5-year HF risk estimates. Incident HF was defined by the International Classification of Diseases codes. We assessed model discrimination and calibration, comparing predicted and observed rates for incident HF.

Results

Among 31 256 eligible adults, mean age was 51.4 years, 57% were women and 11% Black. Incident HF occurred in 568 patients (1.8%) over 5-year follow-up. The modified Pooled Cohort Equations to Prevent HF model for 5-year risk prediction of HF had excellent discrimination in White men (C-statistic 0.82 [95% CI, 0.79-0.86]) and women (0.82 [0.78-0.87]) and adequate discrimination in Black men (0.69 [0.60-0.78]) and women (0.69 [0.52-0.76]). Calibration was fair in all race-sex subgroups (χ²<20).

Conclusions

A novel sex- and race-specific risk score predicts incident HF in a real-world, electronic health record-based cohort. Integration of HF risk into the electronic health record may allow for risk-based discussion, enhanced surveillance, and targeted preventive interventions to reduce the public health burden of HF.

SUBMITTER: Bavishi A

PROVIDER: S-EPMC7674211 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Reduction of preventable hospital readmissions that result from chronic or acute conditions like stroke, heart failure, myocardial infarction and pneumonia remains a significant challenge for improving the outcomes and decreasing the cost of healthcare delivery in the United States. Patient readmission rates are relatively high for conditions like heart failure (HF) despite the implementation of high-quality healthcare delivery operation guidelines created by regulatory authorities. Multiple predictive models are currently available to evaluate potential 30-day readmission rates of patients. Most of these models are hypothesis driven and repetitively assess the predictive abilities of the same set of biomarkers as predictive features. In this manuscript, we discuss our attempt to develop a data-driven, electronic-medical record-wide (EMR-wide) feature selection approach and subsequent machine learning to predict readmission probabilities. We have assessed a large repertoire of variables from electronic medical records of heart failure patients in a single center. The cohort included 1,068 patients with 178 patients were readmitted within a 30-day interval (16.66% readmission rate). A total of 4,205 variables were extracted from EMR including diagnosis codes (n=1,763), medications (n=1,028), laboratory measurements (n=846), surgical procedures (n=564) and vital signs (n=4). We designed a multistep modeling strategy using the Naïve Bayes algorithm. In the first step, we created individual models to classify the cases (readmitted) and controls (non-readmitted). In the second step, features contributing to predictive risk from independent models were combined into a composite model using a correlation-based feature selection (CFS) method. All models were trained and tested using a 5-fold cross-validation method, with 70% of the cohort used for training and the remaining 30% for testing. Compared to existing predictive models for HF readmission rates (AUCs in the range of 0.6-0.7), results from our EMR-wide predictive model (AUC=0.78; Accuracy=83.19%) and phenome-wide feature selection strategies are encouraging and reveal the utility of such datadriven machine learning. Fine tuning of the model, replication using multi-center cohorts and prospective clinical trial to evaluate the clinical utility would help the adoption of the model as a clinical decision system for evaluating readmission status.

Project description:ImportanceAccurate, real-time case identification is needed to target interventions to improve quality and outcomes for hospitalized patients with heart failure. Problem lists may be useful for case identification but are often inaccurate or incomplete. Machine-learning approaches may improve accuracy of identification but can be limited by complexity of implementation.ObjectiveTo develop algorithms that use readily available clinical data to identify patients with heart failure while in the hospital.Design, setting, and participantsWe performed a retrospective study of hospitalizations at an academic medical center. Hospitalizations for patients 18 years or older who were admitted after January 1, 2013, and discharged before February 28, 2015, were included. From a random 75% sample of hospitalizations, we developed 5 algorithms for heart failure identification using electronic health record data: (1) heart failure on problem list; (2) presence of at least 1 of 3 characteristics: heart failure on problem list, inpatient loop diuretic, or brain natriuretic peptide level of 500 pg/mL or higher; (3) logistic regression of 30 clinically relevant structured data elements; (4) machine-learning approach using unstructured notes; and (5) machine-learning approach using structured and unstructured data.Main outcomes and measuresHeart failure diagnosis based on discharge diagnosis and physician review of sampled medical records.ResultsA total of 47 119 hospitalizations were included in this study (mean [SD] age, 60.9 [18.15] years; 23 952 female [50.8%], 5258 black/African American [11.2%], and 3667 Hispanic/Latino [7.8%] patients). Of these hospitalizations, 6549 (13.9%) had a discharge diagnosis of heart failure. Inclusion of heart failure on the problem list (algorithm 1) had a sensitivity of 0.40 and a positive predictive value (PPV) of 0.96 for heart failure identification. Algorithm 2 improved sensitivity to 0.77 at the expense of a PPV of 0.64. Algorithms 3, 4, and 5 had areas under the receiver operating characteristic curves of 0.953, 0.969, and 0.974, respectively. With a PPV of 0.9, these algorithms had associated sensitivities of 0.68, 0.77, and 0.83, respectively.Conclusions and relevanceThe problem list is insufficient for real-time identification of hospitalized patients with heart failure. The high predictive accuracy of machine learning using free text demonstrates that support of such analytics in future electronic health record systems can improve cohort identification.

Project description:AimsThe diagnosis of heart failure (HF) is an important problem in primary care. We previously demonstrated a 74% increase in registered HF diagnoses in primary care electronic health records (EHRs) following an extended audit procedure. What remains unclear is the accuracy of registered HF pre-audit and which EHR variables are most important in the extended audit strategy. This study aims to describe the diagnostic HF classification sequence at different stages, assess general practitioner (GP) HF misclassification, and test the predictive performance of an optimized audit.Methods and resultsThis is a secondary analysis of the OSCAR-HF study, a prospective observational trial including 51 participating GPs. OSCAR used an extended audit based on typical HF risk factors, signs, symptoms, and medications in GPs' EHR. This resulted in a list of possible HF patients, which participating GPs had to classify as HF or non-HF. We compared registered HF diagnoses before and after GPs' assessment. For our analysis of audit performance, we used GPs' assessment of HF as primary outcome and audit queries as dichotomous predictor variables for a gradient boosted machine (GBM) decision tree algorithm and logistic regression model. Of the 18 011 patients eligible for the audit intervention, 4678 (26.0%) were identified as possible HF patients and submitted for GPs' assessment in the audit stage. There were 310 patients with registered HF before GP assessment, of whom 146 (47.1%) were judged not to have HF by their GP (over-registration). There were 538 patients with registered HF after GP assessment, of whom 374 (69.5%) did not have registered HF before GP assessment (under-registration). The GBM and logistic regression model had a comparable predictive performance (area under the curve of 0.70 [95% confidence interval 0.65-0.77] and 0.69 [95% confidence interval 0.64-0.75], respectively). This was not significantly impacted by reducing the set of predictor variables to the 10 most important variables identified in the GBM model (free-text and coded cardiomyopathy, ischaemic heart disease and atrial fibrillation, digoxin, mineralocorticoid receptor antagonists, and combinations of renin-angiotensin system inhibitors and beta-blockers with diuretics). This optimized query set was enough to identify 86% (n = 461/538) of GPs' self-assessed HF population with a 33% reduction (n = 1537/4678) in screening caseload.ConclusionsDiagnostic coding of HF in primary care health records is inaccurate with a high degree of under-registration and over-registration. An optimized query set enabled identification of more than 80% of GPs' self-assessed HF population.

Project description:BackgroundElectronic health record (EHR) prediction models may be easier to use in busy clinical settings since EHR data can be auto-populated into models. This study assessed whether adding functional status and/or Medicare claims data (which are often not available in EHRs) improves the accuracy of a previously developed Veterans Affairs (VA) EHR-based mortality index.MethodsThis was a retrospective cohort study of veterans aged 75 years and older enrolled in VA primary care clinics followed from January 2014 to April 2020 (n = 62,014). We randomly split participants into development (n = 49,612) and validation (n = 12,402) cohorts. The primary outcome was all-cause mortality. We performed logistic regression with backward stepwise selection to develop a 100-predictor base model using 854 EHR candidate variables, including demographics, laboratory values, medications, healthcare utilization, diagnosis codes, and vitals. We incorporated functional measures in a base + function model by adding activities of daily living (range 0-5) and instrumental activities of daily living (range 0-7) scores. Medicare data, including healthcare utilization (e.g., emergency department visits, hospitalizations) and diagnosis codes, were incorporated in a base + Medicare model. A base + function + Medicare model included all data elements. We assessed model performance with the c-statistic, reclassification metrics, fraction of new information provided, and calibration plots.ResultsIn the overall cohort, mean age was 82.6 years and 98.6% were male. At the end of follow-up, 30,263 participants (48.8%) had died. The base model c-statistic was 0.809 (95% CI 0.805-0.812) in the development cohort and 0.804 (95% CI 0.796-0.812) in the validation cohort. Validation cohort c-statistics for the base + function, base + Medicare, and base + function + Medicare models were 0.809 (95% CI 0.801-0.816), 0.811 (95% CI 0.803-0.818), and 0.814 (95% CI 0.807-0.822), respectively. Adding functional status and Medicare data resulted in similarly small improvements among other model performance measures. All models showed excellent calibration.ConclusionsIncorporation of functional status and Medicare data into a VA EHR-based mortality index led to small but likely clinically insignificant improvements in model performance.

Project description:BackgroundAcute heart failure (AHF) is associated with significant morbidity and mortality. Effective patient risk stratification is essential to guiding hospitalization decisions and the clinical management of AHF. Clinical decision support systems can be used to improve predictions of mortality made in emergency care settings for the purpose of AHF risk stratification. In this study, several models for the prediction of seven-day mortality among AHF patients were developed by applying machine learning techniques to retrospective patient data from 236,275 total emergency department (ED) encounters, 1881 of which were considered positive for AHF and were used for model training and testing. The models used varying subsets of age, sex, vital signs, and laboratory values. Model performance was compared to the Emergency Heart Failure Mortality Risk Grade (EHMRG) model, a commonly used system for prediction of seven-day mortality in the ED with similar (or, in some cases, more extensive) inputs. Model performance was assessed in terms of area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity.ResultsWhen trained and tested on a large academic dataset, the best-performing model and EHMRG demonstrated test set AUROCs of 0.84 and 0.78, respectively, for prediction of seven-day mortality. Given only measurements of respiratory rate, temperature, mean arterial pressure, and FiO2, one model produced a test set AUROC of 0.83. Neither a logistic regression comparator nor a simple decision tree outperformed EHMRG.ConclusionsA model using only the measurements of four clinical variables outperforms EHMRG in the prediction of seven-day mortality in AHF. With these inputs, the model could not be replaced by logistic regression or reduced to a simple decision tree without significant performance loss. In ED settings, this minimal-input risk stratification tool may assist clinicians in making critical decisions about patient disposition by providing early and accurate insights into individual patient's risk profiles.

Project description:ImportanceThe Pooled Cohort Equations to Prevent Heart Failure (PCP-HF) estimate the 10-year risk for symptomatic heart failure (HF) from routine clinical data. The PCP-HF score should detect asymptomatic individuals with cardiac maladaptation preceding HF symptoms for it to be a useful HF prediction tool in primary prevention.ObjectiveTo assess the concordance between PCP-HF risk scoring and the presence of subclinical cardiac maladaptation in the community.Design, setting, and participantsThis cross-sectional analysis included participants enrolled in the Flemish Study on Environment, Genes and Health Outcomes who underwent a clinical examination including echocardiography between May 2005 and January 2015. Participants younger than 30 years, older than 79 years, had prevalent cardiovascular disease, and/or had suboptimal echocardiographic imaging quality were excluded. Analysis began February 2020 and ended April 2020.ExposuresTen-year HF risk as calculated from routine clinical data using the PCP-HF. Based on tertile limits, participants were categorized as having low (≤0.4%), intermediate (0.4%-2.4%), and high (≥2.4%) 10-year HF risk score.Main outcomes and measuresEchocardiographic profiles of subclinical heart remodeling and dysfunction.ResultsA total of 1020 individuals were analyzed (mean [SD] age, 52.8 [11.4] years; 541 female [53.0%]). The prevalence of left ventricular (LV) remodeling and dysfunction was significantly higher from low to intermediate and high 10-year HF risk score. A doubling in 10-year HF risk score was associated with higher odds for LV concentric remodeling (odds ratio [OR], 1.48; 95% CI, 1.36-1.61; P < .001), LV hypertrophy (OR, 1.66; 95% CI, 1.51-1.83; P < .001), abnormal LV longitudinal strain (OR, 1.12; 95% CI, 1.05-1.19; P < .001), and LV diastolic dysfunction (OR, 2.28; 95% CI, 1.94-2.69; P < .001). Moreover, the PCP-HF score detected echocardiographic abnormalities with an accuracy of 74% (LV concentric remodeling), 78% (LV hypertrophy), 59% (abnormal LV longitudinal strain), and 87% (LV diastolic dysfunction). The likelihood of LV concentric remodeling, hypertrophy, and diastolic dysfunction were 3.1, 3.8, and 9.4 times higher in participants with high 10-year HF risk score than the average population risk, respectively (P < .001). Of all PCP-HF score components, age, body mass index, and systolic blood pressure were key correlates of echocardiographic abnormalities in multivariable-adjusted analysis.Conclusions and relevancePCP-HF risk scoring adequately detected individuals with subclinical heart maladaptation that precedes HF symptoms by years. Thus, it may be a valuable HF prediction tool in primary prevention.