Dataset Information

A machine learning approach to identify distinct subgroups of veterans at risk for hospitalization or death using administrative and electronic health record data.

ABSTRACT:

Background

Identifying individuals at risk for future hospitalization or death has been a major priority of population health management strategies. High-risk individuals are a heterogeneous group, and existing studies describing heterogeneity in high-risk individuals have been limited by data focused on clinical comorbidities and not socioeconomic or behavioral factors. We used machine learning clustering methods and linked comorbidity-based, sociodemographic, and psychobehavioral data to identify subgroups of high-risk Veterans and study long-term outcomes, hypothesizing that factors other than comorbidities would characterize several subgroups.

Methods and findings

In this cross-sectional study, we used data from the VA Corporate Data Warehouse, a national repository of VA administrative claims and electronic health data. To identify high-risk Veterans, we used the Care Assessment Needs (CAN) score, a routinely-used VA model that predicts a patient's percentile risk of hospitalization or death at one year. Our study population consisted of 110,000 Veterans who were randomly sampled from 1,920,436 Veterans with a CAN score?75th percentile in 2014. We categorized patient-level data into 119 independent variables based on demographics, comorbidities, pharmacy, vital signs, laboratories, and prior utilization. We used a previously validated density-based clustering algorithm to identify 30 subgroups of high-risk Veterans ranging in size from 50 to 2,446 patients. Mean CAN score ranged from 72.4 to 90.3 among subgroups. Two-year mortality ranged from 0.9% to 45.6% and was highest in the home-based care and metastatic cancer subgroups. Mean inpatient days ranged from 1.4 to 30.5 and were highest in the post-surgery and blood loss anemia subgroups. Mean emergency room visits ranged from 1.0 to 4.3 and were highest in the chronic sedative use and polysubstance use with amphetamine predominance subgroups. Five subgroups were distinguished by psychobehavioral factors and four subgroups were distinguished by sociodemographic factors.

Conclusions

High-risk Veterans are a heterogeneous population consisting of multiple distinct subgroups-many of which are not defined by clinical comorbidities-with distinct utilization and outcome patterns. To our knowledge, this represents the largest application of ML clustering methods to subgroup a high-risk population. Further study is needed to determine whether distinct subgroups may benefit from individualized interventions.

SUBMITTER: Parikh RB

PROVIDER: S-EPMC7894856 | biostudies-literature | 2021

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A machine learning approach to identify distinct subgroups of veterans at risk for hospitalization or death using administrative and electronic health record data.

Parikh Ravi B RB Linn Kristin A KA Yan Jiali J Maciejewski Matthew L ML Rosland Ann-Marie AM Volpp Kevin G KG Groeneveld Peter W PW Navathe Amol S AS

PloS one 20210219 2

<h4>Background</h4>Identifying individuals at risk for future hospitalization or death has been a major priority of population health management strategies. High-risk individuals are a heterogeneous group, and existing studies describing heterogeneity in high-risk individuals have been limited by data focused on clinical comorbidities and not socioeconomic or behavioral factors. We used machine learning clustering methods and linked comorbidity-based, sociodemographic, and psychobehavioral data ...[more]

PMID: 33606819

Similar Datasets

Project description:BackgroundIt is unclear whether machine learning methods yield more accurate electronic health record (EHR) prediction models compared with traditional regression methods.ObjectiveThe objective of this study was to compare machine learning and traditional regression models for 10-year mortality prediction using EHR data.DesignThis was a cohort study.SettingVeterans Affairs (VA) EHR data.ParticipantsVeterans age above 50 with a primary care visit in 2005, divided into separate training and testing cohorts (n= 124,360 each).Measurements and analytic methodsThe primary outcome was 10-year all-cause mortality. We considered 924 potential predictors across a wide range of EHR data elements including demographics (3), vital signs (9), medication classes (399), disease diagnoses (293), laboratory results (71), and health care utilization (149). We compared discrimination (c-statistics), calibration metrics, and diagnostic test characteristics (sensitivity, specificity, and positive and negative predictive values) of machine learning and regression models.ResultsOur cohort mean age (SD) was 68.2 (10.5), 93.9% were male; 39.4% died within 10 years. Models yielded testing cohort c-statistics between 0.827 and 0.837. Utilizing all 924 predictors, the Gradient Boosting model yielded the highest c-statistic [0.837, 95% confidence interval (CI): 0.835-0.839]. The full (unselected) logistic regression model had the highest c-statistic of regression models (0.833, 95% CI: 0.830-0.835) but showed evidence of overfitting. The discrimination of the stepwise selection logistic model (101 predictors) was similar (0.832, 95% CI: 0.830-0.834) with minimal overfitting. All models were well-calibrated and had similar diagnostic test characteristics.LimitationOur results should be confirmed in non-VA EHRs.ConclusionThe differences in c-statistic between the best machine learning model (924-predictor Gradient Boosting) and 101-predictor stepwise logistic models for 10-year mortality prediction were modest, suggesting stepwise regression methods continue to be a reasonable method for VA EHR mortality prediction model development.

Project description:BACKGROUND:Systemic sclerosis (SSc) is a rare disease with studies limited by small sample sizes. Electronic health records (EHRs) represent a powerful tool to study patients with rare diseases such as SSc, but validated methods are needed. We developed and validated EHR-based algorithms that incorporate billing codes and clinical data to identify SSc patients in the EHR. METHODS:We used a de-identified EHR with over 3 million subjects and identified 1899 potential SSc subjects with at least 1 count of the SSc ICD-9 (710.1) or ICD-10-CM (M34*) codes. We randomly selected 200 as a training set for chart review. A subject was a case if diagnosed with SSc by a rheumatologist, dermatologist, or pulmonologist. We selected the following algorithm components based on clinical knowledge and available data: SSc ICD-9 and ICD-10-CM codes, positive antinuclear antibody (ANA) (titer ??1:80), and a keyword of Raynaud's phenomenon (RP). We performed both rule-based and machine learning techniques for algorithm development. Positive predictive values (PPVs), sensitivities, and F-scores (which account for PPVs and sensitivities) were calculated for the algorithms. RESULTS:PPVs were low for algorithms using only 1 count of the SSc ICD-9 code. As code counts increased, the PPVs increased. PPVs were higher for algorithms using ICD-10-CM codes versus the ICD-9 code. Adding a positive ANA and RP keyword increased the PPVs of algorithms only using ICD billing codes. Algorithms using ??3 or ??4 counts of the SSc ICD-9 or ICD-10-CM codes and ANA positivity had the highest PPV at 100% but a low sensitivity at 50%. The algorithm with the highest F-score of 91% was ??4 counts of the ICD-9 or ICD-10-CM codes with an internally validated PPV of 90%. A machine learning method using random forests yielded an algorithm with a PPV of 84%, sensitivity of 92%, and F-score of 88%. The most important feature was RP keyword. CONCLUSIONS:Algorithms using only ICD-9 codes did not perform well to identify SSc patients. The highest performing algorithms incorporated clinical data with billing codes. EHR-based algorithms can identify SSc patients across a healthcare system, enabling researchers to examine important outcomes.

Project description:ImportanceComparisons of antimicrobial use among hospitals are difficult to interpret owing to variations in patient case mix. Risk-adjustment strategies incorporating larger numbers of variables haves been proposed as a method to improve comparisons for antimicrobial stewardship assessments.ObjectiveTo evaluate whether variables of varying complexity and feasibility of measurement, derived retrospectively from the electronic health records, accurately identify inpatient antimicrobial use.Design, setting, and participantsRetrospective cohort study, using a 2-stage random forests machine learning modeling analysis of electronic health record data. Data were split into training and testing sets to measure model performance using area under the curve and absolute error. All adult and pediatric inpatient encounters from October 1, 2015, to September 30, 2017, at 2 community hospitals and 1 academic medical center in the Duke University Health System were analyzed. A total of 204 candidate variables were categorized into 4 tiers based on feasibility of measurement from the electronic health records.Main outcomes and measuresAntimicrobial exposure was measured at the encounter level in 2 ways: binary (ever or never) and number of days of therapy. Analyses were stratified by age (pediatric or adult), unit type, and antibiotic group.ResultsThe data set included 170 294 encounters and 204 candidate variables from 3 hospitals during the 3-year study period. Antimicrobial exposure occurred in 80 190 encounters (47%); 64 998 (38%) received 1 to 6 days of therapy, and 15 192 (9%) received 7 or more days of therapy. Two-stage models identified antimicrobial use with high fidelity (mean area under the curve, 0.85; mean absolute error, 1.0 days of therapy). Addition of more complex variables increased accuracy, with largest improvements occurring with inclusion of diagnosis information. Accuracy varied based on location and antibiotic group. Models underestimated the number of days of therapy of encounters with long lengths of stay.Conclusions and relevanceModels using variables derived from electronic health records identified antimicrobial exposure accurately. Future risk-adjustment strategies incorporating encounter-level information may make comparisons of antimicrobial use more meaningful for hospital antimicrobial stewardship assessments.

Project description:Electronic health records (EHR) offer medical and pharmacogenomics research unprecedented opportunities to identify and classify patients at risk. EHRs are collections of highly inter-dependent records that include biological, anatomical, physiological, and behavioral observations. They comprise a patient's clinical phenome, where each patient has thousands of date-stamped records distributed across many relational tables. Development of EHR computer-based phenotyping algorithms require time and medical insight from clinical experts, who most often can only review a small patient subset representative of the total EHR records, to identify phenotype features. In this research we evaluate whether relational machine learning (ML) using inductive logic programming (ILP) can contribute to addressing these issues as a viable approach for EHR-based phenotyping.Two relational learning ILP approaches and three well-known WEKA (Waikato Environment for Knowledge Analysis) implementations of non-relational approaches (PART, J48, and JRIP) were used to develop models for nine phenotypes. International Classification of Diseases, Ninth Revision (ICD-9) coded EHR data were used to select training cohorts for the development of each phenotypic model. Accuracy, precision, recall, F-Measure, and Area Under the Receiver Operating Characteristic (AUROC) curve statistics were measured for each phenotypic model based on independent manually verified test cohorts. A two-sided binomial distribution test (sign test) compared the five ML approaches across phenotypes for statistical significance.We developed an approach to automatically label training examples using ICD-9 diagnosis codes for the ML approaches being evaluated. Nine phenotypic models for each ML approach were evaluated, resulting in better overall model performance in AUROC using ILP when compared to PART (p=0.039), J48 (p=0.003) and JRIP (p=0.003).ILP has the potential to improve phenotyping by independently delivering clinically expert interpretable rules for phenotype definitions, or intuitive phenotypes to assist experts.Relational learning using ILP offers a viable approach to EHR-driven phenotyping.

Project description:OBJECTIVES:Identifying subgroups of ICU patients with similar clinical needs and trajectories may provide a framework for more efficient ICU care through the design of care platforms tailored around patients' shared needs. However, objective methods for identifying these ICU patient subgroups are lacking. We used a machine learning approach to empirically identify ICU patient subgroups through clustering analysis and evaluate whether these groups might represent appropriate targets for care redesign efforts. DESIGN:We performed clustering analysis using data from patients' hospital stays to retrospectively identify patient subgroups from a large, heterogeneous ICU population. SETTING:Kaiser Permanente Northern California, a healthcare delivery system serving 3.9 million members. PATIENTS:ICU patients 18 years old or older with an ICU admission between January 1, 2012, and December 31, 2012, at one of 21 Kaiser Permanente Northern California hospitals. INTERVENTIONS:None. MEASUREMENTS AND MAIN RESULTS:We used clustering analysis to identify putative clusters among 5,000 patients randomly selected from 24,884 ICU patients. To assess cluster validity, we evaluated the distribution and frequency of patient characteristics and the need for invasive therapies. We then applied a classifier built from the sample cohort to the remaining 19,884 patients to compare the derivation and validation clusters. Clustering analysis successfully identified six clinically recognizable subgroups that differed significantly in all baseline characteristics and clinical trajectories, despite sharing common diagnoses. In the validation cohort, the proportion of patients assigned to each cluster was similar and demonstrated significant differences across clusters for all variables. CONCLUSIONS:A machine learning approach revealed important differences between empirically derived subgroups of ICU patients that are not typically revealed by admitting diagnosis or severity of illness alone. Similar data-driven approaches may provide a framework for future organizational innovations in ICU care tailored around patients' shared needs.

Project description:BACKGROUND:The limitations of existing HIV risk prediction tools are a barrier to implementation of pre-exposure prophylaxis (PrEP). We developed and validated an HIV prediction model to identify potential PrEP candidates in a large health-care system. METHODS:Our study population was HIV-uninfected adult members of Kaiser Permanente Northern California, a large integrated health-care system, who were not yet using PrEP and had at least 2 years of previous health plan enrolment with at least one outpatient visit from Jan 1, 2007, to Dec 31, 2017. Using 81 electronic health record (EHR) variables, we applied least absolute shrinkage and selection operator (LASSO) regression to predict incident HIV diagnosis within 3 years on a subset of patients who entered the cohort in 2007-14 (development dataset), assessing ten-fold cross-validated area under the receiver operating characteristic curve (AUC) and 95% CIs. We compared the full model to simpler models including only men who have sex with men (MSM) status and sexually transmitted infection (STI) positivity, testing, and treatment. Models were validated prospectively with data from an independent set of patients who entered the cohort in 2015-17. We computed predicted probabilities of incident HIV diagnosis within 3 years (risk scores), categorised as low risk (<0·05%), moderate risk (0·05% to <0·20%), high risk (0·20% to <1·0%), and very high risk (?1·0%), for all patients in the validation dataset. FINDINGS:Of 3?750?664 patients in 2007-17 (3?143?963 in the development dataset and 606?701 in the validation dataset), there were 784 incident HIV cases within 3 years of baseline. The LASSO procedure retained 44 predictors in the full model, with an AUC of 0·86 (95% CI 0·85-0·87) for incident HIV cases in 2007-14. Model performance remained high in the validation dataset (AUC 0·84, 0·80-0·89). The full model outperformed simpler models including only MSM status and STI positivity. For the full model, flagging 13?463 (2·2%) patients with high or very high HIV risk scores in the validation dataset identified 32 (38·6%) of the 83 incident HIV cases, including 32 (46·4%) of 69 male cases and none of the 14 female cases. The full model had equivalent sensitivity by race whereas simpler models identified fewer black than white HIV cases. INTERPRETATION:Prediction models using EHR data can identify patients at high risk of HIV acquisition who could benefit from PrEP. Future studies should optimise EHR-based HIV risk prediction tools and evaluate their effect on prescription of PrEP. FUNDING:Kaiser Permanente Community Benefit Research Program and the US National Institutes of Health.

Dataset Information

A machine learning approach to identify distinct subgroups of veterans at risk for hospitalization or death using administrative and electronic health record data.

Background

Methods and findings

Conclusions

Publications

A machine learning approach to identify distinct subgroups of veterans at risk for hospitalization or death using administrative and electronic health record data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets