Dataset Information

Clinical Analytics Prediction Engine (CAPE): Development, electronic health record integration and prospective validation of hospital mortality, 180-day mortality and 30-day readmission risk prediction models.

ABSTRACT: BACKGROUND:Numerous predictive models in the literature stratify patients by risk of mortality and readmission. Few prediction models have been developed to optimize impact while sustaining sufficient performance. OBJECTIVE:We aimed to derive models for hospital mortality, 180-day mortality and 30-day readmission, implement these models within our electronic health record and prospectively validate these models for use across an entire health system. MATERIALS & METHODS:We developed, integrated into our electronic health record and prospectively validated three predictive models using logistic regression from data collected from patients 18 to 99 years old who had an inpatient or observation admission at NorthShore University HealthSystem, a four-hospital integrated system in the United States, from January 2012 to September 2018. We analyzed the area under the receiver operating characteristic curve (AUC) for model performance. RESULTS:Models were derived and validated at three time points: retrospective, prospective at discharge, and prospective at 4 hours after presentation. AUCs of hospital mortality were 0.91, 0.89 and 0.77, respectively. AUCs for 30-day readmission were 0.71, 0.71 and 0.69, respectively. 180-day mortality models were only retrospectively validated with an AUC of 0.85. DISCUSSION:We were able to retain good model performance while optimizing potential model impact by also valuing model derivation efficiency, usability, sensitivity, generalizability and ability to prescribe timely interventions to reduce underlying risk. Measuring model impact by tying prediction models to interventions that are then rapidly tested will establish a path for meaningful clinical improvement and implementation.

SUBMITTER: Shah N

PROVIDER: S-EPMC7451512 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Clinical Analytics Prediction Engine (CAPE): Development, electronic health record integration and prospective validation of hospital mortality, 180-day mortality and 30-day readmission risk prediction models.

Shah Nirav N Konchak Chad C Chertok Daniel D Au Loretta L Kozlov Alex A Ravichandran Urmila U McNulty Patrick P Liao Linning L Steele Kate K Kharasch Maureen M Boyle Chris C Hensing Tom T Lovinger David D Birnberg Jonathan J Solomonides Anthony A Halasyamani Lakshmi L

PloS one 20200827 8

<h4>Background</h4>Numerous predictive models in the literature stratify patients by risk of mortality and readmission. Few prediction models have been developed to optimize impact while sustaining sufficient performance.<h4>Objective</h4>We aimed to derive models for hospital mortality, 180-day mortality and 30-day readmission, implement these models within our electronic health record and prospectively validate these models for use across an entire health system.<h4>Materials & methods</h4>We ...[more]

PMID: 32853223

Similar Datasets

Project description:ImportanceIn the US, more than 600 000 adults will experience an acute myocardial infarction (AMI) each year, and up to 20% of the patients will be rehospitalized within 30 days. This study highlights the need for consideration of calibration in these risk models.ObjectiveTo compare multiple machine learning risk prediction models using an electronic health record (EHR)-derived data set standardized to a common data model.Design, setting, and participantsThis was a retrospective cohort study that developed risk prediction models for 30-day readmission among all inpatients discharged from Vanderbilt University Medical Center between January 1, 2007, and December 31, 2016, with a primary diagnosis of AMI who were not transferred from another facility. The model was externally validated at Dartmouth-Hitchcock Medical Center from April 2, 2011, to December 31, 2016. Data analysis occurred between January 4, 2019, and November 15, 2020.ExposuresAcute myocardial infarction that required hospital admission.Main outcomes and measuresThe main outcome was thirty-day hospital readmission. A total of 141 candidate variables were considered from administrative codes, medication orders, and laboratory tests. Multiple risk prediction models were developed using parametric models (elastic net, least absolute shrinkage and selection operator, and ridge regression) and nonparametric models (random forest and gradient boosting). The models were assessed using holdout data with area under the receiver operating characteristic curve (AUROC), percentage of calibration, and calibration curve belts.ResultsThe final Vanderbilt University Medical Center cohort included 6163 unique patients, among whom the mean (SD) age was 67 (13) years, 4137 were male (67.1%), 1019 (16.5%) were Black or other race, and 933 (15.1%) were rehospitalized within 30 days. The final Dartmouth-Hitchcock Medical Center cohort included 4024 unique patients, with mean (SD) age of 68 (12) years; 2584 (64.2%) were male, 412 (10.2%) were rehospitalized within 30 days, and most of the cohort were non-Hispanic and White. The final test set AUROC performance was between 0.686 to 0.695 for the parametric models and 0.686 to 0.704 for the nonparametric models. In the validation cohort, AUROC performance was between 0.558 to 0.655 for parametric models and 0.606 to 0.608 for nonparametric models.Conclusions and relevanceIn this study, 5 machine learning models were developed and externally validated to predict 30-day readmission AMI hospitalization. These models can be deployed within an EHR using routinely collected data.

Project description:BackgroundExisting readmission reduction solutions tend to focus on complementing inpatient care with enhanced care transition and postdischarge interventions. These solutions are initiated near or after discharge, when clinicians' impact on inpatient care is ending. Preventive intervention during hospitalization is an underexplored area that holds potential for reducing readmission risk. However, it is challenging to predict readmission risk at the early stage of hospitalization because few data are available.ObjectiveThe objective of this study was to build an early prediction model of unplanned 30-day hospital readmission using a large and diverse sample. We were also interested in identifying novel readmission risk factors and protective factors.MethodsWe extracted the medical records of 96,550 patients in 205 participating Cerner client hospitals across four US census regions in 2016 from the Health Facts database. The model was built with index admission data that can become available within 24 hours and data from previous encounters up to 1 year before the index admission. The candidate models were evaluated for performance, timeliness, and generalizability. Multivariate logistic regression analysis was used to identify readmission risk factors and protective factors.ResultsWe developed six candidate readmission models with different machine learning algorithms. The best performing model of extreme gradient boosting (XGBoost) achieved an area under the receiver operating characteristic curve of 0.753 on the development data set and 0.742 on the validation data set. By multivariate logistic regression analysis, we identified 14 risk factors and 2 protective factors of readmission that have never been reported.ConclusionsThe performance of our model is better than that of the most widely used models in US health care settings. This model can help clinicians identify readmission risk at the early stage of hospitalization so that they can pay extra attention during the care process of high-risk patients. The 14 novel risk factors and 2 novel protective factors can aid understanding of the factors associated with readmission.

Project description:BackgroundLowering hospital readmission rates has become a primary target for the Centers for Medicare & Medicaid Services, but studies of the relationship between adherence to the recommended hospital care processes and readmission rates have provided inconsistent and inconclusive results.ObjectiveTo examine the association between hospital performance on Medicare's Hospital Compare process quality measures and 30-day readmission rates for patients with acute myocardial infarction (AMI), heart failure and pneumonia, and for those undergoing major surgery.Design, setting and participantsWe assessed hospital performance on process measures using the 2007 Hospital Inpatient Quality Reporting Program. The process measures for each condition were aggregated in two separate measures: Overall Measure (OM) and Appropriate Care Measure (ACM) scores. Readmission rates were calculated using Medicare claims.Main outcome measureRisk-standardized 30-day all-cause readmission rate was calculated as the ratio of predicted to expected rate standardized by the overall mean readmission rate. We calculated predicted readmission rate using hierarchical generalized linear models and adjusting for patient-level factors.ResultsAmong patients aged ≥ 66 years, the median OM score ranged from 79.4 % for abdominal surgery to 95.7 % for AMI, and the median ACM scores ranged from 45.8 % for abdominal surgery to 87.9 % for AMI. We observed a statistically significant, but weak, correlation between performance scores and readmission rates for pneumonia (correlation coefficient R = 0.07), AMI (R = 0.10), and orthopedic surgery (R = 0.06). The difference in the mean readmission rate between hospitals in the 1st and 4th quartiles of process measure performance was statistically significant only for AMI (0.25 percentage points) and pneumonia (0.31 percentage points). Performance on process measures explained less than 1 % of hospital-level variation in readmission rates.ConclusionsHospitals with greater adherence to recommended care processes did not achieve meaningfully better 30-day hospital readmission rates compared to those with lower levels of performance.

Project description:BackgroundPredictive models based on electronic health records (EHRs) are used to identify patients at high risk for 30-day hospital readmission. However, these models' ability to accurately detect who could benefit from inclusion in prevention interventions, also termed "perceived impactibility", has yet to be realized.ObjectiveWe aimed to explore healthcare providers' perspectives of patient characteristics associated with decisions about which patients should be referred to readmission prevention programs (RPPs) beyond the EHR preadmission readmission detection model (PREADM).DesignThis cross-sectional study employed a multi-source mixed-method design, combining EHR data with nurses' and physicians' self-reported surveys from 15 internal medicine units in three general hospitals in Israel between May 2016 and June 2017, using a mini-Delphi approach.ParticipantsNurses and physicians were asked to provide information about patients 65 years or older who were hospitalized at least one night.Main measuresWe performed a decision-tree analysis to identify characteristics for consideration when deciding whether a patient should be included in an RPP.Key resultsWe collected 817 questionnaires on 435 patients. PREADM score and RPP inclusion were congruent in 65% of patients, whereas 19% had a high PREADM score but were not referred to an RPP, and 16% had a low-medium PREADM score but were referred to an RPP. The decision-tree analysis identified five patient characteristics that were statistically associated with RPP referral: high PREADM score, eligibility for a nursing home, having a condition not under control, need for social-services support, and need for special equipment at home.ConclusionsOur study provides empirical evidence for the partial congruence between classifications of a high PREADM score and perceived impactibility. Findings emphasize the need for additional research to understand the extent to which combining EHR data with provider insights leads to better selection of patients for RPP inclusion.

Project description:Background and Purpose- Kidney dysfunction is common among patients hospitalized for ischemic stroke. Understanding the association of kidney disease with poststroke outcomes is important to properly adjust for case mix in outcome studies, payment models and risk-standardized hospital readmission rates. Methods- In this cohort study of fee-for-service Medicare patients admitted with ischemic stroke to 1579 Get With The Guidelines-Stroke participating hospitals between 2009 and 2014, adjusted multivariable Cox proportional hazards models were used to determine the independent associations of estimated glomerular filtration rate (eGFR) and dialysis status with 30-day and 1-year postdischarge mortality and rehospitalizations. Results- Of 204?652 patients discharged alive (median age [25th-75th percentile] 80 years [73.0-86.0], 57.6% women, 79.8% white), 48.8% had an eGFR ?60, 26.5% an eGFR 45 to 59, 16.3% an eGFR 30 to 44, 5.1% an eGFR 15 to 29, 0.6% an eGFR <15 without dialysis, and 2.8% were receiving dialysis. Compared with eGFR ?60, and after adjusting for relevant variables, eGFR <45 was associated with increased 30-day mortality with the risk highest among those with eGFR <15 without dialysis (hazard ratio [HR], 2.09; 95% CI, 1.66-2.63). An eGFR <60 was associated with increased 1-year poststroke mortality that was highest among patients on dialysis (HR, 2.65; 95% CI, 2.49-2.81). Dialysis was also associated with the highest 30-day and 1-year rehospitalization rates (HR, 2.10; 95% CI, 1.95-2.26 and HR, 2.55; 95% CI, 2.44-2.66, respectively) and 30-day and 1-year composite of mortality and rehospitalization (HR, 2.04; 95% CI, 1.90-2.18 and HR, 2.46; 95% CI, 2.36-2.56, respectively). Conclusions- Within the first year after index hospitalization for ischemic stroke, eGFR and dialysis status on admission are associated with poststroke mortality and hospital readmissions. Kidney function should be included in risk-stratification models for poststroke outcomes.

Project description:RESEARCH OBJECTIVE:Pharmacists are an expensive and limited resource in the hospital and outpatient setting. A pharmacist can spend up to 25% of their day planning. Time spent planning is time not spent delivering an intervention. A readmission risk adjustment model has potential to be used as a universal outcome-based prioritization tool to help pharmacists plan their interventions more efficiently. Pharmacy-specific predictors have not been used in the constructs of current readmission risk models. We assessed the impact of adding pharmacy-specific predictors on performance of readmission risk prediction models. STUDY DESIGN:We used an observational retrospective cohort study design to assess whether pharmacy-specific predictors such as an aggregate pharmacy score and drug classes would improve the prediction of 30-day readmission. A model of age, sex, length of stay, and admission category predictors was used as the reference model. We added predictor variables in sequential models to evaluate the incremental effect of additional predictors on the performance of the reference. We used logistic regression to regress the outcomes on predictors in our derivation dataset. We derived and internally validated our models through a 50:50 split validation of our dataset. POPULATION STUDIED:Our study population (n=350,810) was of adult admissions at hospitals in a large integrated health care delivery system. PRINCIPAL FINDINGS:Individually, the aggregate pharmacy score and drug classes caused a nearly identical but moderate increase in model performance over the reference. As a single predictor, the comorbidity burden score caused the greatest increase in model performance when added to the reference. Adding the severity of illness score, comorbidity burden score and the aggregate pharmacy score to the reference caused a cumulative increase in model performance with good discrimination (c statistic, 0.712; Nagelkerke R, 0.112). The best performing model included all predictors: severity of illness score, comorbidity burden score, aggregate pharmacy score, diagnosis groupings, and drug subgroups. CONCLUSIONS:Adding the aggregate pharmacy score to the reference model significantly increased the c statistic but was out-performed by the comorbidity burden score model in predicting readmission. The need for a universal prioritization tool for pharmacists may therefore be potentially met with the comorbidity burden score model. However, the aggregate pharmacy score and drug class models still out-performed current Medicare readmission risk adjustment models. IMPLICATIONS FOR POLICY OR PRACTICE:Pharmacists have a great role in preventing readmission, and therefore can potentially use one of our models: comorbidity burden score model, aggregate pharmacy score model, drug class model or complex model (a combination of all 5 major predictors) to prioritize their interventions while exceeding Medicare performance measures on readmission. The choice of model to use should be based on the availability of these predictors in the health care system.

Project description:Risk-standardized 30-day mortality and hospital readmission rates for pneumonia are increasingly being tied to hospital reimbursement to incentivize the delivery of high-quality care. Such measures may be susceptible to gaming by recoding patients with pneumonia to a primary diagnosis of sepsis or respiratory failure. We sought to determine the degree to which hospitals can game mortality or readmission measures and change their rankings by recoding patients with pneumonia.Simulated experimental study of 2,906 U.S. acute care hospitals with at least 25 admissions for pneumonia using 2009 Medicare data.Elderly (age ? 65 yr) Medicare fee-for-service beneficiaries hospitalized with pneumonia. Patients eligible for recoding to sepsis or respiratory failure were those with a principal International Classification of Diseases, 9th Edition, Clinical Modification, discharge code for pneumonia and secondary codes for respiratory failure or acute organ dysfunction.None.We measured the number of hospitals that improved their pneumonia mortality or readmission rates after recoding eligible patients. When a sample of 100 hospitals with pneumonia mortality rates above the 50th percentile recoded all eligible patients to sepsis or respiratory failure, 90 hospitals (95% CI, 84-95) improved their mortality rate (mean improvement, 1.09%; 95% CI, 0.94-1.28%) and 41 hospitals dropped below the 50th percentile (95% CI, 33-52). When a sample of 100 hospitals with pneumonia readmission rates above the 50th percentile recoded all eligible patients, 66 hospitals (95% CI, 54-75) improved their readmission rate (mean improvement, 0.34%; 95% CI, 0.19-0.45%) and 15 hospitals (95% CI, 9-22) dropped below the 50th percentile.Hospitals can improve apparent pneumonia mortality and readmission rates by recoding pneumonia patients. Centers for Medicare and Medicaid Services should consider changes to their methods used to calculate hospital-level pneumonia outcome measures to make them less susceptible to gaming.

Dataset Information

Clinical Analytics Prediction Engine (CAPE): Development, electronic health record integration and prospective validation of hospital mortality, 180-day mortality and 30-day readmission risk prediction models.

Publications

Clinical Analytics Prediction Engine (CAPE): Development, electronic health record integration and prospective validation of hospital mortality, 180-day mortality and 30-day readmission risk prediction models.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets