Dataset Information

Derivation of an electronic frailty index for predicting short-term mortality in heart failure: a machine learning approach.

ABSTRACT:

Aims

Frailty may be found in heart failure patients especially in the elderly and is associated with a poor prognosis. However, assessment of frailty status is time-consuming, and the electronic frailty indices developed using health records have served as useful surrogates. We hypothesized that an electronic frailty index developed using machine learning can improve short-term mortality prediction in patients with heart failure.

Methods and results

This was a retrospective observational study that included patients admitted to nine public hospitals for heart failure from Hong Kong between 2013 and 2017. Age, sex, variables in the modified frailty index, Deyo's Charlson co-morbidity index (≥2), neutrophil-to-lymphocyte ratio (NLR), and prognostic nutritional index at baseline were analysed. Gradient boosting, which is a supervised sequential ensemble learning algorithm with weak prediction submodels (typically decision trees), was applied to predict mortality. Variables were ranked in the order of importance with a total score of 100 and used to build the frailty models. Comparisons were made with decision tree and multivariable logistic regression. A total of 8893 patients (median: age 81, Q1-Q3: 71-87 years old) were included, in whom 9% had 30 day mortality and 17% had 90 day mortality. Prognostic nutritional index, age, and NLR were the most important variables predicting 30 day mortality (importance score: 37.4, 32.1, and 20.5, respectively) and 90 day mortality (importance score: 35.3, 36.3, and 14.6, respectively). Gradient boosting significantly outperformed decision tree and multivariable logistic regression. The area under the curve from a five-fold cross validation was 0.90 for gradient boosting and 0.87 and 0.86 for decision tree and logistic regression in predicting 30 day mortality. For the prediction of 90 day mortality, the area under the curve was 0.92, 0.89, and 0.86 for gradient boosting, decision tree, and logistic regression, respectively.

Conclusions

The electronic frailty index based on co-morbidities, inflammation, and nutrition information can readily predict mortality outcomes. Their predictive performances were significantly improved by gradient boosting techniques.

SUBMITTER: Ju C

PROVIDER: S-EPMC8318426 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:AimsThis study aimed to review the performance of machine learning (ML) methods compared with conventional statistical models (CSMs) for predicting readmission and mortality in patients with heart failure (HF) and to present an approach to formally evaluate the quality of studies using ML algorithms for prediction modelling.Methods and resultsFollowing Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we performed a systematic literature search using MEDLINE, EPUB, Cochrane CENTRAL, EMBASE, INSPEC, ACM Library, and Web of Science. Eligible studies included primary research articles published between January 2000 and July 2020 comparing ML and CSMs in mortality and readmission prognosis of initially hospitalized HF patients. Data were extracted and analysed by two independent reviewers. A modified CHARMS checklist was developed in consultation with ML and biostatistics experts for quality assessment and was utilized to evaluate studies for risk of bias. Of 4322 articles identified and screened by two independent reviewers, 172 were deemed eligible for a full-text review. The final set comprised 20 articles and 686 842 patients. ML methods included random forests (n = 11), decision trees (n = 5), regression trees (n = 3), support vector machines (n = 9), neural networks (n = 12), and Bayesian techniques (n = 3). CSMs included logistic regression (n = 16), Cox regression (n = 3), or Poisson regression (n = 3). In 15 studies, readmission was examined at multiple time points ranging from 30 to 180 day readmission, with the majority of studies (n = 12) presenting prediction models for 30 day readmission outcomes. Of a total of 21 time-point comparisons, ML-derived c-indices were higher than CSM-derived c-indices in 16 of the 21 comparisons. In seven studies, mortality was examined at 9 time points ranging from in-hospital mortality to 1 year survival; of these nine, seven reported higher c-indices using ML. Two of these seven studies reported survival analyses utilizing random survival forests in their ML prediction models. Both reported higher c-indices when using ML compared with CSMs. A limitation of studies using ML techniques was that the majority were not externally validated, and calibration was rarely assessed. In the only study that was externally validated in a separate dataset, ML was superior to CSMs (c-indices 0.913 vs. 0.835).ConclusionsML algorithms had better discrimination than CSMs in most studies aiming to predict risk of readmission and mortality in HF patients. Based on our review, there is a need for external validation of ML-based studies of prediction modelling. We suggest that ML-based studies should also be evaluated using clinical quality standards for prognosis research. Registration: PROSPERO CRD42020134867.

Project description:AimsIndividual risk stratification is a fundamental strategy in managing patients with heart failure (HF). Artificial intelligence, particularly machine learning (ML), can develop superior models for predicting the prognosis of HF patients, and administrative claim data (ACD) are suitable for ML analysis because ACD is a structured database. The objective of this study was to analyse ACD using an ML algorithm, predict the 1 year mortality of patients with HF, and finally develop an easy-to-use prediction model with high accuracy using the top predictors identified by the ML algorithm.Methods and resultsMachine learning-based prognostic prediction models were developed from the ACD on 10 175 HF patients from the Japanese Registry of Acute Decompensated Heart Failure with 17% mortality during 1 year follow-up. The top predictors for prognosis in HF were identified by the permutation feature importance technique, and an easy-to-use prediction model was developed based on these predictors. The c-statistics and Brier scores of the developed ML-based models were compared with those of conventional risk models: Seattle Heart Failure Model (SHFM) and Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC). A voting classifier algorithm (ACD-VC) achieved the highest c-statistics among the six ML algorithms. The permutation feature importance technique enabled identification of the top predictors such as Barthel index, age, body mass index, duration of hospitalization, last hospitalization, renal disease, and non-loop diuretics use (feature importance values were 0.054, 0.025, 0.010, 0.005, 0.005, 0.004, and 0.004, respectively). Upon combination of some of the predictors that can be assessed from a brief interview, the Simple Model by ARTificial intelligence for HF risk stratification (SMART-HF) was established as an easy-to-use prediction model. Compared with the conventional models, SMART-HF achieved a higher c-statistic {ACD-VC: 0.777 [95% confidence interval (CI) 0.751-0.803], SMART-HF: 0.765 [95% CI 0.739-0.791], SHFM: 0.713 [95% CI 0.684-0.742], MAGGIC: 0.726 [95% CI 0.698-0.753]} and better Brier scores (ACD-VC: 0.121, SMART-HF: 0.124, SHFM: 0.139, MAGGIC: 0.130).ConclusionsThe ML model based on ACD predicted the 1 year mortality of HF patients with high accuracy, and SMART-HF along with the ML model achieved superior performance to that of the conventional risk models. The SMART-HF model has the clear merit of easy operability even by non-healthcare providers with a user-friendly online interface (https://hfriskcalculator.herokuapp.com/). Risk models developed using SMART-HF may provide a novel modality for risk stratification of patients with HF.

Project description:BackgroundThe long-term prognosis of the cardio-metabolic and renal complications, in addition to mortality in patients with newly diagnosed pulmonary hypertension, are unclear. This study aims to develop a scalable predictive model in the form of an electronic frailty index (eFI) to predict different adverse outcomes.MethodsThis was a population-based cohort study of patients diagnosed with pulmonary hypertension between January 1st, 2000 and December 31st, 2017, in Hong Kong public hospitals. The primary outcomes were mortality, cardiovascular complications, renal diseases, and diabetes mellitus. The univariable and multivariable Cox regression analyses were applied to identify the significant risk factors, which were fed into the non-parametric random survival forest (RSF) model to develop an eFI.ResultsA total of 2,560 patients with a mean age of 63.4 years old (interquartile range: 38.0-79.0) were included. Over a follow-up, 1,347 died and 1,878, 437, and 684 patients developed cardiovascular complications, diabetes mellitus, and renal disease, respectively. The RSF-model-identified age, average readmission, anti-hypertensive drugs, cumulative length of stay, and total bilirubin were among the most important risk factors for predicting mortality. Pair-wise interactions of factors including diagnosis age, average readmission interval, and cumulative hospital stay were also crucial for the mortality prediction. Patients who developed all-cause mortality had higher values of the eFI compared to those who survived (P < 0.0001). An eFI ≥ 9.5 was associated with increased risks of mortality [hazard ratio (HR): 1.90; 95% confidence interval [CI]: 1.70-2.12; P < 0.0001]. The cumulative hazards were higher among patients who were 65 years old or above with eFI ≥ 9.5. Using the same cut-off point, the eFI predicted a long-term mortality over 10 years (HR: 1.71; 95% CI: 1.53-1.90; P < 0.0001). Compared to the multivariable Cox regression, the precision, recall, area under the curve (AUC), and C-index were significantly higher for RSF in the prediction of outcomes.ConclusionThe RSF models identified the novel risk factors and interactions for the development of complications and mortality. The eFI constructed by RSF accurately predicts the complications and mortality of patients with pulmonary hypertension, especially among the elderly.

Project description:BackgroundAcute heart failure (AHF) is associated with significant morbidity and mortality. Effective patient risk stratification is essential to guiding hospitalization decisions and the clinical management of AHF. Clinical decision support systems can be used to improve predictions of mortality made in emergency care settings for the purpose of AHF risk stratification. In this study, several models for the prediction of seven-day mortality among AHF patients were developed by applying machine learning techniques to retrospective patient data from 236,275 total emergency department (ED) encounters, 1881 of which were considered positive for AHF and were used for model training and testing. The models used varying subsets of age, sex, vital signs, and laboratory values. Model performance was compared to the Emergency Heart Failure Mortality Risk Grade (EHMRG) model, a commonly used system for prediction of seven-day mortality in the ED with similar (or, in some cases, more extensive) inputs. Model performance was assessed in terms of area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity.ResultsWhen trained and tested on a large academic dataset, the best-performing model and EHMRG demonstrated test set AUROCs of 0.84 and 0.78, respectively, for prediction of seven-day mortality. Given only measurements of respiratory rate, temperature, mean arterial pressure, and FiO2, one model produced a test set AUROC of 0.83. Neither a logistic regression comparator nor a simple decision tree outperformed EHMRG.ConclusionsA model using only the measurements of four clinical variables outperforms EHMRG in the prediction of seven-day mortality in AHF. With these inputs, the model could not be replaced by logistic regression or reduced to a simple decision tree without significant performance loss. In ED settings, this minimal-input risk stratification tool may assist clinicians in making critical decisions about patient disposition by providing early and accurate insights into individual patient's risk profiles.

Project description:BackgroundExisting operational definitions of frailty are personnel-costly and time-consuming, resulting in estimates with a small sample size that cannot be generalized to the population level. The objectives were to develop a multimorbidity frailty index using Taiwan's claim database, and to understand its ability to predict adverse event.MethodsThis is a retrospective cohort study. Subjects aged 65 to 100 years who have full National Health Insurance coverage in 2005 were included. We constructed the multimorbidity frailty index using cumulative deficit approach and categorized study population according to the multimorbidity frailty index quartiles: fit, mild frailty, moderate frailty and severe frailty. The multimorbidity frailty index included deficits from outpatient and inpatient diagnosis. Associations with all-cause mortality, unplanned hospitalization and intensive care unit admission were assessed using Kaplan-Meier curves and Cox regression analyses.ResultsThe multimorbidity frailty index incorporated 32 deficits, with mean multimorbidity frailty index score of 0.052 (standard deviation = 0.060) among 86,133 subjects included. Compared to subjects in fit category, subjects with severe frailty were associated with a 5.0-fold (adjusted hazard ratio, aHR 4.97; 95% confidence interval, 95% CI 4.49-5.50) increased risk of death at 1 year after adjusting for age and gender. Subjects with moderate frailty or mild frailty was associated with 3.1- (adjusted HR 3.08; 95% CI 2.80-3.39) or 1.9- (adjusted HR 1.86; 95% CI 1.71-2.01) folds increased risk, respectively.4.49-5.50). The risk trend of unplanned hospitalization and intensive care unit admission is similar among the study population. Besides, the association between the frailty categories and all three outcomes was slightly stronger among women.ConclusionThe multimorbidity frailty index was highly associated with all-cause mortality, unplanned hospitalization and ICU admission. It could serve as an efficient tool for stratifying older adults into different risk groups for planning care management programs.

Project description:BackgroundMortality prediction models are useful to guide clinical decision-making based on prognosis. The frailty index, which allows prognostication and personalized care planning, has not been directly compared with validated prognostic models.ObjectiveTo compare the discrimination of mortality, disability, falls, and hospitalization between a frailty index and validated prognostic indices.DesignSecondary Analysis of the National Health and Aging Trends Study.ParticipantsSeven thousand thirty-three Medicare beneficiaries 65 years or older.MeasurementsWe measured a deficit-accumulation frailty index, Schonberg index, and Lee index at the 2011 baseline assessment. Primary outcome was mortality at 5 years. Secondary outcomes were decline in activities of daily living (ADL), decline in instrumental activities of daily living (IADL), fall, and hospitalization at 1 year. We used C-statistics to compare discrimination between indices, adjusting for age and sex.ResultsThe study population included 4146 (44.8%) with age ≥ 75 years, with a median frailty index of 0.15 (interquartile range 0.09-0.25). A total of 1385 participants died (14.7%) and 2386 (35.2%) were lost to follow-up. Frailty, Schonberg, and Lee indices predicted mortality similarly: C-statistics (95% confidence interval) were 0.78 (0.77-0.80) for frailty index; 0.79 (0.78-0.81) for Schonberg index; and 0.78 (0.77-0.80) for Lee index. The frailty index had higher C-statistics for decline in ADL function (frailty index, 0.80 [0.78-0.83]; Schonberg, 0.74 [0.72-0.76]; Lee, 0.74 [0.71-0.77]) and falls (frailty index, 0.66 [0.65-0.68]; Schonberg, 0.61 [0.58-0.63]; Lee, 0.61 [0.59-0.63]). C-statistics were similar for decline in IADL function (frailty index, 0.61 [0.59-0.63]; Schonberg, 0.60 [0.59-0.62]; Lee, 0.60 [0.58-0.62]) and hospitalizations (frailty index, 0.68 [0.66-0.70]; Schonberg, 0.68 [0.66-0.69]; Lee, 0.65 [0.63-0.67]).ConclusionsA deficit-accumulation frailty index performs as well as prognostic indices for mortality prediction, and better predicts ADL disability and falls in community-dwelling older adults. Frailty assessment offers a unifying approach to risk stratification for key health outcomes relevant to older adults.