Dataset Information

Development and external evaluation of predictions models for mortality of COVID-19 patients using machine learning method.

ABSTRACT: To predict the mortality of patients with coronavirus disease 2019 (COVID-19). We collected clinical data of COVID-19 patients between January 18 and March 29 2020 in Wuhan, China . Gradient boosting decision tree (GBDT), logistic regression (LR) model, and simplified LR were built to predict the mortality of COVID-19. We also evaluated different models by computing area under curve (AUC), accuracy, positive predictive value (PPV), and negative predictive value (NPV) under fivefold cross-validation. A total of 2924 patients were included in our evaluation, with 257 (8.8%) died and 2667 (91.2%) survived during hospitalization. Upon admission, there were 21 (0.7%) mild cases, 2051 (70.1%) moderate case, 779 (26.6%) severe cases, and 73 (2.5%) critically severe cases. The GBDT model exhibited the highest fivefold AUC, which was 0.941, followed by LR (0.928) and LR-5 (0.913). The diagnostic accuracies of GBDT, LR, and LR-5 were 0.889, 0.868, and 0.887, respectively. In particular, the GBDT model demonstrated the highest sensitivity (0.899) and specificity (0.889). The NPV of all three models exceeded 97%, while their PPV values were relatively low, resulting in 0.381 for LR, 0.402 for LR-5, and 0.432 for GBDT. Regarding severe and critically severe cases, the GBDT model also performed the best with a fivefold AUC of 0.918. In the external validation test of the LR-5 model using 72 cases of COVID-19 from Brunei, leukomonocyte (%) turned to show the highest fivefold AUC (0.917), followed by urea (0.867), age (0.826), and SPO2 (0.704). The findings confirm that the mortality prediction performance of the GBDT is better than the LR models in confirmed cases of COVID-19. The performance comparison seems independent of disease severity.

Supplementary information

The online version contains supplementary material available at(10.1007/s00521-020-05592-1).

SUBMITTER: Li S

PROVIDER: S-EPMC7783503 | biostudies-literature | 2021 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and external evaluation of predictions models for mortality of COVID-19 patients using machine learning method.

Li Simin S Lin Yulan Y Zhu Tong T Fan Mengjie M Xu Shicheng S Qiu Weihao W Chen Can C Li Linfeng L Wang Yao Y Yan Jun J Wong Justin J Naing Lin L Xu Shabei S

Neural computing & applications 20210105 18

To predict the mortality of patients with coronavirus disease 2019 (COVID-19). We collected clinical data of COVID-19 patients between January 18 and March 29 2020 in Wuhan, China . Gradient boosting decision tree (GBDT), logistic regression (LR) model, and simplified LR were built to predict the mortality of COVID-19. We also evaluated different models by computing area under curve (AUC), accuracy, positive predictive value (PPV), and negative predictive value (NPV) under fivefold cross-validat ...[more]

PMID: 33424133

Similar Datasets

Project description:IntroductionTimely diagnosis of patients affected by an emerging infectious disease plays a crucial role in treating patients and avoiding disease spread. In prior research, we developed an approach by using machine learning (ML) algorithms to predict serious acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection based on clinical features of patients visiting an emergency department (ED) during the early coronavirus 2019 (COVID-19) pandemic. In this study, we aimed to externally validate this approach within a distinct ED population.MethodsTo create our training/validation cohort (model development) we collected data retrospectively from suspected COVID-19 patients at a US ED from February 23-May 12, 2020. Another dataset was collected as an external validation (testing) cohort from an ED in another country from May 12-June 15, 2021. Clinical features including patient demographics and triage information were used to train and test the models. The primary outcome was the confirmed diagnosis of COVID-19, defined as a positive reverse transcription polymerase chain reaction test result for SARS-CoV-2. We employed three different ML algorithms, including gradient boosting, random forest, and extra trees classifiers, to construct the predictive model. The predictive performances were evaluated with the area under the receiver operating characteristic curve (AUC) in the testing cohort.ResultsIn total, 580 and 946 ED patients were included in the training and testing cohorts, respectively. Of them, 98 (16.9%) and 180 (19.0%) were diagnosed with COVID-19. All the constructed ML models showed acceptable discrimination, as indicated by the AUC. Among them, random forest (0.785, 95% confidence interval [CI] 0.747-0.822) performed better than gradient boosting (0.774, 95% CI 0.739-0.811) and extra trees classifier (0.72, 95% CI 0.677-0.762). There was no significant difference between the constructed models.ConclusionOur study validates the use of ML for predicting COVID-19 in the ED and demonstrates its potential for predicting emerging infectious diseases based on models built by clinical features with temporal and spatial heterogeneity. This approach holds promise for scenarios where effective diagnostic tools for an emerging infectious disease may be lacking in the future.

Project description:BackgroundEchocardiography (echo) based machine learning (ML) models may be useful in identifying patients at high-risk of all-cause mortality.MethodsWe developed ML models (ResNet deep learning using echo videos and CatBoost gradient boosting using echo measurements) to predict 1-year, 3-year, and 5-year mortality. Models were trained on the Mackay dataset, Taiwan (6083 echos, 3626 patients) and validated in the Alberta HEART dataset, Canada (997 echos, 595 patients). We examined the performance of the models overall, and in subgroups (healthy controls, at risk of heart failure (HF), HF with reduced ejection fraction (HFrEF) and HF with preserved ejection fraction (HFpEF)). We compared the models' performance to the MAGGIC risk score, and examined the correlation between the models' predicted probability of death and baseline quality of life as measured by the Kansas City Cardiomyopathy Questionnaire (KCCQ).FindingsMortality rates at 1-, 3- and 5-years were 14.9%, 28.6%, and 42.5% in the Mackay cohort, and 3.0%, 10.3%, and 18.7%, in the Alberta HEART cohort. The ResNet and CatBoost models achieved area under the receiver-operating curve (AUROC) between 85% and 92% in internal validation. In external validation, the AUROCs for the ResNet (82%, 82%, and 78%) were significantly better than CatBoost (78%, 73%, and 75%), for 1-, 3- and 5-year mortality prediction respectively, with better or comparable performance to the MAGGIC score. ResNet models predicted higher probability of death in the HFpEF and HFrEF (30%-50%) subgroups than in controls and at risk patients (5%-20%). The predicted probabilities of death correlated with KCCQ scores (all p < 0.05).InterpretationEcho-based ML models to predict mortality had good internal and external validity, were generalizable, correlated with patients' quality of life, and are comparable to an established HF risk score. These models can be leveraged for automated risk stratification at point-of-care.FundingFunding for Alberta HEART was provided by an Alberta Innovates - Health Solutions Interdisciplinary Team Grant no. AHFMRITG 200801018. P.K. holds a Canadian Institutes of Health Research (CIHR) Sex and Gender Science Chair and a Heart & Stroke Foundation Chair in Cardiovascular Research. A.V. and V.S. received funding from the Mitacs Globalink Research Internship.

Project description:BackgroundDiabetic kidney disease (DKD) and diabetic retinopathy (DR) are major diabetic microvascular complications, contributing significantly to morbidity, disability, and mortality worldwide. The kidney and the eye, having similar microvascular structures and physiological and pathogenic features, may experience similar metabolic changes in diabetes.ObjectiveThis study aimed to use machine learning (ML) methods integrated with metabolic data to identify biomarkers associated with DKD and DR in a multiethnic Asian population with diabetes, as well as to improve the performance of DKD and DR detection models beyond traditional risk factors.MethodsWe used ML algorithms (logistic regression [LR] with Least Absolute Shrinkage and Selection Operator and gradient-boosting decision tree) to analyze 2772 adults with diabetes from the Singapore Epidemiology of Eye Diseases study, a population-based cross-sectional study conducted in Singapore (2004-2011). From 220 circulating metabolites and 19 risk factors, we selected the most important variables associated with DKD (defined as an estimated glomerular filtration rate <60 mL/min/1.73 m2) and DR (defined as an Early Treatment Diabetic Retinopathy Study severity level ≥20). DKD and DR detection models were developed based on the variable selection results and externally validated on a sample of 5843 participants with diabetes from the UK biobank (2007-2010). Machine-learned model performance (area under the receiver operating characteristic curve [AUC] with 95% CI, sensitivity, and specificity) was compared to that of traditional LR adjusted for age, sex, diabetes duration, hemoglobin A1c, systolic blood pressure, and BMI.ResultsSingapore Epidemiology of Eye Diseases participants had a median age of 61.7 (IQR 53.5-69.4) years, with 49.1% (1361/2772) being women, 20.2% (555/2753) having DKD, and 25.4% (685/2693) having DR. UK biobank participants had a median age of 61.0 (IQR 55.0-65.0) years, with 35.8% (2090/5843) being women, 6.7% (374/5570) having DKD, and 6.1% (355/5843) having DR. The ML algorithms identified diabetes duration, insulin usage, age, and tyrosine as the most important factors of both DKD and DR. DKD was additionally associated with cardiovascular disease history, antihypertensive medication use, and 3 metabolites (lactate, citrate, and cholesterol esters to total lipids ratio in intermediate-density lipoprotein), while DR was additionally associated with hemoglobin A1c, blood glucose, pulse pressure, and alanine. Machine-learned models for DKD and DR detection outperformed traditional LR models in both internal (AUC 0.838 vs 0.743 for DKD and 0.790 vs 0.764 for DR) and external validation (AUC 0.791 vs 0.691 for DKD and 0.778 vs 0.760 for DR).ConclusionsThis study highlighted diabetes duration, insulin usage, age, and circulating tyrosine as important factors in detecting DKD and DR. The integration of ML with biomedical big data enables biomarker discovery and improves disease detection beyond traditional risk factors.

Project description:ImportanceMachine-learning algorithms offer better predictive accuracy than traditional prognostic models but are too complex and opaque for clinical use.ObjectiveTo compare different machine learning methods in predicting overall mortality in cirrhosis and to use machine learning to select easily scored clinical variables for a novel cirrhosis prognostic model.Design, setting, and participantsThis prognostic study used a retrospective cohort of adult patients with cirrhosis or its complications seen in 130 hospitals and affiliated ambulatory clinics in the integrated, national Veterans Affairs health care system from October 1, 2011, to September 30, 2015. Patients were followed up through December 31, 2018. Data were analyzed from October 1, 2017, to May 31, 2020.ExposuresPotential predictors included demographic characteristics; liver disease etiology, severity, and complications; use of health care resources; comorbid conditions; and comprehensive laboratory and medication data. Patients were randomly selected for model development (66.7%) and validation (33.3%). Three different statistical and machine learning methods were evaluated: gradient descent boosting, logistic regression with least absolute shrinkage and selection operator (LASSO) regularization, and logistic regression with LASSO constrained to select no more than 10 predictors (partial pathway model). Predictor inclusion and model performance were evaluated in a 5-fold cross-validation. Last, the predictors identified in the most parsimonious (the partial path) model were refit using maximum-likelihood estimation (Cirrhosis Mortality Model [CiMM]), and its predictive performance was compared with that of the widely used Model for End Stage Liver Disease with sodium (MELD-Na) score.Main outcomes and measuresAll-cause mortality.ResultsOf the 107 939 patients with cirrhosis (mean [SD] age, 62.7 [9.6] years; 96.6% male; 66.3% white, 18.4% African American), the annual mortality rate ranged from 8.8% to 15.3%. In total, 32.7% of patients died within 3 years, and 46.2% died within 5 years after the index date. Models predicting 1-year mortality had good discrimination for the gradient descent boosting (area under the receiver operating characteristics curve [AUC], 0.81; 95% CI, 0.80-0.82), logistic regression with LASSO regularization (AUC, 0.78; 95% CI, 0.77-0.79), and the partial path logistic model (AUC, 0.78; 95% CI, 0.76-0.78). All models showed good calibration. The final CiMM model with machine learning-derived clinical variables offered significantly better discrimination than the MELD-Na score, with AUCs of 0.78 (95% CI, 0.77-0.79) vs 0.67 (95% CI, 0.66-0.68) for 1-year mortality, respectively (DeLong z = 17.00; P < .001).Conclusions and relevanceIn this study, simple machine learning techniques performed as well as the more advanced ensemble gradient boosting. Using the clinical variables identified from simple machine learning in a cirrhosis mortality model produced a new score more transparent than machine learning and more predictive than the MELD-Na score.

Project description:IntroductionPatients with sepsis who present to an emergency department (ED) have highly variable underlying disease severity, and can be categorized from low to high risk. Development of a risk stratification tool for these patients is important for appropriate triage and early treatment. The aim of this study was to develop machine learning models predicting 31-day mortality in patients presenting to the ED with sepsis and to compare these to internal medicine physicians and clinical risk scores.MethodsA single-center, retrospective cohort study was conducted amongst 1,344 emergency department patients fulfilling sepsis criteria. Laboratory and clinical data that was available in the first two hours of presentation from these patients were randomly partitioned into a development (n = 1,244) and validation dataset (n = 100). Machine learning models were trained and evaluated on the development dataset and compared to internal medicine physicians and risk scores in the independent validation dataset. The primary outcome was 31-day mortality.ResultsA number of 1,344 patients were included of whom 174 (13.0%) died. Machine learning models trained with laboratory or a combination of laboratory + clinical data achieved an area-under-the ROC curve of 0.82 (95% CI: 0.80-0.84) and 0.84 (95% CI: 0.81-0.87) for predicting 31-day mortality, respectively. In the validation set, models outperformed internal medicine physicians and clinical risk scores in sensitivity (92% vs. 72% vs. 78%;p<0.001,all comparisons) while retaining comparable specificity (78% vs. 74% vs. 72%;p>0.02). The model had higher diagnostic accuracy with an area-under-the-ROC curve of 0.85 (95%CI: 0.78-0.92) compared to abbMEDS (0.63,0.54-0.73), mREMS (0.63,0.54-0.72) and internal medicine physicians (0.74,0.65-0.82).ConclusionMachine learning models outperformed internal medicine physicians and clinical risk scores in predicting 31-day mortality. These models are a promising tool to aid in risk stratification of patients presenting to the ED with sepsis.

Dataset Information

Development and external evaluation of predictions models for mortality of COVID-19 patients using machine learning method.

Supplementary information

Publications

Development and external evaluation of predictions models for mortality of COVID-19 patients using machine learning method.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets