Dataset Information

Gradient boosted trees with individual explanations: An alternative to logistic regression for viability prediction in the first trimester of pregnancy.

ABSTRACT:

Background

Clinical models to predict first trimester viability are traditionally based on multivariable logistic regression (LR) which is not directly interpretable for non-statistical experts like physicians. Furthermore, LR requires complete datasets and pre-established variables specifications. In this study, we leveraged the internal non-linearity, feature selection and missing values handling mechanisms of machine learning algorithms, along with a post-hoc interpretability strategy, as potential advantages over LR for clinical modeling.

Methods

The dataset included 1154 patients with 2377 individual scans and was obtained from a prospective observational cohort study conducted at a hospital in London, UK, from March 2014 to May 2019. The data were split into a training (70%) and a test set (30%). Parsimonious and complete multivariable models were developed from two algorithms to predict first trimester viability at 11-14 weeks gestational age (GA): LR and light gradient boosted machine (LGBM). Missing values were handled by multiple imputation where appropriate. The SHapley Additive exPlanations (SHAP) framework was applied to derive individual explanations of the models.

Results

The parsimonious LGBM model had similar discriminative and calibration performance as the parsimonious LR (AUC 0.885 vs 0.860; calibration slope: 1.19 vs 1.18). The complete models did not outperform the parsimonious models. LGBM was robust to the presence of missing values and did not require multiple imputation unlike LR. Decision path plots and feature importance analysis revealed different algorithm behaviors despite similar predictive performance. The main driving variable from the LR model was the pre-specified interaction between fetal heart presence and mean sac diameter. The crown-rump length variable and a proxy variable reflecting the difference in GA between expected and observed GA were the two most important variables of LGBM. Finally, while variable interactions must be specified upfront with LR, several interactions were ranked by the SHAP framework among the most important features learned automatically by the LGBM algorithm.

Conclusions

Gradient boosted algorithms performed similarly to carefully crafted LR models in terms of discrimination and calibration for first trimester viability prediction. By handling multi-collinearity, missing values, feature selection and variable interactions internally, the gradient boosted trees algorithm, combined with SHAP, offers a serious alternative to traditional LR models.

SUBMITTER: Vaulet T

PROVIDER: S-EPMC8674730 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundWaist circumference is becoming recognized as a useful predictor of health risks in clinical research. However, clinical datasets tend to lack this measurement and self-reported values tend to be inaccurate. Predicting waist circumference from standard physical features could be a viable method for generating this information when it is missing or mitigating the impact of inaccurate self-reports. This study determined the degree to which the XGBoost advanced machine learning algorithm could build models that predict waist circumference from height, weight, calculated Body Mass Index, age, race/ethnicity and sex, whether they perform better than current models based on linear regression, and the relative importance of each feature in this prediction.MethodsWe trained tree-based models (via XGBoost gradient boosting) and linear models (via regression) to predict waist circumference from height, weight, Body Mass Index, age, race/ethnicity and sex (n = 60,740 participants). We created 10 iterations of each model, each using 90% of the dataset for training and the remaining 10% for testing performance (this group was different for each iteration). We calculated model performance and feature importance as an average across 10 iterations. We then externally validated the ensembled version of the top model.ResultsThe XGBoost model predicted waist circumference with a mean bias ± standard deviation of 0.0 ± 0.04 cm and a root mean squared error of 4.7 ± 0.05 cm, with performance varying slightly by sex and race/ethnicity. The XGBoost model showed varying degrees of improvement over linear regression models. The top 3 predictors were Body Mass Index, weight and race (Asian). External validation found that on average this model overestimated waist circumference by 4.65 cm in the United Kingdom population (mainly due to overprediction in females) and underestimated waist circumference by 1.7 cm in the Chinese population. The respective root mean squared errors were 7.7 cm and 7.1 cm.ConclusionsXGBoost-based models accurately predict waist circumference from standard physical features. Waist circumference prediction using this approach would be valuable for epidemiological research and beyond.

Project description:Pseudomonas syringae is a genetically diverse bacterial species complex responsible for numerous agronomically important crop diseases. Individual P. syringae isolates are assigned pathovar designations based on their host of isolation and the associated disease symptoms, and these pathovar designations are often assumed to reflect host specificity although this assumption has rarely been rigorously tested. Here we developed a rapid seed infection assay to measure the virulence of 121 diverse P. syringae isolates on common bean (Phaseolus vulgaris). This collection includes P. syringae phylogroup 2 (PG2) bean isolates (pathovar syringae) that cause bacterial spot disease and P. syringae phylogroup 3 (PG3) bean isolates (pathovar phaseolicola) that cause the more serious halo blight disease. We found that bean isolates in general were significantly more virulent on bean than non-bean isolates and observed no significant virulence difference between the PG2 and PG3 bean isolates. However, when we compared virulence within PGs we found that PG3 bean isolates were significantly more virulent than PG3 non-bean isolates, while there was no significant difference in virulence between PG2 bean and non-bean isolates. These results indicate that PG3 strains have a higher level of host specificity than PG2 strains. We then used gradient boosting machine learning to predict each strain's virulence on bean based on whole genome k-mers, type III secreted effector k-mers, and the presence/absence of type III effectors and phytotoxins. Our model performed best using whole genome data and was able to predict virulence with high accuracy (mean absolute error = 0.05). Finally, we functionally validated the model by predicting virulence for 16 strains and found that 15 (94%) had virulence levels within the bounds of estimated predictions. This study strengthens the hypothesis that P. syringae PG2 strains have evolved a different lifestyle than other P. syringae strains as reflected in their lower level of host specificity. It also acts as a proof-of-principle to demonstrate the power of machine learning for predicting host specific adaptation.