Dataset Information

Don't dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning.

ABSTRACT:

Background

Machine learning approaches have become increasingly popular modeling techniques, relying on data-driven heuristics to arrive at its solutions. Recent comparisons between these algorithms and traditional statistical modeling techniques have largely ignored the superiority gained by the former approaches due to involvement of model-building search algorithms. This has led to alignment of statistical and machine learning approaches with different types of problems and the under-development of procedures that combine their attributes. In this context, we hoped to understand the domains of applicability for each approach and to identify areas where a marriage between the two approaches is warranted. We then sought to develop a hybrid statistical-machine learning procedure with the best attributes of each.

Methods

We present three simple examples to illustrate when to use each modeling approach and posit a general framework for combining them into an enhanced logistic regression model building procedure that aids interpretation. We study 556 benchmark machine learning datasets to uncover when machine learning techniques outperformed rudimentary logistic regression models and so are potentially well-equipped to enhance them. We illustrate a software package, InteractionTransformer, which embeds logistic regression with advanced model building capacity by using machine learning algorithms to extract candidate interaction features from a random forest model for inclusion in the model. Finally, we apply our enhanced logistic regression analysis to two real-word biomedical examples, one where predictors vary linearly with the outcome and another with extensive second-order interactions.

Results

Preliminary statistical analysis demonstrated that across 556 benchmark datasets, the random forest approach significantly outperformed the logistic regression approach. We found a statistically significant increase in predictive performance when using hybrid procedures and greater clarity in the association with the outcome of terms acquired compared to directly interpreting the random forest output.

Conclusions

When a random forest model is closer to the true model, hybrid statistical-machine learning procedures can substantially enhance the performance of statistical procedures in an automated manner while preserving easy interpretation of the results. Such hybrid methods may help facilitate widespread adoption of machine learning techniques in the biomedical setting.

SUBMITTER: Levy JJ

PROVIDER: S-EPMC7325087 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Don't dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning.

Levy Joshua J JJ O'Malley A James AJ

BMC medical research methodology 20200629 1

<h4>Background</h4>Machine learning approaches have become increasingly popular modeling techniques, relying on data-driven heuristics to arrive at its solutions. Recent comparisons between these algorithms and traditional statistical modeling techniques have largely ignored the superiority gained by the former approaches due to involvement of model-building search algorithms. This has led to alignment of statistical and machine learning approaches with different types of problems and the under- ...[more]

PMID: 32600277

Similar Datasets

Project description:ObjectiveTo predict preterm birth in nulliparous women using logistic regression and machine learning.DesignPopulation-based retrospective cohort.ParticipantsNulliparous women (N = 112,963) with a singleton gestation who gave birth between 20-42 weeks gestation in Ontario hospitals from April 1, 2012 to March 31, 2014.MethodsWe used data during the first and second trimesters to build logistic regression and machine learning models in a "training" sample to predict overall and spontaneous preterm birth. We assessed model performance using various measures of accuracy including sensitivity, specificity, positive predictive value, negative predictive value, and area under the receiver operating characteristic curve (AUC) in an independent "validation" sample.ResultsDuring the first trimester, logistic regression identified 13 variables associated with preterm birth, of which the strongest predictors were diabetes (Type I: adjusted odds ratio (AOR): 4.21; 95% confidence interval (CI): 3.23-5.42; Type II: AOR: 2.68; 95% CI: 2.05-3.46) and abnormal pregnancy-associated plasma protein A concentration (AOR: 2.04; 95% CI: 1.80-2.30). During the first trimester, the maximum AUC was 60% (95% CI: 58-62%) with artificial neural networks in the validation sample. During the second trimester, 17 variables were significantly associated with preterm birth, among which complications during pregnancy had the highest AOR (13.03; 95% CI: 12.21-13.90). During the second trimester, the AUC increased to 65% (95% CI: 63-66%) with artificial neural networks in the validation sample. Including complications during the pregnancy yielded an AUC of 80% (95% CI: 79-81%) with artificial neural networks. All models yielded 94-97% negative predictive values for spontaneous PTB during the first and second trimesters.ConclusionAlthough artificial neural networks provided slightly higher AUC than logistic regression, prediction of preterm birth in the first trimester remained elusive. However, including data from the second trimester improved prediction to a moderate level by both logistic regression and machine learning approaches.

Project description:The objective of this work is to advance the understanding of helicopter accidents by examining and quantifying the association between helicopter-specific configurations (number of main rotor blades, number of engines, rotor diameter, and takeoff weight) and the likelihood of accidents. We leverage a dataset of 8,338 turboshaft helicopters in the U.S. civil fleet and 825 accidents from 2005 to 2015. We use the dataset to develop a logistic regression model using the method of purposeful selection, which we exploit for inferential purposes and highlight the novel insights it reveals. For example, one important question for the design and acquisition of helicopters is whether twin-engine turboshaft helicopters exhibit a smaller likelihood of accidents than their single-engine counterparts, all else being equal. The evidence-based result we derive indicates that the answer is contingent on other covariates, and that a tipping point exists in terms of the rotor diameter beyond which the likelihood of accidents of twin-engines is higher (worse) than that of their single-engine counterparts. Another important result derived here is the association between the number of main rotor blades and likelihood of accidents. We found that for single-engine turboshaft helicopters, the four-bladed ones are associated with the lowest likelihood of accidents. We also identified a clear coupling between the number of engines and the rotor diameter in terms of likelihood of accidents. In summary, we establish important relationships between the different helicopter configurations here considered and the likelihood of accident, but these are associations, not causal in nature. The causal pathway, if it exists, may be confounded or mediated by other variables not accounted for here. The results provided here lend themselves to a rich set of interpretive possibilities, and because of their significant safety implications they deserve careful attention from the rotorcraft community.

Dataset Information

Don't dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning.

Background

Methods

Results

Conclusions

Publications

Don't dismiss logistic regression: the case for sensible extraction of interactions in the era of machine learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets