Dataset Information

Does including machine learning predictions in ALS clinical trial analysis improve statistical power?

ABSTRACT: OBJECTIVE:Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease which leads to progressive muscle weakness and eventually death. The increasing availability of large ALS clinical trial datasets have generated much interest in developing predictive models for disease progression. However, the utility of predictive modeling on clinical trial analysis has not been thoroughly evaluated. METHODS:We evaluated a predictive modeling approach for ALS disease progression measured by ALSFRS-R using the PRO-ACT database and validated our findings in a novel test set from a former clinical trial. We examined clinical trial scenarios where model predictions could improve statistical power for detecting treatment effects with simulated clinical trials. RESULTS:Models constructed with imputed PRO-ACT data have better external validation results than those fitted with complete observations. When fitted with imputed data, super learner (R2 = 0.71, MSPE = 19.7) and random forest (R2 = 0.70, MSPE = 19.6) have similar performance in the external validation and slightly outperform the linear mixed effects model (R2 = 0.69, MSPE = 20.5). Simulation studies suggest including machine learning predictions as a covariate in the analysis model of a 12-month clinical study can increase the trial's effective sample size by 16% when there is a hypothetical treatment effect of 25% reduction in ALSFRS-R mean rate of change. INTERPRETATION:Predictive modeling approaches for ALSFRS-R are able to explain a moderate amount of variability in longitudinal change, which is improved by robust missing data handling for baseline characteristics. Including ALSFRS-R post-baseline model prediction results as a covariate in the model for primary analysis may increase power under moderate treatment effects.

SUBMITTER: Zhou N

PROVIDER: S-EPMC7545604 | biostudies-literature | 2020 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Does including machine learning predictions in ALS clinical trial analysis improve statistical power?

Zhou Nina N Manser Paul P

Annals of clinical and translational neurology 20200830 10

<h4>Objective</h4>Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease which leads to progressive muscle weakness and eventually death. The increasing availability of large ALS clinical trial datasets have generated much interest in developing predictive models for disease progression. However, the utility of predictive modeling on clinical trial analysis has not been thoroughly evaluated.<h4>Methods</h4>We evaluated a predictive modeling approach for ALS disease progression measur ...[more]

PMID: 32862509

Similar Datasets

Project description:A key factor in designing randomized clinical trials is the sample size required to achieve a particular level of power to detect the benefit of a treatment. Sample size calculations depend upon the expected benefits of a treatment (effect size), the accuracy of measurement of the primary outcome, and the level of power specified by the investigators. In this study, we show that radiomic models, which leverage complex brain MRI patterns and machine learning, can be utilized in clinical trials with protocols that incorporate baseline MR imaging to significantly increase statistical power to detect treatment effects. Akin to the historical control paradigm, we propose to utilize a radiomic prediction model to generate a pseudo-control sample for each individual in the trial of interest. Because the variability of expected outcome across patients can mask our ability to detect treatment effects, we can increase the power to detect a treatment effect in a clinical trial by reducing that variability through using radiomic predictors as surrogates. We illustrate this method with simulations based on data from two cohorts in different neurologic diseases, Alzheimer's disease and glioblastoma multiforme. We present sample size requirements across a range of effect sizes using conventional analysis and models that include a radiomic predictor. For our Alzheimer's disease cohort, at an effect size of 0.35, total sample size requirements for 80% power declined from 246 to 212 for the endpoint cognitive decline. For our glioblastoma multiforme cohort, at an effect size of 1.65 with the endpoint survival time, total sample size requirements declined from 128 to 74. This methodology can decrease the required sample sizes by as much as 50%, depending on the strength of the radiomic predictor. The power of this method grows with increased accuracy of radiomic prediction, and furthermore, this method is most helpful when treatment effect sizes are small. Neuroimaging biomarkers are a powerful and increasingly common suite of tools that are, in many cases, highly predictive of disease outcomes. Here, we explore the possibility of using MRI-based radiomic biomarkers for the purpose of improving statistical power in clinical trials in the contexts of brain cancer and prodromal Alzheimer's disease. These methods can be applied to a broad range of neurologic diseases using a broad range of predictors of outcome to make clinical trials more efficient.

Dataset Information

Does including machine learning predictions in ALS clinical trial analysis improve statistical power?

Publications

Does including machine learning predictions in ALS clinical trial analysis improve statistical power?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets