Dataset Information

Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research.

ABSTRACT: The replicability of research results has been a cause of increasing concern to the scientific community. The long-held belief that experimental standardization begets replicability has also been recently challenged, with the observation that the reduction of variability within studies can lead to idiosyncratic, lab-specific results that cannot be replicated. An alternative approach is to, instead, deliberately introduce heterogeneity, known as "heterogenization" of experimental design. Here, we explore a novel perspective in the heterogenization program in a meta-analysis of variability in observed phenotypic outcomes in both control and experimental animal models of ischemic stroke. First, by quantifying interindividual variability across control groups, we illustrate that the amount of heterogeneity in disease state (infarct volume) differs according to methodological approach, for example, in disease induction methods and disease models. We argue that such methods may improve replicability by creating diverse and representative distribution of baseline disease state in the reference group, against which treatment efficacy is assessed. Second, we illustrate how meta-analysis can be used to simultaneously assess efficacy and stability (i.e., mean effect and among-individual variability). We identify treatments that have efficacy and are generalizable to the population level (i.e., low interindividual variability), as well as those where there is high interindividual variability in response; for these, latter treatments translation to a clinical setting may require nuance. We argue that by embracing rather than seeking to minimize variability in phenotypic outcomes, we can motivate the shift toward heterogenization and improve both the replicability and generalizability of preclinical research.

SUBMITTER: Usui T

PROVIDER: S-EPMC8168858 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:The Stroke Preclinical Assessment Network (SPAN) is a multicenter preclinical trial platform using rodent models of transient focal cerebral ischemia to address translational failure in experimental stroke. In addition to centralized randomization and blinding and large samples, SPAN aimed to introduce heterogeneity to simulate the heterogeneity embodied in clinical trials for robust conclusions. Here, we report the heterogeneity introduced by allowing the 6 SPAN laboratories to vary most of the biological and experimental model variables and the impact of this heterogeneity on middle cerebral artery occlusion (MCAo) performance. We included the modified intention-to-treat population of the control mouse cohort of the first SPAN trial (n=421) and examined the biological and procedural independent variables and their covariance. We then determined their impact on the dependent variables cerebral blood flow drop during MCAo, time to achieve MCAo, and total anesthesia duration using multivariable analyses. We found heterogeneity in biological and procedural independent variables introduced mainly by the site. Consequently, all dependent variables also showed heterogeneity among the sites. Multivariable analyses with the site as a random effect variable revealed filament choice as an independent predictor of cerebral blood flow drop after MCAo. Comorbidity, sex, use of laser Doppler flow to monitor cerebral blood flow, days after trial onset, and maintaining anesthesia throughout the MCAo emerged as independent predictors of time to MCAo. Total anesthesia duration was predicted by most independent variables. We present with high granularity the heterogeneity introduced by the biological and model selections by the testing sites in the first trial of cerebroprotection in rodent transient filament MCAo by SPAN. Rather than trying to homogenize all variables across all sites, we embraced the heterogeneity to better approximate clinical trials. Awareness of the heterogeneity, its sources, and how it impacts the study performance may further improve the study design and statistical modeling for future multicenter preclinical trials.

Dataset Information

Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets