Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics.
Ontology highlight
ABSTRACT: Objective
We investigated the associations between risk of bias judgments from Cochrane reviews for sequence generation, allocation concealment and blinding, and between-trial heterogeneity.Study design and setting
Bayesian hierarchical models were fitted to binary data from 117 meta-analyses, to estimate the ratio λ by which heterogeneity changes for trials at high/unclear risk of bias compared with trials at low risk of bias. We estimated the proportion of between-trial heterogeneity in each meta-analysis that could be explained by the bias associated with specific design characteristics.Results
Univariable analyses showed that heterogeneity variances were, on average, increased among trials at high/unclear risk of bias for sequence generation (λˆ 1.14, 95% interval: 0.57-2.30) and blinding (λˆ 1.74, 95% interval: 0.85-3.47). Trials at high/unclear risk of bias for allocation concealment were on average less heterogeneous (λˆ 0.75, 95% interval: 0.35-1.61). Multivariable analyses showed that a median of 37% (95% interval: 0-71%) heterogeneity variance could be explained by trials at high/unclear risk of bias for sequence generation, allocation concealment, and/or blinding. All 95% intervals for changes in heterogeneity were wide and included the null of no difference.Conclusion
Our interpretation of the results is limited by imprecise estimates. There is some indication that between-trial heterogeneity could be partially explained by reported design characteristics, and hence adjustment for bias could potentially improve accuracy of meta-analysis results.
SUBMITTER: Rhodes KM
PROVIDER: S-EPMC5828111 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BACKGROUND: Recently developed measures such as I2 and H allow the evaluation of the impact of heterogeneity in conventional meta-analyses. There has been no examination of the development of heterogeneity in the context of a cumulative meta-analysis. METHODS: Cumulative meta-analyses of five smoking cessation interventions (clonidine, nicotine replacement therapy using gum and patch, physician advice and acupuncture) were used to calculate I2 and H. These values were plotted by year of publication, control event rate and sample size to trace the development of heterogeneity over these covariates. RESULTS: The cumulative evaluation of heterogeneity varied according to the measure of heterogeneity used and the basis of cumulation. Plots produced from the calculations revealed areas of heterogeneity useful in the consideration of potential sources for further study. CONCLUSION: The examination of heterogeneity in conjunction with summary effect estimates in a cumulative meta-analysis offered valuable insight into the evolution of variation. Such information is not available in the context of conventional meta-analysis and has the potential to lead to the development of a richer picture of the effectiveness of interventions.
| S-EPMC487901 | biostudies-other
Project description:OBJECTIVE:Randomised controlled trials (RCT) are the gold standard to provide unbiased data. However, when patients have a treatment preference, randomisation may influence participation and outcomes (eg, external and internal validity). The aim of this study was to assess the influence of patients' preference in RCTs by analysing partially randomised patient preference trials (RPPT); an RCT and preference cohort combined. DESIGN:Systematic review and meta-analyses. DATA SOURCES:MEDLINE, Embase, PsycINFO and the Cochrane Library. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:RPPTs published between January 2005 and October 2018 reporting on allocation of patients to randomised and preference cohorts were included. DATA EXTRACTION AND SYNTHESIS:Two independent reviewers extracted data. The main outcomes were the difference in external validity (participation and baseline characteristics) and internal validity (lost to follow-up, crossover and the primary outcome) between the randomised and the preference cohort within each RPPT, compared in a meta-regression using a Wald test. Risk of bias was not assessed, as no quality assessment for RPPTs has yet been developed. RESULTS:In total, 117 of 3734 identified articles met screening criteria and 44 were eligible (24 873 patients). The participation rate in RPPTs was >95% in 14 trials (range: 48%-100%) and the randomisation refusal rate was >50% in 26 trials (range: 19%-99%). Higher education, female, older age, race and prior experience with one treatment arm were characteristics of patients declining randomisation. The lost to follow-up and cross-over rate were significantly higher in the randomised cohort compared with the preference cohort. Following the meta-analysis, the reported primary outcomes were comparable between both cohorts of the RPPTs, mean difference 0.093 (95% CI -0.178 to 0.364, p=0.502). CONCLUSIONS:Patients' preference led to a substantial proportion of a specific patient group refusing randomisation, while it did not influence the primary outcome within an RPPT. Therefore, RPPTs could increase external validity without compromising the internal validity compared with RCTs. PROSPERO REGISTRATION NUMBER:CRD42019094438.
| S-EPMC6797441 | biostudies-other
Project description:In a network meta-analysis, between-study heterogeneity variances are often very imprecisely estimated because data are sparse, so standard errors of treatment differences can be highly unstable. External evidence can provide informative prior distributions for heterogeneity and, hence, improve inferences. We explore approaches for specifying informative priors for multiple heterogeneity variances in a network meta-analysis. First, we assume equal heterogeneity variances across all pairwise intervention comparisons (approach 1); incorporating an informative prior for the common variance is then straightforward. Models allowing unequal heterogeneity variances are more realistic; however, care must be taken to ensure implied variance-covariance matrices remain valid. We consider three strategies for specifying informative priors for multiple unequal heterogeneity variances. Initially, we choose different informative priors according to intervention comparison type and assume heterogeneity to be proportional across comparison types and equal within comparison type (approach 2). Next, we allow all heterogeneity variances in the network to differ, while specifying a common informative prior for each. We explore two different approaches to this: placing priors on variances and correlations separately (approach 3) or using an informative inverse Wishart distribution (approach 4). Our methods are exemplified through application to two network metaanalyses. Appropriate informative priors are obtained from previously published evidence-based distributions for heterogeneity. Relevant prior information on between-study heterogeneity can be incorporated into network meta-analyses, without needing to assume equal heterogeneity across treatment comparisons. The approaches proposed will be beneficial in sparse data sets and provide more appropriate intervals for treatment differences than those based on imprecise heterogeneity estimates.
| S-EPMC6492109 | biostudies-literature
Project description:OBJECTIVES:To demonstrate how individual participant data (IPD) meta-analyses have impacted directly on the design and conduct of trials and highlight other advantages IPD might offer. STUDY DESIGN AND SETTING:Potential examples of the impact of IPD meta-analyses on trials were identified at an international workshop, attended by individuals with experience in the conduct of IPD meta-analyses and knowledge of trials in their respective clinical areas. Experts in the field who did not attend were asked to provide any further examples. We then examined relevant trial protocols, publications, and Web sites to verify the impacts of the IPD meta-analyses. A subgroup of workshop attendees sought further examples and identified other aspects of trial design and conduct that may inform IPD meta-analyses. RESULTS:We identified 52 examples of IPD meta-analyses thought to have had a direct impact on the design or conduct of trials. After screening relevant trial protocols and publications, we identified 28 instances where IPD meta-analyses had clearly impacted on trials. They have influenced the selection of comparators and participants, sample size calculations, analysis and interpretation of subsequent trials, and the conduct and analysis of ongoing trials, sometimes in ways that would not possible with systematic reviews of aggregate data. We identified additional potential ways that IPD meta-analyses could be used to influence trials. CONCLUSIONS:IPD meta-analysis could be better used to inform the design, conduct, analysis, and interpretation of trials.
| S-EPMC4635379 | biostudies-literature
Project description:BACKGROUND:Pairwise and network meta-analyses using fixed effect and random effects models are commonly applied to synthesize evidence from randomized controlled trials. The models differ in their assumptions and the interpretation of the results. The model choice depends on the objective of the analysis and knowledge of the included studies. Fixed effect models are often used because there are too few studies with which to estimate the between-study SD from the data alone. OBJECTIVES:The aim of this study was to propose a framework for eliciting an informative prior distribution for the between-study SD in a Bayesian random effects meta-analysis model to genuinely represent heterogeneity when data are sparse. METHODS:We developed an elicitation method using external information, such as empirical evidence and expert beliefs, on the "range" of treatment effects to infer the prior distribution for the between-study SD. We also developed the method to be implemented in R. RESULTS:The 3-stage elicitation approach allows uncertainty to be represented by a genuine prior distribution to avoid making misleading inferences. It is flexible to what judgments an expert can provide and is applicable to all types of outcome measures for which a treatment effect can be constructed on an additive scale. CONCLUSIONS:The choice between using a fixed effect or random effects meta-analysis model depends on the inferences required and not on the number of available studies. Our elicitation framework captures external evidence about heterogeneity and overcomes the assumption that studies are estimating the same treatment effect, thereby improving the quality of inferences in decision making.
| S-EPMC5950028 | biostudies-literature
Project description:BACKGROUND: Meta-analysis is the systematic and quantitative synthesis of effect sizes and the exploration of their diversity across different studies. Meta-analyses are increasingly applied to synthesize data from genome-wide association (GWA) studies and from other teams that try to replicate the genetic variants that emerge from such investigations. Between-study heterogeneity is important to document and may point to interesting leads. METHODOLOGY/PRINCIPAL FINDINGS: To exemplify these issues, we used data from three GWA studies on type 2 diabetes and their replication efforts where meta-analyses of all data using fixed effects methods (not incorporating between-study heterogeneity) have already been published. We considered 11 polymorphisms that at least one of the three teams has suggested as susceptibility loci for type 2 diabetes. The I2 inconsistency metric (measuring the amount of heterogeneity not due to chance) was different from 0 (no detectable heterogeneity) for 6 of the 11 genetic variants; inconsistency was moderate to very large (I2 = 32-77%) for 5 of them. For these 5 polymorphisms, random effects calculations incorporating between-study heterogeneity revealed more conservative p-values for the summary effects compared with the fixed effects calculations. These 5 associations were perused in detail to highlight potential explanations for between-study heterogeneity. These include identification of a marker for a correlated phenotype (e.g. FTO rs8050136 being associated with type 2 diabetes through its effect on obesity); differential linkage disequilibrium across studies of the identified genetic markers with the respective culprit polymorphisms (e.g., possibly the case for CDKAL1 polymorphisms or for rs9300039 and markers in linkage disequilibrium, as shown by additional studies); and potential bias. Results were largely similar, when we treated the discovery and replication data from each GWA investigation as separate studies. SIGNIFICANCE: Between-study heterogeneity is useful to document in the synthesis of data from GWA investigations and can offer valuable insights for further clarification of gene-disease associations.
| S-EPMC1950790 | biostudies-literature
Project description:The hypothesis that constitutive and inducible plant resistance against herbivores should trade-off because they use the same resources and impose costs to plant fitness has been postulated for a long time. Negative correlations between modes of deployment of resistance and defences have been observed across and within species in common garden experiments. It was therefore tested whether that pattern of resistance across genotypes follows a similar variation in patterns of gene expression and chemical defence production. Using the genetically tractable model Arabidopsis thaliana and different modes of induction, including the generalist herbivore Spodoptera littoralis, the specialist herbivore Pieris brassicae, and jasmonate application, constitutive and inducibility of resistance was measured across seven A. thaliana accessions that were previously selected based on constitutive levels of defence gene expression. According to theory, it was found that modes of resistance traded-off among accessions, particularly against S. littoralis, in which accessions investing in high constitutive resistance did not increase it substantially after attack and vice-versa. Accordingly, the average expression of eight genes involved in glucosinolate production negatively predicted larval growth across the seven accessions. Glucosinolate production and genes related to defence induction on healthy and herbivore-damaged plants were measured next. Surprisingly, only a partial correlation between glucosinolate production, gene expression, and the herbivore resistance results was found. These results suggest that the defence outcome of plants against herbivores goes beyond individual molecules or genes but stands on a complex network of interactions.
| S-EPMC4986863 | biostudies-literature
Project description:PurposeMost randomized controlled trials (RCTs) in patients with acute respiratory distress syndrome (ARDS) revealed indeterminate or conflicting study results. We aimed to systematically evaluate between-trial heterogeneity in reporting standards and trial outcome.MethodsA systematic review of RCTs published between 2000 and 2019 was performed including adult ARDS patients receiving lung-protective ventilation. A random-effects meta-regression model was applied to quantify heterogeneity (non-random variability) and to evaluate trial and patient characteristics as sources of heterogeneity.ResultsIn total, 67 RCTs were included. The 28-day control-group mortality rate ranged from 10 to 67% with large non-random heterogeneity (I2 = 88%, p < 0.0001). Reported baseline patient characteristics explained some of the outcome heterogeneity, but only six trials (9%) reported all four independently predictive variables (mean age, mean lung injury score, mean plateau pressure and mean arterial pH). The 28-day control group mortality adjusted for patient characteristics (i.e. the residual heterogeneity) ranged from 18 to 45%. Trials with significant benefit in the primary outcome reported a higher control group mortality than trials with an indeterminate outcome or harm (mean 28-day control group mortality: 44% vs. 28%; p = 0.001).ConclusionAmong ARDS RCTs in the lung-protective ventilation era, there was large variability in the description of baseline characteristics and significant unexplainable heterogeneity in 28-day control group mortality. These findings signify problems with the generalizability of ARDS research and underline the urgent need for standardized reporting of trial and baseline characteristics.
| S-EPMC7955690 | biostudies-literature
Project description:The positive relationship between habitat heterogeneity and species richness is a cornerstone of ecology. Recently, it was suggested that this relationship should be unimodal rather than linear due to a tradeoff between environmental heterogeneity and population sizes. Increased environmental heterogeneity will decrease effective habitat sizes, which in turn will increase the rate of local species extinctions. The occurrence of the unimodal richness-heterogeneity relationship at the habitat scale was confirmed in both empirical and theoretical studies. However, it is unclear whether it can occur at broader spatial scales, for meta-communities in diverse and patchy landscapes. Here, I used a spatially explicit meta-community model to quantify the roles of two species-level characteristics, niche width and immigration rates, on the type of the richness-heterogeneity relationship at the landscape scale. I found that both positive and unimodal richness-heterogeneity relationships can occur in meta-communities in patchy landscapes. The type of the relationship was affected by the interactions between inter-patch immigration rates and species' niche widths. Unimodal relationships were prominent in meta-communities comprising species with wide niches but low inter-patch immigration rates. In contrast, meta-communities consisting of species with narrow niches and high immigration rates exhibited positive relationships. Meta-communities comprising generalist species are therefore likely to exhibit unimodal richness-heterogeneity relationships as long as low immigration rates prevent rescue effects and patches are small. The richness-heterogeneity relationship at the landscape scale is dictated by species' niche widths and inter-patch immigration rates. These immigration rates, in turn, depend on the interaction between species dispersal capabilities and habitat connectivity, highlighting the roles of both species traits and landscape structure in generating the richness-heterogeneity relationship at the landscape scale.
| S-EPMC4359120 | biostudies-literature
Project description:BackgroundRandomized controlled trials (RCTs) with continuous outcomes usually only examine mean differences in response between trial arms. If the intervention has heterogeneous effects, then outcome variances will also differ between arms. Power of an individual trial to assess heterogeneity is lower than the power to detect the same size of main effect.MethodsWe describe several methods for assessing differences in variance in trial arms and apply them to a single trial with individual patient data and to meta-analyses using summary data. Where individual data are available, we use regression-based methods to examine the effects of covariates on variation. We present an additional method to meta-analyze differences in variances with summary data.ResultsIn the single trial, there was agreement between methods, and the difference in variance was largely due to differences in prevalence of depression at baseline. In two meta-analyses, most individual trials did not show strong evidence of a difference in variance between arms, with wide confidence intervals. However, both meta-analyses showed evidence of greater variance in the control arm, and in one example, this was perhaps because mean outcome in the control arm was higher.ConclusionsUsing meta-analysis, we overcame low power of individual trials to examine differences in variance using meta-analysis. Evidence of differences in variance should be followed up to identify potential effect modifiers and explore other possible causes such as varying compliance.
| S-EPMC8478324 | biostudies-literature