Dataset Information

Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data.

ABSTRACT:

Objectives

Estimation of between-study heterogeneity is problematic in small meta-analyses. Bayesian meta-analysis is beneficial because it allows incorporation of external evidence on heterogeneity. To facilitate this, we provide empirical evidence on the likely heterogeneity between studies in meta-analyses relating to specific research settings.

Study design and setting

Our analyses included 6,492 continuous-outcome meta-analyses within the Cochrane Database of Systematic Reviews. We investigated the influence of meta-analysis settings on heterogeneity by modeling study data from all meta-analyses on the standardized mean difference scale. Meta-analysis setting was described according to outcome type, intervention comparison type, and medical area. Predictive distributions for between-study variance expected in future meta-analyses were obtained, which can be used directly as informative priors.

Results

Among outcome types, heterogeneity was found to be lowest in meta-analyses of obstetric outcomes. Among intervention comparison types, heterogeneity was lowest in meta-analyses comparing two pharmacologic interventions. Predictive distributions are reported for different settings. In two example meta-analyses, incorporating external evidence led to a more precise heterogeneity estimate.

Conclusion

Heterogeneity was influenced by meta-analysis characteristics. Informative priors for between-study variance were derived for each specific setting. Our analyses thus assist the incorporation of realistic prior information into meta-analyses including few studies.

SUBMITTER: Rhodes KM

PROVIDER: S-EPMC4270451 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundPower and precision are greater in meta-analyses than individual study analyses. However, dichotomisation of continuous outcomes in certain studies poses a problem as estimates from primary studies can only be pooled if they have a common outcome. Meta-analyses may include pooled summaries of either or both the continuous and dichotomous forms, and potentially have a different combination of studies for each depending on whether the outcome was dichotomised in the primary studies or not. This dual-outcome issue can lead to loss of power and/or selection bias. In this study we aimed to illustrate how dichotomisation of a continuous outcome in primary studies may result in biased estimates of pooled risk and odds ratios in meta-analysis using secondary analyses of published meta-analyses with the outcome, birthweight, which is commonly analysed both as continuous, and dichotomous (low birthweight: birthweight < 2,500 g).MethodsMeta-analyses published in January 2010 - December 2011 were obtained using searches in PubMed, Embase, Web of Science, and Cochrane Database of Systematic Reviews with the outcome birthweight. We used a distributional method to estimate the pooled odds/risk ratio of low birthweight and its standard error as a function of the data reported in the primary studies of the included meta-analyses where accessible.ResultsSeventy-six meta-analyses were identified. Thirty-seven percent (28/76) of the meta-analyses reported only the dichotomous form of the outcome while 26% (20/76) reported only the continuous form. In one meta-analysis (1/76), birthweight was analysed as continuous for one intervention and as binary for another and 36% (27/76) presented both dichotomous and continuous birthweight summaries. In meta-analyses with a continuous outcome, primary studies data were accessible in 39/48 and secondary analyses using the distributional approach provided consistent inferences for both the continuous and distributional estimates in 38/39.ConclusionThe distributional method applied in primary studies allows both a continuous and dichotomous outcome to be estimated providing consistent inferences. The use of this method in primary studies may restrict selective outcome bias in meta-analyses.

Project description:BackgroundMany meta-analyses contain only a small number of studies, which makes it difficult to estimate the extent of between-study heterogeneity. Bayesian meta-analysis allows incorporation of external evidence on heterogeneity, and offers advantages over conventional random-effects meta-analysis. To assist in this, we provide empirical evidence on the likely extent of heterogeneity in particular areas of health care.MethodsOur analyses included 14?886 meta-analyses from the Cochrane Database of Systematic Reviews. We classified each meta-analysis according to the type of outcome, type of intervention comparison and medical specialty. By modelling the study data from all meta-analyses simultaneously, using the log odds ratio scale, we investigated the impact of meta-analysis characteristics on the underlying between-study heterogeneity variance. Predictive distributions were obtained for the heterogeneity expected in future meta-analyses.ResultsBetween-study heterogeneity variances for meta-analyses in which the outcome was all-cause mortality were found to be on average 17% (95% CI 10-26) of variances for other outcomes. In meta-analyses comparing two active pharmacological interventions, heterogeneity was on average 75% (95% CI 58-95) of variances for non-pharmacological interventions. Meta-analysis size was found to have only a small effect on heterogeneity. Predictive distributions are presented for nine different settings, defined by type of outcome and type of intervention comparison. For example, for a planned meta-analysis comparing a pharmacological intervention against placebo or control with a subjectively measured outcome, the predictive distribution for heterogeneity is a log-normal (-2.13, 1.58(2)) distribution, which has a median value of 0.12. In an example of meta-analysis of six studies, incorporating external evidence led to a smaller heterogeneity estimate and a narrower confidence interval for the combined intervention effect.ConclusionsMeta-analysis characteristics were strongly associated with the degree of between-study heterogeneity, and predictive distributions for heterogeneity differed substantially across settings. The informative priors provided will be very beneficial in future meta-analyses including few studies.

Project description:BackgroundHeterogeneity has a key role in meta-analysis methods and can greatly affect conclusions. However, true levels of heterogeneity are unknown and often researchers assume homogeneity. We aim to: a) investigate the prevalence of unobserved heterogeneity and the validity of the assumption of homogeneity; b) assess the performance of various meta-analysis methods; c) apply the findings to published meta-analyses.Methods and findingsWe accessed 57,397 meta-analyses, available in the Cochrane Library in August 2012. Using simulated data we assessed the performance of various meta-analysis methods in different scenarios. The prevalence of a zero heterogeneity estimate in the simulated scenarios was compared with that in the Cochrane data, to estimate the degree of unobserved heterogeneity in the latter. We re-analysed all meta-analyses using all methods and assessed the sensitivity of the statistical conclusions. Levels of unobserved heterogeneity in the Cochrane data appeared to be high, especially for small meta-analyses. A bootstrapped version of the DerSimonian-Laird approach performed best in both detecting heterogeneity and in returning more accurate overall effect estimates. Re-analysing all meta-analyses with this new method we found that in cases where heterogeneity had originally been detected but ignored, 17-20% of the statistical conclusions changed. Rates were much lower where the original analysis did not detect heterogeneity or took it into account, between 1% and 3%.ConclusionsWhen evidence for heterogeneity is lacking, standard practice is to assume homogeneity and apply a simpler fixed-effect meta-analysis. We find that assuming homogeneity often results in a misleading analysis, since heterogeneity is very likely present but undetected. Our new method represents a small improvement but the problem largely remains, especially for very small meta-analyses. One solution is to test the sensitivity of the meta-analysis conclusions to assumed moderate and large degrees of heterogeneity. Equally, whenever heterogeneity is detected, it should not be ignored.