Multivariate meta-analysis of multiple outcomes: characteristics and predictors of borrowing of strength from Cochrane reviews.
Ontology highlight
ABSTRACT: Objectives
Multivariate meta-analysis allows the joint synthesis of multiple outcomes accounting for their correlation. This enables borrowing of strength (BoS) across outcomes, which may lead to greater efficiency and even different conclusions compared to separate univariate meta-analyses. However, multivariate meta-analysis is complex to apply, so guidance is needed to flag (in advance of analysis) when the approach is most useful.Study design and setting
We use 43 Cochrane intervention reviews to empirically investigate the characteristics of meta-analysis datasets that are associated with a larger BoS statistic (from 0 to 100%) when applying a bivariate meta-analysis of binary outcomes.Results
Four characteristics were identified as strongly associated with BoS: the total number of studies, the number of studies with the outcome of interest, the percentage of studies missing the outcome of interest, and the largest absolute within-study correlation. Using these characteristics, we then develop a model for predicting BoS in a new dataset, which is shown to have good performance (an adjusted R2 of 50%). Applied examples are used to illustrate the use of the BoS prediction model.Conclusions
Cochrane reviewers mainly use univariate meta-analysis methods, but the identified characteristics associated with BoS and our subsequent prediction model for BoS help to flag when a multivariate meta-analysis may also be beneficial in Cochrane reviews with multiple binary outcomes. Extension to non-Cochrane reviews and other outcome types is still required.
SUBMITTER: Hattle M
PROVIDER: S-EPMC9316363 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BACKGROUND:Multivariate meta-analysis (MVMA) jointly synthesizes effects for multiple correlated outcomes. The MVMA model is potentially more difficult and time-consuming to apply than univariate models, so if its use makes little difference to parameter estimates, it could be argued that it is redundant. METHODS:We assessed the applicability and impact of MVMA in Cochrane Pregnancy and Childbirth (CPCB) systematic reviews. We applied MVMA to CPCB reviews published between 2011 and 2013 with two or more binary outcomes with at least three studies and compared findings with results of univariate meta-analyses. Univariate random effects meta-analysis models were fitted using restricted maximum likelihood estimation (REML). RESULTS:Eighty CPCB reviews were published. MVMA could not be applied in 70 of these reviews. MVMA was not feasible in three of the remaining 10 reviews because the appropriate models failed to converge. Estimates from MVMA agreed with those of univariate analyses in most of the other seven reviews. Statistical significance changed in two reviews: In one, this was due to a very small change in P value; in the other, the MVMA result for one outcome suggested that previous univariate results may be vulnerable to small-study effects and that the certainty of clinical conclusions needs consideration. CONCLUSIONS:MVMA methods can be applied only in a minority of reviews of interventions in pregnancy and childbirth and can be difficult to apply because of missing correlations or lack of convergence. Nevertheless, clinical and/or statistical conclusions from MVMA may occasionally differ from those from univariate analyses.
| S-EPMC6771837 | biostudies-other
Project description:Network meta-analysis (NMA) enables simultaneous comparison of multiple treatments while preserving randomisation. When summarising evidence to inform an economic evaluation, it is important that the analysis accurately reflects the dependency structure within the data, as correlations between outcomes may have implication for estimating the net benefit associated with treatment. A multivariate NMA offers a framework for evaluating multiple treatments across multiple outcome measures while accounting for the correlation structure between outcomes.The standard NMA model is extended to multiple outcome settings in two stages. In the first stage, information is borrowed across outcomes as well across studies through modelling the within-study and between-study correlation structure. In the second stage, we make use of the additional assumption that intervention effects are exchangeable between outcomes to predict effect estimates for all outcomes, including effect estimates on outcomes where evidence is either sparse or the treatment had not been considered by any one of the studies included in the analysis. We apply the methods to binary outcome data from a systematic review evaluating the effectiveness of nine home safety interventions on uptake of three poisoning prevention practices (safe storage of medicines, safe storage of other household products, and possession of poison centre control telephone number) in households with children. Analyses are conducted in WinBUGS using Markov Chain Monte Carlo (MCMC) simulations.Univariate and the first stage multivariate models produced broadly similar point estimates of intervention effects but the uncertainty around the multivariate estimates varied depending on the prior distribution specified for the between-study covariance structure. The second stage multivariate analyses produced more precise effect estimates while enabling intervention effects to be predicted for all outcomes, including intervention effects on outcomes not directly considered by the studies included in the analysis.Accounting for the dependency between outcomes in a multivariate meta-analysis may or may not improve the precision of effect estimates from a network meta-analysis compared to analysing each outcome separately.
| S-EPMC4142066 | biostudies-literature
Project description:OBJECTIVES:Network meta-analysis (NMA) is increasingly being used to synthesize direct and indirect evidence and help decision makers simultaneously compare multiple treatments. We empirically evaluate the incremental gain in precision achieved by incorporating indirect evidence in NMAs. STUDY DESIGN AND SETTING:We performed both network and pairwise meta-analyses on 40 published data sets of multiple-treatment comparisons. Their results were compared using the recently proposed borrowing of strength (BoS) statistic, which quantifies the percentage reduction in the uncertainty of the effect estimate when adding indirect evidence to an NMA. RESULTS:We analyzed 915 possible treatment comparisons, from which 484 (53%) had no direct evidence (BoS = 100%). In 181 comparisons with only one study contributing direct evidence, NMAs resulted in reduced precision (BoS < 0) and no appreciable improvements in precision (0 < BoS < 30%) for 104 (57.5%) and 23 (12.7%) comparisons, respectively. In 250 comparisons with at least two studies contributing direct evidence, NMAs provided increased precision with BoS ? 30% for 166 (66.4%) comparisons. CONCLUSION:Although NMAs have the potential to provide more precise results than those only based on direct evidence, the incremental gain may reliably occur only when at least two head-to-head studies are available and treatments are well connected. Researchers should routinely report and compare the results from both network and pairwise meta-analyses.
| S-EPMC6338506 | biostudies-literature
Project description:Background Sedentary lifestyle is a major risk factor for noncommunicable diseases such as cardiovascular diseases, cancer and diabetes. It has been estimated that approximately 3.2 million deaths each year are attributable to insufficient levels of physical activity. We evaluated the available evidence from Cochrane systematic reviews (CSRs) on the effectiveness of exercise/physical activity for various health outcomes. Methods Overview and meta-analysis. The Cochrane Library was searched from 01.01.2000 to issue 1, 2019. No language restrictions were imposed. Only CSRs of randomised controlled trials (RCTs) were included. Both healthy individuals, those at risk of a disease, and medically compromised patients of any age and gender were eligible. We evaluated any type of exercise or physical activity interventions; against any types of controls; and measuring any type of health-related outcome measures. The AMSTAR-2 tool for assessing the methodological quality of the included studies was utilised. Results Hundred and fifty CSRs met the inclusion criteria. There were 54 different conditions. Majority of CSRs were of high methodological quality. Hundred and thirty CSRs employed meta-analytic techniques and 20 did not. Limitations for studies were the most common reasons for downgrading the quality of the evidence. Based on 10 CSRs and 187 RCTs with 27,671 participants, there was a 13% reduction in mortality rates risk ratio (RR) 0.87 [95% confidence intervals (CI) 0.78 to 0.96]; I2 =?26.6%, [prediction interval (PI) 0.70, 1.07], median effect size (MES)?=?0.93 [interquartile range (IQR) 0.81, 1.00]. Data from 15 CSRs and 408 RCTs with 32,984 participants showed a small improvement in quality of life (QOL) standardised mean difference (SMD) 0.18 [95% CI 0.08, 0.28]; I2 =?74.3%; PI -0.18, 0.53], MES?=?0.20 [IQR 0.07, 0.39]. Subgroup analyses by the type of condition showed that the magnitude of effect size was the largest among patients with mental health conditions. Conclusion There is a plethora of CSRs evaluating the effectiveness of physical activity/exercise. The evidence suggests that physical activity/exercise reduces mortality rates and improves QOL with minimal or no safety concerns. Trial registration Registered in PROSPERO (CRD42019120295) on 10th January 2019. Supplementary Information The online version contains supplementary material available at 10.1186/s12889-020-09855-3.
| S-EPMC7670795 | biostudies-literature
Project description:In 1996, shortly after the founding of The Cochrane Collaboration, leading figures in test evaluation research established a Methods Group to focus on the relatively new and rapidly evolving methods for the systematic review of studies of diagnostic tests. Seven years later, the Collaboration decided it was time to develop a publication format and methodology for Diagnostic Test Accuracy (DTA) reviews, as well as the software needed to implement these reviews in The Cochrane Library. A meeting hosted by the German Cochrane Centre in 2004 brought together key methodologists in the area, many of whom became closely involved in the subsequent development of the methodological framework for DTA reviews. DTA reviews first appeared in The Cochrane Library in 2008 and are now an integral part of the work of the Collaboration.
| S-EPMC3851548 | biostudies-literature
Project description:BACKGROUND:There is growing interest in evaluating differences in healthcare interventions across routinely collected demographic characteristics. However, individual subgroup analyses in randomized controlled trials are often not prespecified, adjusted for multiple testing, or conducted using the appropriate statistical test for interaction, and therefore frequently lack credibility. Meta-analyses can be used to examine the validity of potential subgroup differences by collating evidence across trials. Here, we characterize the conduct and clinical translation of age-treatment subgroup analyses in Cochrane reviews. METHODS:For a random sample of 928 Cochrane intervention reviews of randomized trials, we determined how often subgroup analyses of age are reported, how often these analyses have a P?<?0.05 from formal interaction testing, how frequently subgroup differences first observed in an individual trial are later corroborated by other trials in the same meta-analysis, and how often statistically significant results are included in commonly used clinical management resources (BMJ Best Practice, UpToDate, Cochrane Clinical Answers, Google Scholar, and Google search). RESULTS:Among 928 Cochrane intervention reviews, 189 (20.4%) included plans to conduct age-treatment subgroup analyses. The vast majority (162 of 189, 85.7%) of the planned analyses were not conducted, commonly because of insufficient trial data. There were 22 reviews that conducted their planned age-treatment subgroup analyses, and another 3 reviews appeared to perform unplanned age-treatment subgroup analyses. These 25 (25 of 928, 2.7%) reviews conducted a total of 97 age-treatment subgroup analyses, of which 65 analyses (in 20 reviews) had non-overlapping subgroup levels. Among the 65 age-treatment subgroup analyses, 14 (21.5%) did not report any formal interaction testing. Seven (10.8%) reported P?<?0.05 from formal age-treatment interaction testing; however, none of these seven analyses were in reviews that discussed the potential biological rationale or clinical significance of the subgroup findings or had results that were included in common clinical practice resources. CONCLUSION:Age-treatment subgroup analyses in Cochrane intervention reviews were frequently planned but rarely conducted, and implications of detected interactions were not discussed in the reviews or mentioned in common clinical resources. When subgroup analyses are performed, authors should report the findings, compare the results to previous studies, and outline any potential impact on clinical care.
| S-EPMC6805640 | biostudies-literature
Project description:Multivariate meta-analysis allows the joint synthesis of effect estimates based on multiple outcomes from multiple studies, accounting for the potential correlations among them. However, standard methods for multivariate meta-analysis for multiple outcomes are restricted to problems where the within-study correlation is known or where individual participant data are available. This paper proposes an approach to approximating the within-study covariances based on information about likely correlations between underlying outcomes. We developed methods for both continuous and dichotomous data and for combinations of the two types. An application to a meta-analysis of treatments for stroke illustrates the use of the approximated covariance in multivariate meta-analysis with correlated outcomes.
| S-EPMC3618374 | biostudies-literature
Project description:PurposeThe aim of the present study was to assess both the credibility and strength of evidence arising from systematic reviews with meta-analyses of observational studies on handgrip strength and health outcomes.MethodsAn umbrella review of systematic reviews with meta-analyses of observational studies was conducted. We assessed meta-analyses of observational studies based on random-effect summary effect sizes and their p values, 95% prediction intervals, heterogeneity, small-study effects, and excess significance. We graded the evidence from convincing (Class I) to weak (Class IV).ResultsFrom 504 articles returned in a search of the literature, 8 systematic reviews were included in our review, with a total of 11 outcomes. Overall, nine of the 11 of the outcomes reported nominally significant summary results (p < 0.05), with 4 associations surviving the application of the more stringent p value (p < 10-6). No outcome presented convincing evidence. Three associations showed Class II evidence (i.e., highly suggestive): (1) higher handgrip values at baseline were associated with a minor reduction in mortality risk in the general population (n = 34 studies; sample size = 1,855,817; relative risk = 0.72, 95% confidence interval (95%CI): 0.67-0.78), (2) cardiovascular death risk in mixed populations (n = 15 studies; relative risk = 0.84, 95%CI: 0.78-0.91), and (3) incidence of disability (n = 7 studies; relative risk = 0.76, 95%CI: 0.66-0.87).ConclusionThe present results show that handgrip strength is a useful indicator for general health status and specifically for early all-cause and cardiovascular mortality, as well as disability. To further inform intervention strategies, future research is now required to fully understand mechanisms linking handgrip strength scores to these health outcomes.
| S-EPMC8167328 | biostudies-literature
Project description:BackgroundSystematic reviews with meta-analyses often contain many statistical tests. This multiplicity may increase the risk of type I error. Few attempts have been made to address the problem of statistical multiplicity in systematic reviews. Before the implications are properly considered, the size of the issue deserves clarification. Because of the emphasis on bias evaluation and because of the editorial processes involved, Cochrane reviews may contain more multiplicity than their non-Cochrane counterparts. This study measured the quantity of statistical multiplicity present in a population of systematic reviews and aimed to assess whether this quantity is different in Cochrane and non-Cochrane reviews.Methods/principal findingsWe selected all the systematic reviews published by the Cochrane Anaesthesia Review Group containing a meta-analysis and matched them with comparable non-Cochrane reviews. We counted the number of statistical tests done in each systematic review. The median number of tests overall was 10 (interquartile range (IQR) 6 to 18). The median was 12 in Cochrane and 8 in non-Cochrane reviews (difference in medians 4 (95% confidence interval (CI) 2.0-19.0). The proportion that used an assessment of risk of bias as a reason for doing extra analyses was 42% in Cochrane and 28% in non-Cochrane reviews (difference in proportions 14% (95% CI -8 to 36). The issue of multiplicity was addressed in 6% of all the reviews.Conclusion/significanceStatistical multiplicity in systematic reviews requires attention. We found more multiplicity in Cochrane reviews than in non-Cochrane reviews. Many of the reasons for the increase in multiplicity may well represent improved methodological approaches and greater transparency, but multiplicity may also cause an increased risk of spurious conclusions. Few systematic reviews, whether Cochrane or non-Cochrane, address the issue of multiplicity.
| S-EPMC3229598 | biostudies-literature
Project description:BACKGROUND:Information on subgroup assessments in systematic reviews (SR) of atrial fibrillation (AF) is limited. This review aims to describe subgroup analyses in AF SRs to inform the design of SRs and randomized trials as well as clinical practice. METHODS:We conducted a cross sectional meta-epidemiological study of Cochrane AF reviews by searching AF (including variants) in the title, abstract, or keyword field without date or language restrictions (Issue 9; September 2018). Two reviewers independently extracted study characteristics to summarize frequency of subgroups pre-specified and conducted and report credibility of subgroup effects claimed. RESULTS:Of 39 Cochrane reviews identified, 17 met inclusion criteria (including 168 reports of 127 randomized trials) and the majority (16; 94.1%) conducted meta-analysis of outcomes. Most (13; 76.5%) planned pre-specified subgroup analyses; 7 of which (41.2%) conducted subgroups. In these 7 reviews, 56 subgroups were planned, 17 (30.4%) conducted and 6 (10.7%) yielded subgroup effects. Variables such as co-morbid disease, stroke risk factors, prior stroke/transient ischemic attack, age, race, and sex represented 44% (24 subgroups) of all planned subgroups (8 conducted; 14.3%); however, information on covariate selection was lacking. Overall, more subgroups were planned than conducted (mean difference (95% CI) 2.3 (1.2-3.5, p?<?0.001)). Of all subgroups conducted, anticoagulant characteristics comprised a third of all subgroup effects (n?=?5, 35.7%). The credibility of subgroups identified (n?=?14) was assessed and less than half (43%) represented one of a small number of pre-specified hypothesis and rarely were effects seen within studies (7%). Of 5 reviews that reported subgroup effects, only 3 discussed subgroup effects as part of the overall conclusions; none discussed credibility of subgroup effects. CONCLUSIONS:This meta-epidemiological review of a subset of Cochrane AF reviews suggests that planning and reporting of subgroup analyses in AF reviews can be improved to better inform clinical management. Most pre-specified subgroup analyses were not performed, important variables (such as stroke, bleeding risk, and other comorbidities) were rarely examined and credibility of subgroup effects claimed was low. Future reviews should aim to identify important subgroups in their protocols and use recommended approaches to test subgroup effects in order to better support clinical decision-making.
| S-EPMC6814034 | biostudies-literature