Dataset Information

Comparison of aggregate and individual participant data approaches to meta-analysis of randomised trials: An observational study.

ABSTRACT:

Background

It remains unclear when standard systematic reviews and meta-analyses that rely on published aggregate data (AD) can provide robust clinical conclusions. We aimed to compare the results from a large cohort of systematic reviews and meta-analyses based on individual participant data (IPD) with meta-analyses of published AD, to establish when the latter are most likely to be reliable and when the IPD approach might be required.

Methods and findings

We used 18 cancer systematic reviews that included IPD meta-analyses: all of those completed and published by the Meta-analysis Group of the MRC Clinical Trials Unit from 1991 to 2010. We extracted or estimated hazard ratios (HRs) and standard errors (SEs) for survival from trial reports and compared these with IPD equivalents at both the trial and meta-analysis level. We also extracted or estimated the number of events. We used paired t tests to assess whether HRs and SEs from published AD differed on average from those from IPD. We assessed agreement, and whether this was associated with trial or meta-analysis characteristics, using the approach of Bland and Altman. The 18 systematic reviews comprised 238 unique trials or trial comparisons, including 37,082 participants. A HR and SE could be generated for 127 trials, representing 53% of the trials and approximately 79% of eligible participants. On average, trial HRs derived from published AD were slightly more in favour of the research interventions than those from IPD (HRAD to HRIPD ratio = 0.95, p = 0.007), but the limits of agreement show that for individual trials, the HRs could deviate substantially. These limits narrowed with an increasing number of participants (p < 0.001) or a greater number (p < 0.001) or proportion (p < 0.001) of events in the AD. On average, meta-analysis HRs from published AD slightly tended to favour the research interventions whether based on fixed-effect (HRAD to HRIPD ratio = 0.97, p = 0.088) or random-effects (HRAD to HRIPD ratio = 0.96, p = 0.044) models, but the limits of agreement show that for individual meta-analyses, agreement was much more variable. These limits tended to narrow with an increasing number (p = 0.077) or proportion of events (p = 0.11) in the AD. However, even when the information size of the AD was large, individual meta-analysis HRs could still differ from their IPD equivalents by a relative 10% in favour of the research intervention to 5% in favour of control. We utilised the results to construct a decision tree for assessing whether an AD meta-analysis includes sufficient information, and when estimates of effects are most likely to be reliable. A lack of power at the meta-analysis level may have prevented us identifying additional factors associated with the reliability of AD meta-analyses, and we cannot be sure that our results are generalisable to all outcomes and effect measures.

Conclusions

In this study we found that HRs from published AD were most likely to agree with those from IPD when the information size was large. Based on these findings, we provide guidance for determining systematically when standard AD meta-analysis will likely generate robust clinical conclusions, and when the IPD approach will add considerable value.

SUBMITTER: Tierney JF

PROVIDER: S-EPMC6993967 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundIncreasingly in network meta-analysis (NMA), there is a need to incorporate non-randomised evidence to estimate relative treatment effects, and in particular in cases with limited randomised evidence, sometimes resulting in disconnected networks of treatments. When combining different sources of data, complex NMA methods are required to address issues associated with participant selection bias, incorporating single-arm trials (SATs), and synthesising a mixture of individual participant data (IPD) and aggregate data (AD). We develop NMA methods which synthesise data from SATs and randomised controlled trials (RCTs), using a mixture of IPD and AD, for a dichotomous outcome.MethodsWe propose methods under both contrast-based (CB) and arm-based (AB) parametrisations, and extend the methods to allow for both within- and across-trial adjustments for covariate effects. To illustrate the methods, we use an applied example investigating the effectiveness of biologic disease-modifying anti-rheumatic drugs for rheumatoid arthritis (RA). We applied the methods to a dataset obtained from a literature review consisting of 14 RCTs and an artificial dataset consisting of IPD from two SATs and AD from 12 RCTs, where the artificial dataset was created by removing the control arms from the only two trials assessing tocilizumab in the original dataset.ResultsWithout adjustment for covariates, the CB method with independent baseline response parameters (CBunadjInd) underestimated the effectiveness of tocilizumab when applied to the artificial dataset compared to the original dataset, albeit with significant overlap in posterior distributions for treatment effect parameters. The CB method with exchangeable baseline response parameters produced effectiveness estimates in agreement with CBunadjInd, when the predicted baseline response estimates were similar to the observed baseline response. After adjustment for RA duration, there was a reduction in across-trial heterogeneity in baseline response but little change in treatment effect estimates.ConclusionsOur findings suggest incorporating SATs in NMA may be useful in some situations where a treatment is disconnected from a network of comparator treatments, due to a lack of comparative evidence, to estimate relative treatment effects. The reliability of effect estimates based on data from SATs may depend on adjustment for covariate effects, although further research is required to understand this in more detail.

Project description:Parenteral anticoagulants may improve outcomes in patients with cancer by reducing risk of venous thromboembolic disease and through a direct antitumour effect. Study-level systematic reviews indicate a reduction in venous thromboembolism and provide moderate confidence that a small survival benefit exists. It remains unclear if any patient subgroups experience potential benefits.First, we will perform a comprehensive systematic search of MEDLINE, EMBASE and The Cochrane Library, hand search scientific conference abstracts and check clinical trials registries for randomised control trials of participants with solid cancers who are administered parenteral anticoagulants. We anticipate identifying at least 15 trials, exceeding 9000 participants. Second, we will perform an individual participant data meta-analysis to explore the magnitude of survival benefit and address whether subgroups of patients are more likely to benefit from parenteral anticoagulants. All analyses will follow the intention-to-treat principle. For our primary outcome, mortality, we will use multivariable hierarchical models with patient-level variables as fixed effects and a categorical trial variable as a random effect. We will adjust analysis for important prognostic characteristics. To investigate whether intervention effects vary by predefined subgroups of patients, we will test interaction terms in the statistical model. Furthermore, we will develop a risk-prediction model for venous thromboembolism, with a focus on control patients of randomised trials.Aside from maintaining participant anonymity, there are no major ethical concerns. This will be the first individual participant data meta-analysis addressing heparin use among patients with cancer and will directly influence recommendations in clinical practice guidelines. Major cancer guideline development organisations will use eventual results to inform their guideline recommendations. Several knowledge users will disseminate results through presentations at clinical rounds as well as national and international conferences. We will prepare an evidence brief and facilitate dialogue to engage policymakers and stakeholders in acting on findings.PROSPERO CRD42013003526.

Project description:IntroductionMeta-analysis traditionally uses aggregate data from published reports. Individual Participant Data (IPD) meta-analysis, which obtains and synthesizes participant-level data, is potentially more informative, but resource-intensive. The impact on the findings of meta-analyses using IPD in comparison with aggregate data has rarely been formally evaluated.MethodsWe conducted a secondary analysis of a Cochrane systematic review of skincare interventions for preventing eczema and food allergy in infants to identify the impact of the analytical choice on the review's findings. We used aggregate data meta-analysis only and contrasted the results against those of the originally published IPD meta-analysis. All meta-analysis used random effects inverse variance models. Certainty of evidence was evaluated using GRADE.ResultsThe pooled treatment effects for the Cochrane systematic review's co-primary outcomes of eczema and food allergy were similar in IPD meta-analysis (eczema RR 1.03, 95% CI 0.81, 1.31; I2 41%, 7 studies 3075 participants), and aggregate meta-analysis (eczema RR 1.01 95% CI 0.77, 1.33; I2 53%, 7 studies, 3089 participants). In aggregate meta-analysis, the statistical heterogeneity could not be explained but using IPD it was explained by one trial which used a different, bathing intervention. For IPD meta-analysis, risk of bias was assessed as lower and more adverse event data were available compared with aggregate meta-analysis. This resulted in higher certainty of evidence, especially for adverse events. IPD meta-analysis enabled analysis of treatment interactions by age and hereditary eczema risk; and analysis of the effect of treatment adherence using pooled complier-adjusted-causal-effect analysis, none of which was possible in aggregate meta-analysis.ConclusionsFor this systematic review, IPD did not significantly change primary outcome risk ratios compared with aggregate data meta-analysis. However, certainty of evidence, safety outcomes, subgroup and adherence analyses were significantly different using IPD. This demonstrates benefits of adopting an IPD approach to meta-analysis.

Project description:OBJECTIVE: To assess the effect of the FTO genotype on weight loss after dietary, physical activity, or drug based interventions in randomised controlled trials. DESIGN: Systematic review and random effects meta-analysis of individual participant data from randomised controlled trials. DATA SOURCES: Ovid Medline, Scopus, Embase, and Cochrane from inception to November 2015. ELIGIBILITY CRITERIA FOR STUDY SELECTION: Randomised controlled trials in overweight or obese adults reporting reduction in body mass index, body weight, or waist circumference by FTO genotype (rs9939609 or a proxy) after dietary, physical activity, or drug based interventions. Gene by treatment interaction models were fitted to individual participant data from all studies included in this review, using allele dose coding for genetic effects and a common set of covariates. Study level interactions were combined using random effect models. Metaregression and subgroup analysis were used to assess sources of study heterogeneity. RESULTS: We identified eight eligible randomised controlled trials for the systematic review and meta-analysis (n=9563). Overall, differential changes in body mass index, body weight, and waist circumference in response to weight loss intervention were not significantly different between FTO genotypes. Sensitivity analyses indicated that differential changes in body mass index, body weight, and waist circumference by FTO genotype did not differ by intervention type, intervention length, ethnicity, sample size, sex, and baseline body mass index and age category. CONCLUSIONS: We have observed that carriage of the FTO minor allele was not associated with differential change in adiposity after weight loss interventions. These findings show that individuals carrying the minor allele respond equally well to dietary, physical activity, or drug based weight loss interventions and thus genetic predisposition to obesity associated with the FTO minor allele can be at least partly counteracted through such interventions. SYSTEMATIC REVIEW REGISTRATION: PROSPERO CRD42015015969.

Project description:BackgroundLow-dose aspirin is of definite and substantial net benefit for many people who already have occlusive vascular disease. We have assessed the benefits and risks in primary prevention.MethodsWe undertook meta-analyses of serious vascular events (myocardial infarction, stroke, or vascular death) and major bleeds in six primary prevention trials (95,000 individuals at low average risk, 660,000 person-years, 3554 serious vascular events) and 16 secondary prevention trials (17,000 individuals at high average risk, 43,000 person-years, 3306 serious vascular events) that compared long-term aspirin versus control. We report intention-to-treat analyses of first events during the scheduled treatment period.FindingsIn the primary prevention trials, aspirin allocation yielded a 12% proportional reduction in serious vascular events (0.51% aspirin vs 0.57% control per year, p=0.0001), due mainly to a reduction of about a fifth in non-fatal myocardial infarction (0.18%vs 0.23% per year, p<0.0001). The net effect on stroke was not significant (0.20%vs 0.21% per year, p=0.4: haemorrhagic stroke 0.04%vs 0.03%, p=0.05; other stroke 0.16%vs 0.18% per year, p=0.08). Vascular mortality did not differ significantly (0.19%vs 0.19% per year, p=0.7). Aspirin allocation increased major gastrointestinal and extracranial bleeds (0.10%vs 0.07% per year, p<0.0001), and the main risk factors for coronary disease were also risk factors for bleeding. In the secondary prevention trials, aspirin allocation yielded a greater absolute reduction in serious vascular events (6.7%vs 8.2% per year, p<0.0001), with a non-significant increase in haemorrhagic stroke but reductions of about a fifth in total stroke (2.08%vs 2.54% per year, p=0.002) and in coronary events (4.3%vs 5.3% per year, p<0.0001). In both primary and secondary prevention trials, the proportional reductions in the aggregate of all serious vascular events seemed similar for men and women.InterpretationIn primary prevention without previous disease, aspirin is of uncertain net value as the reduction in occlusive events needs to be weighed against any increase in major bleeds. Further trials are in progress.FundingUK Medical Research Council, British Heart Foundation, Cancer Research UK, and the European Community Biomed Programme.

Project description:BackgroundThe vascular and gastrointestinal effects of non-steroidal anti-inflammatory drugs (NSAIDs), including selective COX-2 inhibitors (coxibs) and traditional non-steroidal anti-inflammatory drugs (tNSAIDs), are not well characterised, particularly in patients at increased risk of vascular disease. We aimed to provide such information through meta-analyses of randomised trials.MethodsWe undertook meta-analyses of 280 trials of NSAIDs versus placebo (124,513 participants, 68,342 person-years) and 474 trials of one NSAID versus another NSAID (229,296 participants, 165,456 person-years). The main outcomes were major vascular events (non-fatal myocardial infarction, non-fatal stroke, or vascular death); major coronary events (non-fatal myocardial infarction or coronary death); stroke; mortality; heart failure; and upper gastrointestinal complications (perforation, obstruction, or bleed).FindingsMajor vascular events were increased by about a third by a coxib (rate ratio [RR] 1·37, 95% CI 1·14-1·66; p=0·0009) or diclofenac (1·41, 1·12-1·78; p=0·0036), chiefly due to an increase in major coronary events (coxibs 1·76, 1·31-2·37; p=0·0001; diclofenac 1·70, 1·19-2·41; p=0·0032). Ibuprofen also significantly increased major coronary events (2·22, 1·10-4·48; p=0·0253), but not major vascular events (1·44, 0·89-2·33). Compared with placebo, of 1000 patients allocated to a coxib or diclofenac for a year, three more had major vascular events, one of which was fatal. Naproxen did not significantly increase major vascular events (0·93, 0·69-1·27). Vascular death was increased significantly by coxibs (1·58, 99% CI 1·00-2·49; p=0·0103) and diclofenac (1·65, 0·95-2·85, p=0·0187), non-significantly by ibuprofen (1·90, 0·56-6·41; p=0·17), but not by naproxen (1·08, 0·48-2·47, p=0·80). The proportional effects on major vascular events were independent of baseline characteristics, including vascular risk. Heart failure risk was roughly doubled by all NSAIDs. All NSAID regimens increased upper gastrointestinal complications (coxibs 1·81, 1·17-2·81, p=0·0070; diclofenac 1·89, 1·16-3·09, p=0·0106; ibuprofen 3·97, 2·22-7·10, p<0·0001; and naproxen 4·22, 2·71-6·56, p<0·0001).InterpretationThe vascular risks of high-dose diclofenac, and possibly ibuprofen, are comparable to coxibs, whereas high-dose naproxen is associated with less vascular risk than other NSAIDs. Although NSAIDs increase vascular and gastrointestinal risks, the size of these risks can be predicted, which could help guide clinical decision making.FundingUK Medical Research Council and British Heart Foundation.

Project description:BackgroundStatin therapy is effective for the prevention of atherosclerotic cardiovascular disease and is widely prescribed, but there are persisting concerns that statin therapy might frequently cause muscle pain or weakness. We aimed to address these through an individual participant data meta-analysis of all recorded adverse muscle events in large, long-term, randomised, double-blind trials of statin therapy.MethodsRandomised trials of statin therapy were eligible if they aimed to recruit at least 1000 participants with a scheduled treatment duration of at least 2 years, and involved a double-blind comparison of statin versus placebo or of a more intensive versus a less intensive statin regimen. We analysed individual participant data from 19 double-blind trials of statin versus placebo (n=123 940) and four double-blind trials of a more intensive versus a less intensive statin regimen (n=30 724). Standard inverse-variance-weighted meta-analyses of the effects on muscle outcomes were conducted according to a prespecified protocol.FindingsAmong 19 placebo-controlled trials (mean age 63 years [SD 8], with 34 533 [27·9%] women, 59 610 [48·1%] participants with previous vascular disease, and 22 925 [18·5%] participants with diabetes), during a weighted average median follow-up of 4·3 years, 16 835 (27·1%) allocated statin versus 16 446 (26·6%) allocated placebo reported muscle pain or weakness (rate ratio [RR] 1·03; 95% CI 1·01-1·06). During year 1, statin therapy produced a 7% relative increase in muscle pain or weakness (1·07; 1·04-1·10), corresponding to an absolute excess rate of 11 (6-16) events per 1000 person-years, which indicates that only one in 15 ([1·07-1·00]/1·07) of these muscle-related reports by participants allocated to statin therapy were actually due to the statin. After year 1, there was no significant excess in first reports of muscle pain or weakness (0·99; 0·96-1·02). For all years combined, more intensive statin regimens (ie, 40-80 mg atorvastatin or 20-40 mg rosuvastatin once per day) yielded a higher RR than less intensive or moderate-intensity regimens (1·08 [1·04-1·13] vs 1·03 [1·00-1·05]) compared with placebo, and a small excess was present (1·05 [0·99-1·12]) for more intensive regimens after year 1. There was no clear evidence that the RR differed for different statins, or in different clinical circumstances. Statin therapy yielded a small, clinically insignificant increase in median creatine kinase values of approximately 0·02 times the upper limit of normal.InterpretationStatin therapy caused a small excess of mostly mild muscle pain. Most (>90%) of all reports of muscle symptoms by participants allocated statin therapy were not due to the statin. The small risks of muscle symptoms are much lower than the known cardiovascular benefits. There is a need to review the clinical management of muscle symptoms in patients taking a statin.FundingBritish Heart Foundation, Medical Research Council, Australian National Health and Medical Research Council.