Dataset Information

A comparison of one-stage vs two-stage individual patient data meta-analysis methods: A simulation study.

ABSTRACT:

Background

Individual patient data (IPD) meta-analysis allows for the exploration of heterogeneity and can identify subgroups that most benefit from an intervention (or exposure), much more successfully than meta-analysis of aggregate data. One-stage or two-stage IPD meta-analysis is possible, with the former using mixed-effects regression models and the latter obtaining study estimates through simpler regression models before aggregating using standard meta-analysis methodology. However, a comprehensive comparison of the two methods, in practice, is lacking.

Methods

We generated 1000 datasets for each of many simulation scenarios covering different IPD sizes and different between-study variance (heterogeneity) assumptions at various levels (intercept and exposure). Numerous simulation settings of different assumptions were also used, while we evaluated performance both on main effects and interaction effects. Performance was assessed on mean bias, mean error, coverage, and power.

Results

Fully specified one-stage models (random study intercept or fixed study-specific intercept; random exposure effect; and fixed study-specific effects for covariate) were the best performers overall, especially when investigating interactions. For main effects, performance was almost identical across models unless intercept heterogeneity was present, in which case the fully specified one-stage and the two-stage models performed better. For interaction effects, differences across models were greater with the two-stage model consistently outperformed by the two fully specified one-stage models.

Conclusions

A fully specified one-stage model should be preferred (accounting for potential exposure, intercept, and, possibly, interaction heterogeneity), especially when investigating interactions. If non-convergence is encountered with a random study intercept, the fixed study-specific intercept one-stage model should be used instead.

SUBMITTER: Kontopantelis E

PROVIDER: S-EPMC6175226 | biostudies-literature | 2018 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A comparison of one-stage vs two-stage individual patient data meta-analysis methods: A simulation study.

Kontopantelis Evangelos E

Research synthesis methods 20180621 3

<h4>Background</h4>Individual patient data (IPD) meta-analysis allows for the exploration of heterogeneity and can identify subgroups that most benefit from an intervention (or exposure), much more successfully than meta-analysis of aggregate data. One-stage or two-stage IPD meta-analysis is possible, with the former using mixed-effects regression models and the latter obtaining study estimates through simpler regression models before aggregating using standard meta-analysis methodology. However ...[more]

PMID: 29786975

Similar Datasets

Project description:A fundamental aspect of epidemiological studies concerns the estimation of factor-outcome associations to identify risk factors, prognostic factors and potential causal factors. Because reliable estimates for these associations are important, there is a growing interest in methods for combining the results from multiple studies in individual participant data meta-analyses (IPD-MA). When there is substantial heterogeneity across studies, various random-effects meta-analysis models are possible that employ a one-stage or two-stage method. These are generally thought to produce similar results, but empirical comparisons are few.We describe and compare several one- and two-stage random-effects IPD-MA methods for estimating factor-outcome associations from multiple risk-factor or predictor finding studies with a binary outcome. One-stage methods use the IPD of each study and meta-analyse using the exact binomial distribution, whereas two-stage methods reduce evidence to the aggregated level (e.g. odds ratios) and then meta-analyse assuming approximate normality. We compare the methods in an empirical dataset for unadjusted and adjusted risk-factor estimates.Though often similar, on occasion the one-stage and two-stage methods provide different parameter estimates and different conclusions. For example, the effect of erythema and its statistical significance was different for a one-stage (OR?=?1.35, [Formula: see text]) and univariate two-stage (OR?=?1.55, [Formula: see text]). Estimation issues can also arise: two-stage models suffer unstable estimates when zero cell counts occur and one-stage models do not always converge.When planning an IPD-MA, the choice and implementation (e.g. univariate or multivariate) of a one-stage or two-stage method should be prespecified in the protocol as occasionally they lead to different conclusions about which factors are associated with outcome. Though both approaches can suffer from estimation challenges, we recommend employing the one-stage method, as it uses a more exact statistical approach and accounts for parameter correlation.

Project description:BACKGROUND:Researchers and funders should consider the statistical power of planned Individual Participant Data (IPD) meta-analysis projects, as they are often time-consuming and costly. We propose simulation-based power calculations utilising a two-stage framework, and illustrate the approach for a planned IPD meta-analysis of randomised trials with continuous outcomes where the aim is to identify treatment-covariate interactions. METHODS:The simulation approach has four steps: (i) specify an underlying (data generating) statistical model for trials in the IPD meta-analysis; (ii) use readily available information (e.g. from publications) and prior knowledge (e.g. number of studies promising IPD) to specify model parameter values (e.g. control group mean, intervention effect, treatment-covariate interaction); (iii) simulate an IPD meta-analysis dataset of a particular size from the model, and apply a two-stage IPD meta-analysis to obtain the summary estimate of interest (e.g. interaction effect) and its associated p-value; (iv) repeat the previous step (e.g. thousands of times), then estimate the power to detect a genuine effect by the proportion of summary estimates with a significant p-value. RESULTS:In a planned IPD meta-analysis of lifestyle interventions to reduce weight gain in pregnancy, 14 trials (1183 patients) promised their IPD to examine a treatment-BMI interaction (i.e. whether baseline BMI modifies intervention effect on weight gain). Using our simulation-based approach, a two-stage IPD meta-analysis has <?60% power to detect a reduction of 1 kg weight gain for a 10-unit increase in BMI. Additional IPD from ten other published trials (containing 1761 patients) would improve power to over 80%, but only if a fixed-effect meta-analysis was appropriate. Pre-specified adjustment for prognostic factors would increase power further. Incorrect dichotomisation of BMI would reduce power by over 20%, similar to immediately throwing away IPD from ten trials. CONCLUSIONS:Simulation-based power calculations could inform the planning and funding of IPD projects, and should be used routinely.

Project description:BACKGROUND: Meta-analyses (MA) based on individual patient data (IPD) are regarded as the gold standard for meta-analyses and are becoming increasingly common, having several advantages over meta-analyses of summary statistics. These analyses are being undertaken in an increasing diversity of settings, often having a binary outcome. In a previous systematic review of articles published between 1999-2001, the statistical approach was seldom reported in sufficient detail, and the outcome was binary in 32% of the studies considered. Here, we explore statistical methods used for IPD-MA of binary outcomes only, a decade later. METHODS: We selected 56 articles, published in 2011 that presented results from an individual patient data meta-analysis. Of these, 26 considered a binary outcome. Here, we review 26 IPD-MA published during 2011 to consider: the goal of the study and reason for conducting an IPD-MA, whether they obtained all the data they sought, the approach used in their analysis, for instance, a two-stage or a one stage model, and the assumption of fixed or random effects. We also investigated how heterogeneity across studies was described and how studies investigated the effects of covariates. RESULTS: 19 of the 26 IPD-MA used a one-stage approach. 9 IPD-MA used a one-stage random treatment-effect logistic regression model, allowing the treatment effect to vary across studies. Twelve IPD-MA presented some form of statistic to measure heterogeneity across studies, though these were usually calculated using two-stage approach. Subgroup analyses were undertaken in all IPD-MA that aimed to estimate a treatment effect or safety of a treatment,. Sixteen meta-analyses obtained 90% or more of the patients sought. CONCLUSION: Evidence from this systematic review shows that the use of binary outcomes in assessing the effects of health care problems has increased, with random effects logistic regression the most common method of analysis. Methods are still often not reported in enough detail. Results also show that heterogeneity of treatment effects is discussed in most applications.

Dataset Information

A comparison of one-stage vs two-stage individual patient data meta-analysis methods: A simulation study.

Background

Methods

Results

Conclusions

Publications

A comparison of one-stage vs two-stage individual patient data meta-analysis methods: A simulation study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure