Dataset Information

When does the use of individual patient data in network meta-analysis make a difference? A simulation study.

ABSTRACT:

Background

The use of individual patient data (IPD) in network meta-analyses (NMA) is rapidly growing. This study aimed to determine, through simulations, the impact of select factors on the validity and precision of NMA estimates when combining IPD and aggregate data (AgD) relative to using AgD only.

Methods

Three analysis strategies were compared via simulations: 1) AgD NMA without adjustments (AgD-NMA); 2) AgD NMA with meta-regression (AgD-NMA-MR); and 3) IPD-AgD NMA with meta-regression (IPD-NMA). We compared 108 parameter permutations: number of network nodes (3, 5 or 10); proportion of treatment comparisons informed by IPD (low, medium or high); equal size trials (2-armed with 200 patients per arm) or larger IPD trials (500 patients per arm); sparse or well-populated networks; and type of effect-modification (none, constant across treatment comparisons, or exchangeable). Data were generated over 200 simulations for each combination of parameters, each using linear regression with Normal distributions. To assess model performance and estimate validity, the mean squared error (MSE) and bias of treatment-effect and covariate estimates were collected. Standard errors (SE) and percentiles were used to compare estimate precision.

Results

Overall, IPD-NMA performed best in terms of validity and precision. The median MSE was lower in the IPD-NMA in 88 of 108 scenarios (similar results otherwise). On average, the IPD-NMA median MSE was 0.54 times the median using AgD-NMA-MR. Similarly, the SEs of the IPD-NMA treatment-effect estimates were 1/5 the size of AgD-NMA-MR SEs. The magnitude of superior validity and precision of using IPD-NMA varied across scenarios and was associated with the amount of IPD. Using IPD in small or sparse networks consistently led to improved validity and precision; however, in large/dense networks IPD tended to have negligible impact if too few IPD were included. Similar results also apply to the meta-regression coefficient estimates.

Conclusions

Our simulation study suggests that the use of IPD in NMA will considerably improve the validity and precision of estimates of treatment effect and regression coefficients in the most NMA IPD data-scenarios. However, IPD may not add meaningful validity and precision to NMAs of large and dense treatment networks when negligible IPD are used.

SUBMITTER: Kanters S

PROVIDER: S-EPMC7805229 | biostudies-literature | 2021 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

When does the use of individual patient data in network meta-analysis make a difference? A simulation study.

Kanters Steve S Karim Mohammad Ehsanul ME Thorlund Kristian K Anis Aslam A Bansback Nick N

BMC medical research methodology 20210113 1

<h4>Background</h4>The use of individual patient data (IPD) in network meta-analyses (NMA) is rapidly growing. This study aimed to determine, through simulations, the impact of select factors on the validity and precision of NMA estimates when combining IPD and aggregate data (AgD) relative to using AgD only.<h4>Methods</h4>Three analysis strategies were compared via simulations: 1) AgD NMA without adjustments (AgD-NMA); 2) AgD NMA with meta-regression (AgD-NMA-MR); and 3) IPD-AgD NMA with meta- ...[more]

PMID: 33435879

Similar Datasets

Project description:BackgroundThe 2018 World Health Organization HIV guidelines were based on the results of a network meta-analysis (NMA) of published trials. This study employed individual patient-level data (IPD) and aggregate data (AgD) and meta-regression methods to assess the evidence supporting the WHO recommendations and whether they needed any refinements.MethodsAccess to IPD from three trials was granted through ClinicalStudyDataRequest.com (CSDR). Seven modelling approaches were applied and compared: 1) Unadjusted AgD network meta-analysis (NMA) - the original analysis; 2) AgD-NMA with meta-regression; 3) Two-stage IPD-AgD NMA; 4) Unadjusted one-stage IPD-AgD NMA; 5) One-stage IPD-AgD NMA with meta-regression (one-stage approach); 6) Two-stage IPD-AgD NMA with empirical-priors (empirical-priors approach); 7) Hierarchical meta-regression IPD-AgD NMA (HMR approach). The first two were the models used previously. Models were compared with respect to effect estimates, changes in the effect estimates, coefficient estimates, DIC and model fit, rankings and between-study heterogeneity.ResultsIPD were available for 2160 patients, representing 6.5% of the evidence base and 3 of 24 edges. The aspect of the model affected by the choice of modeling appeared to differ across outcomes. HMR consistently generated larger intervals, often with credible intervals (CrI) containing the null value. Discontinuations due to adverse events and viral suppression at 96 weeks were the only two outcomes for which the unadjusted AgD NMA would not be selected. For the first, the selected model shifted the principal comparison of interest from an odds ratio of 0.28 (95% CrI: 10.17, 0.44) to 0.37 (95% CrI: 0.23, 0.58). Throughout all outcomes, the regression estimates differed substantially between AgD and IPD methods, with the latter being more often larger in magnitude and statistically significant.ConclusionsOverall, the use of IPD often impacted the coefficient estimates, but not sufficiently as to necessitate altering the final recommendations of the 2018 WHO Guidelines. Future work should examine the features of a network where adjustments will have an impact, such as how much IPD is required in a given size of network.

Project description:ImportancePersonalized treatment choices would increase the effectiveness of internet-based cognitive behavioral therapy (iCBT) for depression to the extent that patients differ in interventions that better suit them.ObjectiveTo provide personalized estimates of short-term and long-term relative efficacy of guided and unguided iCBT for depression using patient-level information.Data sourcesWe searched PubMed, Embase, PsycInfo, and Cochrane Library to identify randomized clinical trials (RCTs) published up to January 1, 2019.Study selectionEligible RCTs were those comparing guided or unguided iCBT against each other or against any control intervention in individuals with depression. Available individual patient data (IPD) was collected from all eligible studies. Depression symptom severity was assessed after treatment, 6 months, and 12 months after randomization.Data extraction and synthesisWe conducted a systematic review and IPD network meta-analysis and estimated relative treatment effect sizes across different patient characteristics through IPD network meta-regression.Main outcomes and measuresPatient Health Questionnaire-9 (PHQ-9) scores.ResultsOf 42 eligible RCTs, 39 studies comprising 9751 participants with depression contributed IPD to the IPD network meta-analysis, of which 8107 IPD were synthesized. Overall, both guided and unguided iCBT were associated with more effectiveness as measured by PHQ-9 scores than control treatments over the short term and the long term. Guided iCBT was associated with more effectiveness than unguided iCBT (mean difference [MD] in posttreatment PHQ-9 scores, -0.8; 95% CI, -1.4 to -0.2), but we found no evidence of a difference at 6 or 12 months following randomization. Baseline depression was found to be the most important modifier of the relative association for efficacy of guided vs unguided iCBT. Differences between unguided and guided iCBT in people with baseline symptoms of subthreshold depression (PHQ-9 scores 5-9) were small, while guided iCBT was associated with overall better outcomes in patients with baseline PHQ-9 greater than 9.Conclusions and relevanceIn this network meta-analysis with IPD, guided iCBT was associated with more effectiveness than unguided iCBT for individuals with depression, benefits were more substantial in individuals with moderate to severe depression. Unguided iCBT was associated with similar effectiveness among individuals with symptoms of mild/subthreshold depression. Personalized treatment selection is entirely possible and necessary to ensure the best allocation of treatment resources for depression.

Dataset Information

When does the use of individual patient data in network meta-analysis make a difference? A simulation study.

Background

Methods

Results

Conclusions

Publications

When does the use of individual patient data in network meta-analysis make a difference? A simulation study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets