Dataset Information

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study.

ABSTRACT:

Background

Network meta-analyses using individual participant data (IPD-NMAs) have been increasingly used to compare the effects of multiple interventions. Although there have been many studies on statistical methods for IPD-NMAs, it is unclear whether there are statistical defects in published IPD-NMAs and whether the reporting of statistical analyses has improved. This study aimed to investigate statistical methods used and assess the reporting and methodological quality of IPD-NMAs.

Methods

We searched four bibliographic databases to identify published IPD-NMAs. The methodological quality was assessed using AMSTAR-2 and reporting quality assessed based on PRISMA-IPD and PRISMA-NMA. We performed stratified analyses and correlation analyses to explore the factors that might affect quality.

Results

We identified 21 IPD-NMAs. Only 23.8% of the included IPD-NMAs reported statistical techniques used for missing participant data, 42.9% assessed the consistency, and none assessed the transitivity. None of the included IPD-NMAs reported sources of funding for trials included, only 9.5% stated pre-registration of protocols, and 28.6% assessed the risk of bias in individual studies. For reporting quality, compliance rates were lower than 50.0% for more than half of the items. Less than 15.0% of the IPD-NMAs reported data integrity, presented the network geometry, or clarified risk of bias across studies. IPD-NMAs with statistical or epidemiological authors often better assessed the inconsistency (P?=?0.017). IPD-NMAs with a priori protocol were associated with higher reporting quality in terms of search (P?=?0.046), data collection process (P?=?0.031), and syntheses of results (P?=?0.006).

Conclusions

The reporting of statistical methods and compliance rates of methodological and reporting items of IPD-NMAs were suboptimal. Authors of future IPD-NMAs should address the identified flaws and strictly adhere to methodological and reporting guidelines.

SUBMITTER: Gao Y

PROVIDER: S-EPMC7262764 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study.

Gao Ya Y Shi Shuzhen S Li Muyang M Luo Xinyue X Liu Ming M Yang Kelu K Zhang Junhua J Song Fujian F Tian Jinhui J

BMC medicine 20200601 1

<h4>Background</h4>Network meta-analyses using individual participant data (IPD-NMAs) have been increasingly used to compare the effects of multiple interventions. Although there have been many studies on statistical methods for IPD-NMAs, it is unclear whether there are statistical defects in published IPD-NMAs and whether the reporting of statistical analyses has improved. This study aimed to investigate statistical methods used and assess the reporting and methodological quality of IPD-NMAs.<h ...[more]

PMID: 32475340

Similar Datasets

Project description:BackgroundIndividual participant data (IPD) meta-analyses that obtain "raw" data from studies rather than summary data typically adopt a "two-stage" approach to analysis whereby IPD within trials generate summary measures, which are combined using standard meta-analytical methods. Recently, a range of "one-stage" approaches which combine all individual participant data in a single meta-analysis have been suggested as providing a more powerful and flexible approach. However, they are more complex to implement and require statistical support. This study uses a dataset to compare "two-stage" and "one-stage" models of varying complexity, to ascertain whether results obtained from the approaches differ in a clinically meaningful way.Methods and findingsWe included data from 24 randomised controlled trials, evaluating antiplatelet agents, for the prevention of pre-eclampsia in pregnancy. We performed two-stage and one-stage IPD meta-analyses to estimate overall treatment effect and to explore potential treatment interactions whereby particular types of women and their babies might benefit differentially from receiving antiplatelets. Two-stage and one-stage approaches gave similar results, showing a benefit of using anti-platelets (Relative risk 0.90, 95% CI 0.84 to 0.97). Neither approach suggested that any particular type of women benefited more or less from antiplatelets. There were no material differences in results between different types of one-stage model.ConclusionsFor these data, two-stage and one-stage approaches to analysis produce similar results. Although one-stage models offer a flexible environment for exploring model structure and are useful where across study patterns relating to types of participant, intervention and outcome mask similar relationships within trials, the additional insights provided by their usage may not outweigh the costs of statistical support for routine application in syntheses of randomised controlled trials. Researchers considering undertaking an IPD meta-analysis should not necessarily be deterred by a perceived need for sophisticated statistical methods when combining information from large randomised trials.

Project description:ObjectiveTo estimate the associations between gestational weight gain (GWG) during pregnancy and neonatal outcomes in low and middle income countries.DesignIndividual participant data meta-analysis.SettingProspective pregnancy studies from 24 low and middle income countries.Main outcome measuresNine neonatal outcomes related to timing (preterm birth) and anthropometry (weight, length, and head circumference) at birth, stillbirths, and neonatal death.Analysis methodsA systematic search was conducted in PubMed, Embase, and Web of Science which identified 53 prospective pregnancy studies published after the year 2000 with data on GWG, timing and anthropometry at birth, and neonatal mortality. GWG adequacy was defined as the ratio of the observed maternal weight gain over the recommended weight gain based on the Institute of Medicine body mass index specific guidelines, which are derived from data in high income settings, and the INTERGROWTH-21st GWG standards. Study specific estimates, adjusted for confounders, were generated and then pooled using random effects meta-analysis models. Maternal age and body mass index before pregnancy were examined as potential modifiers of the associations between GWG adequacy and neonatal outcomes.ResultsOverall, 55% of participants had severely inadequate (<70%) or moderately inadequate (70% to <90%) GWG, 22% had adequate GWG (90-125%), and 23% had excessive GWG (≥125%). Severely inadequate GWG was associated with a higher risk of low birthweight (adjusted relative risk 1.62, 95% confidence interval 1.51 to 1.72; 48 studies, 93 337 participants; τ2=0.006), small for gestational age (1.44, 1.36 to 1.54; 51 studies, 93 191 participants; τ2=0.016), short for gestational age (1.47, 1.29 to 1.69; 40 studies, 83 827 participants; τ2=0.074), and microcephaly (1.57, 1.31 to 1.88; 31 studies, 80 046 participants; τ2=0.145) compared with adequate GWG. Excessive GWG was associated with a higher risk of preterm birth (1.22, 1.13 to 1.31; 48 studies, 103 762 participants; τ2=0.008), large for gestational age (1.44, 1.33 to 1.57; 47 studies, 90 044 participants; τ2=0.009), and macrosomia (1.52, 1.33 to 1.73; 29 studies, 68 138 participants; τ2=0) compared with adequate GWG. The direction and magnitude of the associations between GWG adequacy and several neonatal outcomes were modified by maternal age and body mass index before pregnancy.ConclusionsInadequate and excessive GWG are associated with a higher risk of adverse neonatal outcomes across settings. Interventions to promote optimal GWG during pregnancy are likely to reduce the burden of adverse neonatal outcomes, however further research is needed to assess optimal ranges of GWG based on data from low and middle income countries.

Project description:BackgroundIndividual participant data meta-analysis (IPD-MA) is considered the gold standard for investigating subgroup effects. Frequently used regression-based approaches to detect subgroups in IPD-MA are: meta-regression, per-subgroup meta-analysis (PS-MA), meta-analysis of interaction terms (MA-IT), naive one-stage IPD-MA (ignoring potential study-level confounding), and centred one-stage IPD-MA (accounting for potential study-level confounding). Clear guidance on the analyses is lacking and clinical researchers may use approaches with suboptimal efficiency to investigate subgroup effects in an IPD setting. Therefore, our aim is to overview and compare the aforementioned methods, and provide recommendations over which should be preferred.MethodsWe conducted a simulation study where we generated IPD of randomised trials and varied the magnitude of subgroup effect (0, 25, 50% relative reduction), between-study treatment effect heterogeneity (none, medium, large), ecological bias (none, quantitative, qualitative), sample size (50,100,200), and number of trials (5,10) for binary, continuous and time-to-event outcomes. For each scenario, we assessed the power, false positive rate (FPR) and bias of aforementioned five approaches.ResultsNaive and centred IPD-MA yielded the highest power, whilst preserving acceptable FPR around the nominal 5% in all scenarios. Centred IPD-MA showed slightly less biased estimates than naïve IPD-MA. Similar results were obtained for MA-IT, except when analysing binary outcomes (where it yielded less power and FPR < 5%). PS-MA showed similar power as MA-IT in non-heterogeneous scenarios, but power collapsed as heterogeneity increased, and decreased even more in the presence of ecological bias. PS-MA suffered from too high FPRs in non-heterogeneous settings and showed biased estimates in all scenarios. Meta-regression showed poor power (< 20%) in all scenarios and completely biased results in settings with qualitative ecological bias.ConclusionsOur results indicate that subgroup detection in IPD-MA requires careful modelling. Naive and centred IPD-MA performed equally well, but due to less bias of the estimates in the presence of ecological bias, we recommend the latter.

Dataset Information

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study.

Background

Methods

Results

Conclusions

Publications

Statistical analyses and quality of individual participant data network meta-analyses were suboptimal: a cross-sectional study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets