Dataset Information

Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers.

ABSTRACT: BACKGROUND: In the last decade, network meta-analysis of randomized controlled trials has been introduced as an extension of pairwise meta-analysis. The advantage of network meta-analysis over standard pairwise meta-analysis is that it facilitates indirect comparisons of multiple interventions that have not been studied in a head-to-head fashion. Although assumptions underlying pairwise meta-analyses are well understood, those concerning network meta-analyses are perceived to be more complex and prone to misinterpretation. DISCUSSION: In this paper, we aim to provide a basic explanation when network meta-analysis is as valid as pairwise meta-analysis. We focus on the primary role of effect modifiers, which are study and patient characteristics associated with treatment effects. Because network meta-analysis includes different trials comparing different interventions, the distribution of effect modifiers cannot only vary across studies for a particular comparison (as with standard pairwise meta-analysis, causing heterogeneity), but also between comparisons (causing inconsistency). If there is an imbalance in the distribution of effect modifiers between different types of direct comparisons, the related indirect comparisons will be biased. If it can be assumed that this is not the case, network meta-analysis is as valid as pairwise meta-analysis. SUMMARY: The validity of network meta-analysis is based on the underlying assumption that there is no imbalance in the distribution of effect modifiers across the different types of direct treatment comparisons, regardless of the structure of the evidence network.

SUBMITTER: Jansen JP

PROVIDER: S-EPMC3707819 | biostudies-other | 2013

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers.

Jansen Jeroen P JP Naci Huseyin H

BMC medicine 20130704

<h4>Background</h4>In the last decade, network meta-analysis of randomized controlled trials has been introduced as an extension of pairwise meta-analysis. The advantage of network meta-analysis over standard pairwise meta-analysis is that it facilitates indirect comparisons of multiple interventions that have not been studied in a head-to-head fashion. Although assumptions underlying pairwise meta-analyses are well understood, those concerning network meta-analyses are perceived to be more comp ...[more]

PMID: 23826681

Similar Datasets

Project description:To examine whether the continuous updating of networks of prospectively planned randomised controlled trials (RCTs) ("living" network meta-analysis) provides strong evidence against the null hypothesis in comparative effectiveness of medical interventions earlier than the updating of conventional, pairwise meta-analysis.Empirical study of the accumulating evidence about the comparative effectiveness of clinical interventions.Database of network meta-analyses of RCTs identified through searches of Medline, Embase, and the Cochrane Database of Systematic Reviews until 14 April 2015.Network meta-analyses published after January 2012 that compared at least five treatments and included at least 20 RCTs. Clinical experts were asked to identify in each network the treatment comparison of greatest clinical interest. Comparisons were excluded for which direct and indirect evidence disagreed, based on side, or node, splitting test (P<0.10).Cumulative pairwise and network meta-analyses were performed for each selected comparison. Monitoring boundaries of statistical significance were constructed and the evidence against the null hypothesis was considered to be strong when the monitoring boundaries were crossed. A significance level was defined as ?=5%, power of 90% (?=10%), and an anticipated treatment effect to detect equal to the final estimate from the network meta-analysis. The frequency and time to strong evidence was compared against the null hypothesis between pairwise and network meta-analyses.49 comparisons of interest from 44 networks were included; most (n=39, 80%) were between active drugs, mainly from the specialties of cardiology, endocrinology, psychiatry, and rheumatology. 29 comparisons were informed by both direct and indirect evidence (59%), 13 by indirect evidence (27%), and 7 by direct evidence (14%). Both network and pairwise meta-analysis provided strong evidence against the null hypothesis for seven comparisons, but for an additional 10 comparisons only network meta-analysis provided strong evidence against the null hypothesis (P=0.002). The median time to strong evidence against the null hypothesis was 19 years with living network meta-analysis and 23 years with living pairwise meta-analysis (hazard ratio 2.78, 95% confidence interval 1.00 to 7.72, P=0.05). Studies directly comparing the treatments of interest continued to be published for eight comparisons after strong evidence had become evident in network meta-analysis.In comparative effectiveness research, prospectively planned living network meta-analyses produced strong evidence against the null hypothesis more often and earlier than conventional, pairwise meta-analyses.

Project description:BACKGROUND:Aerosolized antibiotics have been proposed as a novel and promising treatment option for the treatment of ventilator-associated pneumonia (VAP). However, the optimum aerosolized antibiotics for VAP remain uncertain. METHODS:We included studies from two systematic reviews and searched PubMed, EMBASE, and Cochrane databases for other studies. Eligible studies included randomized controlled trials and observational studies. Extracted data were analyzed by pairwise and network meta-analysis. RESULTS:Eight observational and eight randomized studies were identified for this analysis. By pairwise meta-analysis using intravenous antibiotics as the reference, patients treated with aerosolized antibiotics were associated with significantly higher rates of clinical recovery (risk ratio (RR) 1.21, 95% confidence interval (CI) 1.09-1.34; P?=?0.001) and microbiological eradication (RR 1.42, 95% CI 1.22-1.650; P?<?0.0001). There were no significant differences in the risks of mortality (RR 0.88, 95% CI 0.74-1.04; P?=?0.127) or nephrotoxicity (RR 1.00, 95% CI 0.72-1.39; P?=?0.995). Using network meta-analysis, clinical recovery benefits were seen only with aerosolized tobramycin and colistin (especially tobramycin), and microbiological eradication benefits were seen only with colistin. Aerosolized tobramycin was also associated with significantly lower mortality when compared with aerosolized amikacin and colistin and intravenous antibiotics. The assessment of rank probabilities indicated aerosolized tobramycin presented the greatest likelihood of having benefits for clinical recovery and mortality, and aerosolized colistin presented the best benefits for microbiological eradication. CONCLUSIONS:Aerosolized antibiotics appear to be a useful treatment for VAP with respect to clinical recovery and microbiological eradication, and do not increase mortality or nephrotoxicity risks. Our network meta-analysis in patients with VAP suggests that clinical recovery benefits are associated with aerosolized tobramycin and colistin (especially tobramycin), microbiological eradication with aerosolized colistin, and survival with aerosolized tobramycin, mostly based on observational studies. Due to the low levels of evidence, definitive recommendations cannot be made before additional, large randomized studies are carried out.

Project description:BACKGROUND:Network meta-analyses (NMAs) of psoriasis treatments, undertaken as part of the NICE Single Technology Appraisal (STA) process, have included heterogeneous studies. When there is inconsistency or heterogeneity across the different comparisons or trials within the network of studies, the results of the NMA may not be valid. We explored the impact of including studies with heterogeneous patient characteristics on the results of NMAs of psoriasis treatments. METHODS:All NMAs undertaken for psoriasis STAs were identified and the included studies tabulated, including patient characteristics that may influence relative treatment effects. In addition to the original network of all studies using licensed treatment doses, a range of smaller, less heterogeneous networks were mapped: 'no previous biologic use' (<?25% patients had prior biologic therapy exposure), 'Psoriasis Area and Severity Index score ??25', 'weight ??90 kg' and 'white ethnicity' (??90% patients were white). RESULTS:Sixty-nine studies were included in our synthesis (34,924 participants). A random effects model with a log-normal prior distribution was chosen for each of the subgroup NMAs. Heterogeneity was reduced for the four smaller networks. There were no significant differences in the relative treatment effect (PASI 75 response) for each treatment across the five NMAs, with all credible intervals overlapping, although there were noticeable differences. Treatment rankings based on the median relative risks were also generally consistent across the networks. However, the NMA that included only studies in which <?25% patients had prior biologic therapy exposure had slightly different treatment rankings; the anti-TNF therapies certolizumab pegol and infliximab ranked higher in this network than any other network, although credible intervals were large. CONCLUSIONS:This work has highlighted potential differences in treatment response for biologic-naïve patients. When conducting NMAs in any area, heterogeneity in patient characteristics of included trials should be carefully assessed and effect modification related to certain patient characteristics investigated through clinically relevant subgroup analyses.

Project description:BackgroundPostoperative respiratory failure is associated with increased perioperative complications. Our aim is to compare outcomes between non-invasive ventilation (NIV), high-flow nasal cannula (HFNC), and standard oxygen in patients at high-risk for or with established postoperative respiratory failure.MethodsElectronic databases including PubMed, Embase, and the Cochrane Library were reviewed from inception to September 2019. We included only randomized controlled trials (RCTs) that compared NIV, HFNC, and standard oxygen in patients at high risk for or with established postoperative respiratory failure. We performed a Bayesian network meta-analysis to calculate the odds ratio (OR) and Bayesian 95% credible intervals (CrIs).ResultsNine RCTs representing 1865 patients were included (the mean age was 61.6 ± 10.2 and 64.4% were males). In comparison with standard oxygen, NIV was associated with a significant reduction in intubation rate (OR 0.23; 95% Cr.I. 0.10-0.46), mortality (OR 0.45; 95% Cr.I. 0.27-0.71), and intensive care unit (ICU)-acquired infections (OR 0.43, 95% Cr.I. 0.25-0.70). Compared to standard oxygen, HFNC was associated with a significant reduction in intubation rate (OR 0.28, 95% Cr.I. 0.08-0.76) and ICU-acquired infections (OR 0.41; 95% Cr.I. 0.20-0.80), but not mortality (OR 0.58; 95% Cr.I. 0.26-1.22). There were no significant differences between HFNC and NIV regarding different outcomes. In a subgroup analysis, we observed a mortality benefit with NIV over standard oxygen in patients undergoing cardiothoracic surgeries but not in abdominal surgeries. Furthermore, in comparison with standard oxygen, NIV and HFNC were associated with lower intubation rates following cardiothoracic surgeries while only NIV reduced the intubation rates following abdominal surgeries.ConclusionsAmong patients with post-operative respiratory failure, HFNC and NIV were associated with significantly reduced rates of intubation and ICU-acquired infections compared with standard oxygen. Moreover, NIV was associated with reduced mortality in comparison with standard oxygen.

Project description:BackgroundThe performance of laboratory tests to diagnose pulmonary tuberculosis is dependent on the quality of the sputum sample tested. The relative merits of sputum collection methods to improve tuberculosis diagnosis are poorly characterised. We therefore aimed to investigate the effects of sputum collection methods on tuberculosis diagnosis.MethodsWe did a systematic review and meta-analysis to investigate whether non-invasive sputum collection methods in people aged at least 12 years improve the diagnostic performance of laboratory testing for pulmonary tuberculosis. We searched PubMed, Google Scholar, ProQuest, Web of Science, CINAHL, and Embase up to April 14, 2017, to identify relevant experimental, case-control, or cohort studies. We analysed data by pairwise meta-analyses with a random-effects model and by network meta-analysis. All diagnostic performance data were calculated at the sputum-sample level, except where authors only reported data at the individual patient-level. Heterogeneity was assessed, with potential causes identified by logistic meta-regression.FindingsWe identified 23 eligible studies published between 1959 and 2017, involving 8967 participants who provided 19 252 sputum samples. Brief, on-demand spot sputum collection was the main reference standard. Pooled sputum collection increased tuberculosis diagnosis by microscopy (odds ratio [OR] 1·6, 95% CI 1·3-1·9, p<0·0001) or culture (1·7, 1·2-2·4, p=0·01). Providing instructions to the patient before sputum collection, during observed collection, or together with physiotherapy assistance increased diagnostic performance by microscopy (OR 1·6, 95% CI 1·3-2·0, p<0·0001). Collecting early morning sputum did not significantly increase diagnostic performance of microscopy (OR 1·5, 95% CI 0·9-2·6, p=0·2) or culture (1·4, 0·9-2·4, p=0·2). Network meta-analysis confirmed these findings, and revealed that both pooled and instructed spot sputum collections were similarly effective techniques for increasing the diagnostic performance of microscopy.InterpretationTuberculosis diagnoses were substantially increased by either pooled collection or by providing instruction on how to produce a sputum sample taken at any time of the day. Both interventions had a similar effect to that reported for the introduction of new, expensive laboratory tests, and therefore warrant further exploration in the drive to end the global tuberculosis epidemic.FundingWellcome Trust, Joint Global Health Trials consortium, Innovation For Health and Development, and Bill & Melinda Gates Foundation.

Project description:BackgroundTo investigate the prevalence of robust conclusions in systematic reviews addressing missing (participant) outcome data via a novel framework of sensitivity analyses and examine the agreement with the current sensitivity analysis standards.MethodsWe performed an empirical study on systematic reviews with two or more interventions. Pairwise meta-analyses (PMA) and network meta-analyses (NMA) were identified from empirical studies on the reporting and handling of missing outcome data in systematic reviews. PMAs with at least three studies and NMAs with at least three interventions on one primary outcome were considered eligible. We applied Bayesian methods to obtain the summary effect estimates whilst modelling missing outcome data under the missing-at-random assumption and different assumptions about the missingness mechanism in the compared interventions. The odds ratio in the logarithmic scale was considered for the binary outcomes and the standardised mean difference for the continuous outcomes. We calculated the proportion of primary analyses with robust and frail conclusions, quantified by our proposed metric, the robustness index (RI), and current sensitivity analysis standards. Cohen's kappa statistic was used to measure the agreement between the conclusions derived by the RI and the current sensitivity analysis standards.ResultsOne hundred eight PMAs and 34 NMAs were considered. When studies with a substantial number of missing outcome data dominated the analyses, the number of frail conclusions increased. The RI indicated that 59% of the analyses failed to demonstrate robustness compared to 39% when the current sensitivity analysis standards were employed. Comparing the RI with the current sensitivity analysis standards revealed that two in five analyses yielded contradictory conclusions concerning the robustness of the primary analysis results.ConclusionsCompared with the current sensitivity analysis standards, the RI offers an explicit definition of similar results and does not unduly rely on statistical significance. Hence, it may safeguard against possible spurious conclusions regarding the robustness of the primary analysis results.

Dataset Information

Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers.

Publications

Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets