Dataset Information

Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials.

ABSTRACT:

Background

Indirect comparisons of competing treatments by network meta-analysis (NMA) are increasingly in use. Reporting bias has received little attention in this context. We aimed to assess the impact of such bias in NMAs.

Methods

We used data from 74 FDA-registered placebo-controlled trials of 12 antidepressants and their 51 matching publications. For each dataset, NMA was used to estimate the effect sizes for 66 possible pair-wise comparisons of these drugs, the probabilities of being the best drug and ranking the drugs. To assess the impact of reporting bias, we compared the NMA results for the 51 published trials and those for the 74 FDA-registered trials. To assess how reporting bias affecting only one drug may affect the ranking of all drugs, we performed 12 different NMAs for hypothetical analysis. For each of these NMAs, we used published data for one drug and FDA data for the 11 other drugs.

Findings

Pair-wise effect sizes for drugs derived from the NMA of published data and those from the NMA of FDA data differed in absolute value by at least 100% in 30 of 66 pair-wise comparisons (45%). Depending on the dataset used, the top 3 agents differed, in composition and order. When reporting bias hypothetically affected only one drug, the affected drug ranked first in 5 of the 12 NMAs but second (n?=?2), fourth (n?=?1) or eighth (n?=?2) in the NMA of the complete FDA network.

Conclusions

In this particular network, reporting bias biased NMA-based estimates of treatments efficacy and modified ranking. The reporting bias effect in NMAs may differ from that in classical meta-analyses in that reporting bias affecting only one drug may affect the ranking of all drugs.

SUBMITTER: Trinquart L

PROVIDER: S-EPMC3335054 | biostudies-literature | 2012

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials.

Trinquart Ludovic L Abbé Adeline A Ravaud Philippe P

PloS one 20120420 4

<h4>Background</h4>Indirect comparisons of competing treatments by network meta-analysis (NMA) are increasingly in use. Reporting bias has received little attention in this context. We aimed to assess the impact of such bias in NMAs.<h4>Methods</h4>We used data from 74 FDA-registered placebo-controlled trials of 12 antidepressants and their 51 matching publications. For each dataset, NMA was used to estimate the effect sizes for 66 possible pair-wise comparisons of these drugs, the probabilities ...[more]

PMID: 22536359

Similar Datasets

Project description:BackgroundNetwork meta-analysis (NMA), a generalization of conventional MA, allows for assessing the relative effectiveness of multiple interventions. Reporting bias is a major threat to the validity of MA and NMA. Numerous methods are available to assess the robustness of MA results to reporting bias. We aimed to extend such methods to NMA.MethodsWe introduced 2 adjustment models for Bayesian NMA. First, we extended a meta-regression model that allows the effect size to depend on its standard error. Second, we used a selection model that estimates the propensity of trial results being published and in which trials with lower propensity are weighted up in the NMA model. Both models rely on the assumption that biases are exchangeable across the network. We applied the models to 2 networks of placebo-controlled trials of 12 antidepressants, with 74 trials in the US Food and Drug Administration (FDA) database but only 51 with published results. NMA and adjustment models were used to estimate the effects of the 12 drugs relative to placebo, the 66 effect sizes for all possible pair-wise comparisons between drugs, probabilities of being the best drug and ranking of drugs. We compared the results from the 2 adjustment models applied to published data and NMAs of published data and NMAs of FDA data, considered as representing the totality of the data.ResultsBoth adjustment models showed reduced estimated effects for the 12 drugs relative to the placebo as compared with NMA of published data. Pair-wise effect sizes between drugs, probabilities of being the best drug and ranking of drugs were modified. Estimated drug effects relative to the placebo from both adjustment models were corrected (i.e., similar to those from NMA of FDA data) for some drugs but not others, which resulted in differences in pair-wise effect sizes between drugs and ranking.ConclusionsIn this case study, adjustment models showed that NMA of published data was not robust to reporting bias and provided estimates closer to that of NMA of FDA data, although not optimal. The validity of such methods depends on the number of trials in the network and the assumption that conventional MAs in the network share a common mean bias mechanism.

Project description:Background:There is debate in the literature as to whether inclusion of a placebo arm may alter characteristics of antidepressant trials. However, previous research has focused on response rates of various antidepressants on average only, ignoring potential differences among drugs or other aspects of trial findings. Little is known about the impact of a placebo arm on all-cause dropout and dropout due to adverse events. Methods:We carried out a systematic review of published and unpublished double-blind randomized controlled trials (RCTs) for the acute treatment of unipolar major depression (update: January 2016). The probability of being allocated to placebo (?) was the exposure of interest, and we examined its influence on responders (efficacy), all-cause dropouts (acceptability) and dropouts due to adverse events (tolerability), while accounting for differences in drugs, trials and patient characteristics in multivariate random effects meta-regression. Results:We included 421 studies (68 305 participants) comparing 16 antidepressants or placebo; ? ranged from 20% to 50%. Response rate was lower [risk ratio (RR) 0.87; 95% confidence interval (CI) 0.83, 0.92] and all-cause dropout rate higher (RR 1.19; 95% CI 1.08, 1.31) for the same antidepressants in placebo-controlled trials compared with head-to-head trials. The probability of responding decreased by 3% (95% CI 2-5%) for every 10% increase in ?, whereas the risk of all-cause dropout increased by 4% (95% CI 1-7%). Tolerability was unaffected by ?. Response rate was inversely correlated with dropouts due to any cause (correlation coefficient -0.48; 95% CI -0.58, -0.36) and due to adverse events (-0.34; 95% CI -0.44, -0.23). Conclusions:For the same antidepressant, response rate was on average smaller and dropouts higher when placebo was included; however, no association was found with dropouts due to adverse events. Decreased patient expectations, larger dropout rates and use of inappropriate statistical methods to impute missing data may explain this phenomenon. The findings call for caution in the integration of randomized evidence involving placebo arms.

Project description:BackgroundThe issue of unblinded outcome-assessors and patients has repeatedly been stressed as a flaw in allegedly double-blind antidepressant trials. Unblinding bias can for example result from a drug's marked side effects. If such unblinding bias is present for a given drug, then it might be expected that the placebos of that drug are rated significantly less effective than that of other antidepressants.MethodsTo test this hypothesis, the present exploratory analysis conducted a Bayesian network meta-analysis (NMA) comparing the efficacy of 19 different placebos in placebo-controlled trials provided in the dataset by Cipriani et al. (Lancet 2018; 391: 1357-66). Primary outcome was efficacy (continuous) estimated on the standardized mean difference (SMD) scale and defined as the pre-post change on the Hamilton Depression scale (HAMD-17), on which information was available in N =?258 trials.ResultsComparative placebo ranking suggested mirtazapine-placebo (SMD -2.0 [-?5.0-1.0 95% CrI]) to be the most, and amitriptyline- (SMD 1.2 [-?1.6-3.9 95% CrI]) and trazodone- (SMD 2.1 [-?0.9-5.2 95% CrI]) placebos to be the least effective placebos. Other placebos suggested to be more effective than amitriptyline- and trazodone-placebos (based on 95% CrIs excluding zero) were citalopram, desvenlafaxine, duloxetine, escitalopram, fluoxetine, sertraline, and venlafaxine placebos. These NMA results were corroborated by the observation that the relative efficacy between drug and placebo was considerably larger for amitriptyline and trazodone than for instance mirtazapine, duloxetine, and venlafaxine, supported by a small and insignificant correlation between drug-efficacy and placebo-efficacy (r?=?-?0.202, p =?0.408).DiscussionThe present exploratory NMA indicates that distinguishable side effects of older drugs may unblind outcome-assessors thus resulting in overestimation of the average drug-placebo difference and underrating bias in placebo-arms, particularly for the older antidepressant drugs amitriptyline and trazodone. If confirmed in prospective studies, these findings suggest that efficacy rankings for antidepressants are susceptible to bias and should be considered unreliable or misleading. The analysis is limited by the focus on the single-comparison placebos (76%, i.e., placebos assessed in two-arm trials), since double-comparison placebos (25%, i.e., placebos assessed in three-arm trials) are hard to interpret and therefore not included in the present interpretation. Another limitation is the problem of multiplicity, which was only approximately accounted for in the Bayesian NMA by modelling treatment effects as exchangeable.

Project description:Conventional randomized placebo-controlled study design assumes the absence of drug*placebo interaction. We hypothesized the presence of such an interaction and that conventionally estimated drug effect might be biased. The objectives of the study were to determine the drug*placebo interaction effect (main) and compare conventionally estimated and interaction model-estimated drug effects (secondary).We used a hybrid of balanced placebo and randomized placebo-controlled designs. Four hundred eighty healthy volunteers were randomized to three groups. The first received hydroxyzine (25 mg) described as hydroxyzine or placebo, the second received placebo described as hydroxyzine or placebo, and the third received hydroxyzine and placebo described as unknown; each in a randomized crossover design. Seven participants failed to crossover. Group assignment was concealed from participants and study coordinators. Coordinators were blinded to group and intervention assignment. Participants and coordinators were deceived as to study objectives. Main outcomes were mean area-under-the-curve of drowsiness (therapeutic outcome) and mouth-dryness (adverse outcome), self-reported on 100 mm visual analog scale over 7 h. Drug, placebo, placebo?+?interaction, and total effects were estimated using analysis of covariance by comparing received hydroxyzine/told placebo to received placebo/told placebo, received placebo/told hydroxyzine to received placebo/told placebo, received hydroxyzine/told hydroxyzine to received hydroxyzine/told placebo, and received hydroxyzine/told hydroxyzine to received placebo/told placebo, respectively. Drug effect was also conventionally estimated in the third group.Mean (SD) age was 31.4 (6.6) years, 65% were males. There was significant difference between placebo?+?interaction effect and placebo effect for both drowsiness and mouth-dryness with a mean difference (95% confidence interval) of 35.1 (5.6 to 64.6) and 23.8 (2.4 to 45.2) mm*hr, respectively. Total effect was larger than the sum of drug and placebo effects for drowsiness (139.7 (109.8 to 169.6) vs. 99.1 (68.2 to 130.0) mm*hr) and mouth-dryness (63.6 (41.1 to 86.1) vs. 34.7 (11.1 to 58.4) mm*hr). Conventionally estimated drug effect was larger than interaction model-estimated drug effect for drowsiness (69.2 (45.5 to 92.8) vs. (58.3 (31.6 to 85.0) mm*hr) and mouth-dryness (19.9 (5.3 to 34.5) vs. 9.5 (-9.2 to 28.1) mm*hr).There is significant and important drug*placebo interaction effect that may bias conventionally estimated drug effect.ClinicalTrial.gov identifier: NCT01501591 (registered December 25, 2011).

Project description:BackgroundConditional power of network meta-analysis (NMA) can support the planning of randomized controlled trials (RCTs) assessing medical interventions. Conditional power is the probability that updating existing inconclusive evidence in NMA with additional trial(s) will result in conclusive evidence, given assumptions regarding trial design, anticipated effect sizes, or event probabilities.MethodsThe present work aimed to estimate conditional power for potential future trials on antidepressant treatments. Existing evidence was based on a published network of 502 RCTs conducted between 1979-2018 assessing acute antidepressant treatment in major depressive disorder (MDD). Primary outcomes were efficacy in terms of the symptom change on the Hamilton Depression Scale (HAMD) and tolerability in terms of the dropout rate due to adverse events. The network compares 21 antidepressants consisting of 231 relative treatment comparisons, 164 (efficacy) and 127 (tolerability) of which are currently assumed to have inconclusive evidence.ResultsRequired sample sizes to achieve new conclusive evidence with at least 80% conditional power were estimated to range between N = 894 - 4190 (efficacy) and N = 521 - 1246 (tolerability). Otherwise, sample sizes ranging between N = 49 - 485 (efficacy) and N = 40 - 320 (tolerability) may require stopping for futility based on a boundary at 20% conditional power. Optimizing trial designs by considering multiple trials that contribute both direct and indirect evidence, anticipating alternative effect sizes or alternative event probabilities, may increase conditional power but required sample sizes remain high. Antidepressants having the greatest conditional power associated with smallest required sample sizes were identified as those on which current evidence is low, i.e., clomipramine, levomilnacipran, milnacipran, nefazodone, and vilazodone, with respect to both outcomes.ConclusionsThe present results suggest that conditional power to achieve new conclusive evidence in ongoing or future trials on antidepressant treatments is low. Limiting the use of the presented conditional power analysis are primarily due to the estimated large sample sizes which would be required in future trials as well as due to the well-known small effect sizes in antidepressant treatments. These findings may inform researchers and decision-makers regarding the clinical relevance and justification of research in ongoing or future antidepressant RCTs in MDD.

Dataset Information

Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials.

Background

Methods

Findings

Conclusions

Publications

Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets