Dataset Information

Practices and impact of primary outcome adjustment in randomized controlled trials: meta-epidemiologic study.

ABSTRACT:

Objective

To assess adjustment practices for primary outcomes of randomized controlled trials and their impact on the results.

Design

Meta-epidemiologic study.

Data sources

25 biomedical journals with the highest impact factor according to Journal Citation Reports 2009.

Study selection

Randomized controlled trials published in print in 2009 that reported primary outcomes. The search yielded 684 eligible papers of randomized controlled trials, of which 200 were randomly selected.

Data extraction

Two researchers independently extracted data on study population, intervention, primary outcome, and the adjustment plan for primary outcomes. They also recorded the magnitude and statistical significance of the intervention effect with and without adjustments, and estimated whether adjustment made a difference in the level of nominal significance. They also compared the analysis plan for model adjustment in the published trial versus the trial protocol with information on the protocol collected from registries, design papers, and communication with all corresponding authors.

Results

54% of the trials used stratified randomization, 96% presented baseline characteristics in the compared arms, and 46% also evaluated differences in baseline factors with statistical testing. Half of the trials performed adjusted analyses for the main outcome, as the sole analysis (29%) or along with unadjusted analyses (21%). Adjustment for stratification variables and for baseline variables was performed in 39% (42/108) and 42% (84/199) of the trials, respectively. Among 40 comparisons with both adjusted and unadjusted analyses, 43% had statistically significant effects, 40% had non-significant effects, and 18% had significant effects with only one of the two analyses, but not with the other. Information on analysis plan regarding model adjustment was available in 6% (9/162) of trial registry entries, 78% (21/27) of design papers, and 74% (40/54) of protocols obtained from authors. The analysis plan disagreed between the published trial and the registry, protocol, or design paper in 47% (28/60) of the studies.

Conclusions

There is large diversity on whether and how analyses of primary outcomes are adjusted in randomized controlled trials and these choices can sometimes change the nominal significance of the results. Registered protocols should explicitly specify adjustments plans for main outcomes and analysis should follow these plans.

SUBMITTER: Saquib N

PROVIDER: S-EPMC3709831 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Spin in the reporting of randomized controlled trials, where authors report research in a way that potentially misrepresents results and mislead readers, has been demonstrated in the broader medical literature. We investigated spin in wound care trials with (a) no statistically significant result for the primary outcome and (b) no clearly specified primary outcome.We searched the Cochrane Wounds Group Specialised Register of Trials for randomized controlled trials (RCTs). Eligible studies were: Parallel-group RCTs of interventions for foot, leg or pressure ulcers published in 2004 to 2009 (inclusive) with either a clearly identified primary outcome for which there was a statistically non-significant result (Cohort A) or studies that had no clear primary outcome (Cohort B).We extracted general study details. For both Cohorts A and B we then assessed for the presence of spin. For Cohort A we used a pre-defined process to assess reports for spin. For Cohort B we aimed to assess spin by recording the number of positive treatment effect claims made. We also compared the number of statistically significant and non-significant results reported in the main text and the abstract looking specifically for spin in the form of selective outcome reporting.Of the 71 eligible studies, 28 were eligible for Cohort A; of these, 71% (20/28) contained spin. Cohort B contained 43 studies; of these, 86% (37/43) had abstracts that claimed a favorable treatment claim. Whilst 74% (32/43) of main text results in Cohort B included at least one statistically non-significant result, this was not reflected in the abstract where only 28% contained (12/43) at least one statistically non-significant result.Spin is a frequent phenomenon in reports of RCTs of wound treatments. Studies without statistically significant results for the primary outcome used spin in 71% of cases. Furthermore, 33% (43/132) of reports of wound RCTs did not specify a primary outcome and there was evidence of spin and selective outcome reporting in the abstracts of these. Readers should be wary of only reading the abstracts of reports of RCTs of wound treatments since they are frequently misleading regarding treatment effects.