Dataset Information

Outcome discrepancies and selective reporting: impacting the leading journals?

ABSTRACT: Selective outcome reporting of either interesting or positive research findings is problematic, running the risk of poorly-informed treatment decisions. We aimed to assess the extent of outcome and other discrepancies and possible selective reporting between registry entries and published reports among leading medical journals.Randomized controlled trials published over a 6-month period from July to December 31st, 2013, were identified in five high impact medical journals: The Lancet, British Medical Journal, New England Journal of Medicine, Annals of Internal Medicine and Journal of American Medical Association were obtained. Discrepancies between published studies and registry entries were identified and related to factors including registration timing, source of funding and presence of statistically significant results.Over the 6-month period, 137 RCTs were found. Of these, 18% (n = 25) had discrepancies related to primary outcomes with the primary outcome changed in 15% (n = 20). Moreover, differences relating to non-primary outcomes were found in 64% (n = 87) with both omission of pre-specified non-primary outcomes (39%) and introduction of new non-primary outcomes (44%) common. No relationship between primary or non-primary outcome change and registration timing (prospective or retrospective; P = 0.11), source of funding (P = 0.92) and presence of statistically significant results (P = 0.92) was found.Discrepancies between registry entries and published articles for primary and non-primary outcomes were common among trials published in leading medical journals. Novel approaches are required to address this problem.

SUBMITTER: Fleming PS

PROVIDER: S-EPMC4440809 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Outcome discrepancies and selective reporting: impacting the leading journals?

Fleming Padhraig S PS Koletsi Despina D Dwan Kerry K Pandis Nikolaos N

PloS one 20150521 5

<h4>Background</h4>Selective outcome reporting of either interesting or positive research findings is problematic, running the risk of poorly-informed treatment decisions. We aimed to assess the extent of outcome and other discrepancies and possible selective reporting between registry entries and published reports among leading medical journals.<h4>Methods</h4>Randomized controlled trials published over a 6-month period from July to December 31st, 2013, were identified in five high impact medic ...[more]

PMID: 25996928

Similar Datasets

Project description:BackgroundRandomized controlled trials (RCTs) are not always well reported, especially in terms of their methodological descriptions. This study aimed to investigate the adherence of methodological reporting complying with CONSORT and explore associated trial level variables in the Chinese nursing care field.MethodsIn June 2012, we identified RCTs published in five leading Chinese nursing journals and included trials with details of randomized methods. The quality of methodological reporting was measured through the methods section of the CONSORT checklist and the overall CONSORT methodological items score was calculated and expressed as a percentage. Meanwhile, we hypothesized that some general and methodological characteristics were associated with reporting quality and conducted a regression with these data to explore the correlation. The descriptive and regression statistics were calculated via SPSS 13.0.ResultsIn total, 680 RCTs were included. The overall CONSORT methodological items score was 6.34 ± 0.97 (Mean ± SD). No RCT reported descriptions and changes in "trial design," changes in "outcomes" and "implementation," or descriptions of the similarity of interventions for "blinding." Poor reporting was found in detailing the "settings of participants" (13.1%), "type of randomization sequence generation" (1.8%), calculation methods of "sample size" (0.4%), explanation of any interim analyses and stopping guidelines for "sample size" (0.3%), "allocation concealment mechanism" (0.3%), additional analyses in "statistical methods" (2.1%), and targeted subjects and methods of "blinding" (5.9%). More than 50% of trials described randomization sequence generation, the eligibility criteria of "participants," "interventions," and definitions of the "outcomes" and "statistical methods." The regression analysis found that publication year and ITT analysis were weakly associated with CONSORT score.ConclusionsThe completeness of methodological reporting of RCTs in the Chinese nursing care field is poor, especially with regard to the reporting of trial design, changes in outcomes, sample size calculation, allocation concealment, blinding, and statistical methods.

Project description:IntroductionSelective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored.MethodsOur primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included.ResultsOf 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirty-one major discrepancies were published with a P-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes.ConclusionOur results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.

Project description:ObjectivesTo assess how often stratified randomisation is used, whether analysis adjusted for all balancing variables, and whether the method of randomisation was adequately reported, and to reanalyse a previously reported trial to assess the impact of ignoring balancing factors in the analysis.DesignReview of published trials and reanalysis of a previously reported trial.SettingFour leading general medical journals (BMJ, Journal of the American Medical Association, Lancet, and New England Journal of Medicine) and the second Multicenter Intrapleural Sepsis Trial (MIST2).Participants258 trials published in 2010 in the four journals. Cluster randomised, crossover, non-randomised, single arm, and phase I or II trials were excluded, as were trials reporting secondary analyses, interim analyses, or results that had been previously published in 2010.Main outcome measuresWhether the method of randomisation was adequately reported, how often balanced randomisation was used, and whether balancing factors were adjusted for in the analysis.ResultsReanalysis of MIST2 showed that an unadjusted analysis led to larger P values and a loss of power. The review of published trials showed that balanced randomisation was common, with 163 trials (63%) using at least one balancing variable. The most common methods of balancing were stratified permuted blocks (n=85) and minimisation (n=27). The method of randomisation was unclear in 37% of trials. Most trials that balanced on centre or prognostic factors were not adequately analysed; only 26% of trials adjusted for all balancing factors in their primary analysis. Trials that did not adjust for balancing factors in their analysis were less likely to show a statistically significant result (unadjusted 57% v adjusted 78%, P=0.02).ConclusionBalancing on centre or prognostic factors is common in trials but often poorly described, and the implications of balancing are poorly understood. Trialists should adjust their primary analysis for balancing factors to obtain correct P values and confidence intervals and to avoid an unnecessary loss in power.

Project description:ObjectivesTo evaluate the methodological quality of randomised controlled trial (RCT) abstracts in leading critical care nursing journals against the Consolidated Standards of Reporting Trials-Abstracts (CONSORT-A) checklist and to identify variables related with abstract reporting quality.DesignDescriptive methodological quality review.Data sourceWe searched the PubMed database and the websites of each included journal.Eligibility criteriaWe included RCT abstracts published between 2011 and 2021 in the first 11 Scopus-ranking (2021) critical care nursing journals that reported the results of RCTs in English that referred to the care of adult patients with acute/critical illness or their relatives and/or caregivers and conducted in ICUs.Data extraction and synthesisTwo independent investigators extracted the data using a prespecified 17-item checklist directly derived from CONSORT-A. For the primary outcome, each item was evaluated whether it was adequately reported or not and descriptive statistics were reported. An overall score was calculated by summing the results of all items and multivariate linear regression was conducted to detect potential predictors.ResultsSeventy-eight RCT abstracts were included in this review. The items with the highest CONSORT-A adherence were authors, objective, conclusion, participants, interventions and outcome. The randomisation item had the lowest CONSORT-A adherence, followed by trial registration, funding source, harms or side effects, recruitment, blinding and outcome results. The average CONSORT-A score was 8.5±1.5 points (of the maximum 17.5 points). Multivariate analysis indicated that the categorised word score and publication date were positively associated with the overall CONSORT-A score, while the first author's country being in Asia was negatively associated with the overall CONSORT-A score.ConclusionsThe 78 RCT abstracts showed poor overall adherence to CONSORT-A. The results indicate that the methodological reporting quality of RCT abstracts in critical care nursing journals requires improvement to facilitate assessment of the applicability and relevance of the results reported.

Project description:ObjectiveTo address the bias occurring in the medical literature associated with selective outcome reporting, in 2005, the International Committee of Medical Journal Editors (ICMJE) introduced mandatory trial registration guidelines and member journals required prospective registration of trials prior to patient enrolment as a condition of publication. No research has examined whether these guidelines are impacting psychiatry publications. Our objectives were to determine the extent to which articles published in psychiatry journals adhering to ICMJE guidelines were correctly prospectively registered, whether there was evidence of selective outcome reporting and changes to participant numbers, and whether there was a relationship between registration status and source of funding.Materials and methodsAny clinical trial (as defined by ICMJE) published between 1 January 2009 and 31 July 2013 in the top five psychiatry journals adhering to ICMJE guidelines (The American Journal of Psychiatry, Archives of General Psychiatry/JAMA Psychiatry, Biological Psychiatry, Journal of the American Academy of Child and Adolescent Psychiatry, and The Journal of Clinical Psychiatry) and conducted after July 2005 (or 2007 for two journals) was included. For each identified trial, where possible we extracted trial registration information, changes to POMs between publication and registry to assess selective outcome reporting, changes to participant numbers, and funding type.ResultsOut of 3305 articles, 181 studies were identified as clinical trials requiring registration: 21 (11.6%) were deemed unregistered, 61 (33.7%) were retrospectively registered, 37 (20.4%) had unclear POMs either in the article or the registry and 2 (1.1%) were registered in an inaccessible trial registry. Only 60 (33.1%) studies were prospectively registered with clearly defined POMs; 17 of these 60 (28.3%) showed evidence of selective outcome reporting and 16 (26.7%) demonstrated a change in participant numbers of 20% or more; only 26 (14.4%) of the 181 the trials were prospectively registered and did not alter their POMs or the time frames at which they were measured. Prospective registration with no changes in POMs occurred more frequently with pharmaceutical funding.DiscussionAlthough standards are in place to improve prospective registration and transparency in clinical trials, less than 15% of psychiatry trials were prospectively registered with no changes in POMs. Most trials were either not prospectively registered, changed POMs or the timeframes at some point after registration or changed participant numbers. Authors, journal editors and reviewers need to further efforts to highlight the value of prospective trial registration.

Dataset Information

Outcome discrepancies and selective reporting: impacting the leading journals?

Publications

Outcome discrepancies and selective reporting: impacting the leading journals?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets