Dataset Information

Evidence of selective reporting bias in hematology journals: A systematic review.

ABSTRACT: Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored.Our primary goal was to examine discrepancies between the reported primary and secondary outcomes in registered and published RCTs concerning hematological malignancies reported in hematology journals with a high impact factor. The secondary goals were to address whether outcome reporting discrepancies favored statistically significant outcomes, whether a pattern existed between the funding source and likelihood of outcome reporting bias, and whether temporal trends were present in outcome reporting bias. For trials with major outcome discrepancies, we contacted trialists to determine reasons for these discrepancies. Trials published between January 1, 2010 and December 31, 2015 in Blood; British Journal of Haematology; American Journal of Hematology; Leukemia; and Haematologica were included.Of 499 RCTs screened, 109 RCTs were included. Our analysis revealed 118 major discrepancies and 629 total discrepancies. Among the 118 discrepancies, 30 (25.4%) primary outcomes were demoted, 47 (39.8%) primary outcomes were omitted, and 30 (25.4%) primary outcomes were added. Three (2.5%) secondary outcomes were upgraded to a primary outcome. The timing of assessment for a primary outcome changed eight (6.8%) times. Thirty-one major discrepancies were published with a P-value and twenty-five (80.6%) favored statistical significance. A majority of authors whom we contacted cited a pre-planned subgroup analysis as a reason for outcome changes.Our results suggest that outcome changes occur frequently in hematology trials. Because RCTs ultimately underpin clinical judgment and guide policy implementation, selective reporting could pose a threat to medical decision making.

SUBMITTER: Wayant C

PROVIDER: S-EPMC5453439 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evidence of selective reporting bias in hematology journals: A systematic review.

Wayant Cole C Scheckel Caleb C Hicks Chandler C Nissen Timothy T Leduc Linda L Som Mousumi M Vassar Matt M

PloS one 20170601 6

<h4>Introduction</h4>Selective reporting bias occurs when chance or selective outcome reporting rather than the intervention contributes to group differences. The prevailing concern about selective reporting bias is the possibility of results being modified towards specific conclusions. In this study, we evaluate randomized controlled trials (RCTs) published in hematology journals, a group in which selective outcome reporting has not yet been explored.<h4>Methods</h4>Our primary goal was to exam ...[more]

PMID: 28570573

Similar Datasets

Project description:BACKGROUND:The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. METHODOLOGY/PRINCIPAL FINDINGS:In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS:This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Project description:ObjectiveTo assess whether the completeness of reporting of health research is related to journals' endorsement of reporting guidelines.DesignSystematic review.Data sourcesReporting guidelines from a published systematic review and the EQUATOR Network (October 2011). Studies assessing the completeness of reporting by using an included reporting guideline (termed "evaluations") (1990 to October 2011; addendum searches in January 2012) from searches of either Medline, Embase, and the Cochrane Methodology Register or Scopus, depending on reporting guideline name.Study selectionEnglish language reporting guidelines that provided explicit guidance for reporting, described the guidance development process, and indicated use of a consensus development process were included. The CONSORT statement was excluded, as evaluations of adherence to CONSORT had previously been reviewed. English or French language evaluations of included reporting guidelines were eligible if they assessed the completeness of reporting of studies as a primary intent and those included studies enabled the comparisons of interest (that is, after versus before journal endorsement and/or endorsing versus non-endorsing journals).Data extractionPotentially eligible evaluations of included guidelines were screened initially by title and abstract and then as full text reports. If eligibility was unclear, authors of evaluations were contacted; journals' websites were consulted for endorsement information where needed. The completeness of reporting of reporting guidelines was analyzed in relation to endorsement by item and, where consistent with the authors' analysis, a mean summed score.Results101 reporting guidelines were included. Of 15,249 records retrieved from the search for evaluations, 26 evaluations that assessed completeness of reporting in relation to endorsement for nine reporting guidelines were identified. Of those, 13 evaluations assessing seven reporting guidelines (BMJ economic checklist, CONSORT for harms, PRISMA, QUOROM, STARD, STRICTA, and STROBE) could be analyzed. Reporting guideline items were assessed by few evaluations.ConclusionsThe completeness of reporting of only nine of 101 health research reporting guidelines (excluding CONSORT) has been evaluated in relation to journals' endorsement. Items from seven reporting guidelines were quantitatively analyzed, by few evaluations each. Insufficient evidence exists to determine the relation between journals' endorsement of reporting guidelines and the completeness of reporting of published health research reports. Journal editors and researchers should consider collaborative prospectively designed, controlled studies to provide more robust evidence.Systematic review registrationNot registered; no known register currently accepts protocols for methodology systematic reviews.

Project description:Background and objectivesSurvey research is an important research method used to determine individuals' attitudes, knowledge, and behaviors; however, as with other research methods, inadequate reporting threatens the validity of results. This study aimed to describe the quality of reporting of surveys published between 2001 and 2011 in the field of nephrology.Design, setting, participants, & measurementsThe top nephrology journals were systematically reviewed (2001-2011: American Journal of Kidney Diseases, Nephrology Dialysis Transplantation, and Kidney International; 2006-2011: Clinical Journal of the American Society of Nephrology) for studies whose primary objective was to collect and report survey results. Included were nephrology journals with a heavy focus on clinical research and high impact factors. All titles and abstracts were screened in duplicate. Surveys were excluded if they were part of a multimethod study, evaluated only psychometric characteristics, or used semi-structured interviews. Information was collected on survey and respondent characteristics, questionnaire development (e.g., pilot testing), psychometric characteristics (e.g., validity and reliability), survey methods used to optimize response rate (e.g., system of multiple contacts), and response rate.ResultsAfter a screening of 19,970 citations, 216 full-text articles were reviewed and 102 surveys were included. Approximately 85% of studies reported a response rate. Almost half of studies (46%) discussed how they developed their questionnaire and only a quarter of studies (28%) mentioned the validity or reliability of the questionnaire. The only characteristic that improved over the years was the proportion of articles reporting missing data (2001-2004: 46.4%; 2005-2008: 61.9%; and 2009-2011: 84.8%; respectively) (P<0.01).ConclusionsThe quality of survey reporting in nephrology journals remains suboptimal. In particular, reporting of the validity and reliability of the questionnaire must be improved. Guidelines to improve survey reporting and increase transparency are clearly needed.

Dataset Information

Evidence of selective reporting bias in hematology journals: A systematic review.

Publications

Evidence of selective reporting bias in hematology journals: A systematic review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure