Dataset Information

Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals.

ABSTRACT:

Background

Pre-publication peer review of manuscripts should enhance the value of research publications to readers who may wish to utilize findings in clinical care or health policy-making. Much published research across all medical specialties is not useful, may be misleading, wasteful and even harmful. Reporting guidelines are tools that in addition to helping authors prepare better manuscripts may help peer reviewers in assessing them. We examined journals' instructions to peer reviewers to see if and how reviewers are encouraged to use them.

Methods

We surveyed websites of 116 journals from the McMaster list. Main outcomes were 1) identification of online instructions to peer reviewers and 2) presence or absence of key domains within instructions: on journal logistics, reviewer etiquette and addressing manuscript content (11 domains).

Findings

Only 41/116 journals (35%) provided online instructions. All 41 guided reviewers about the logistics of their review processes, 38 (93%) outlined standards of behaviour expected and 39 (95%) contained instruction about evaluating the manuscript content. There was great variation in explicit instruction for reviewers about how to evaluate manuscript content. Almost half of the online instructions 19/41 (46%) mentioned reporting guidelines usually as general statements suggesting they may be useful or asking whether authors had followed them rather than clear instructions about how to use them. All 19 named CONSORT for reporting randomized trials but there was little mention of CONSORT extensions. PRISMA, QUOROM (forerunner of PRISMA), STARD, STROBE and MOOSE were mentioned by several journals. No other reporting guideline was mentioned by more than two journals.

Conclusions

Although almost half of instructions mentioned reporting guidelines, their value in improving research publications is not being fully realised. Journals have a responsibility to support peer reviewers. We make several recommendations including wider reference to the EQUATOR Network online library (www.equator-network.org/).

SUBMITTER: Hirst A

PROVIDER: S-EPMC3338712 | biostudies-literature | 2012

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals.

Hirst Allison A Altman Douglas G DG

PloS one 20120427 4

<h4>Background</h4>Pre-publication peer review of manuscripts should enhance the value of research publications to readers who may wish to utilize findings in clinical care or health policy-making. Much published research across all medical specialties is not useful, may be misleading, wasteful and even harmful. Reporting guidelines are tools that in addition to helping authors prepare better manuscripts may help peer reviewers in assessing them. We examined journals' instructions to peer review ...[more]

PMID: 22558178

Similar Datasets

Project description:ObjectiveTo assess whether the completeness of reporting of health research is related to journals' endorsement of reporting guidelines.DesignSystematic review.Data sourcesReporting guidelines from a published systematic review and the EQUATOR Network (October 2011). Studies assessing the completeness of reporting by using an included reporting guideline (termed "evaluations") (1990 to October 2011; addendum searches in January 2012) from searches of either Medline, Embase, and the Cochrane Methodology Register or Scopus, depending on reporting guideline name.Study selectionEnglish language reporting guidelines that provided explicit guidance for reporting, described the guidance development process, and indicated use of a consensus development process were included. The CONSORT statement was excluded, as evaluations of adherence to CONSORT had previously been reviewed. English or French language evaluations of included reporting guidelines were eligible if they assessed the completeness of reporting of studies as a primary intent and those included studies enabled the comparisons of interest (that is, after versus before journal endorsement and/or endorsing versus non-endorsing journals).Data extractionPotentially eligible evaluations of included guidelines were screened initially by title and abstract and then as full text reports. If eligibility was unclear, authors of evaluations were contacted; journals' websites were consulted for endorsement information where needed. The completeness of reporting of reporting guidelines was analyzed in relation to endorsement by item and, where consistent with the authors' analysis, a mean summed score.Results101 reporting guidelines were included. Of 15,249 records retrieved from the search for evaluations, 26 evaluations that assessed completeness of reporting in relation to endorsement for nine reporting guidelines were identified. Of those, 13 evaluations assessing seven reporting guidelines (BMJ economic checklist, CONSORT for harms, PRISMA, QUOROM, STARD, STRICTA, and STROBE) could be analyzed. Reporting guideline items were assessed by few evaluations.ConclusionsThe completeness of reporting of only nine of 101 health research reporting guidelines (excluding CONSORT) has been evaluated in relation to journals' endorsement. Items from seven reporting guidelines were quantitatively analyzed, by few evaluations each. Insufficient evidence exists to determine the relation between journals' endorsement of reporting guidelines and the completeness of reporting of published health research reports. Journal editors and researchers should consider collaborative prospectively designed, controlled studies to provide more robust evidence.Systematic review registrationNot registered; no known register currently accepts protocols for methodology systematic reviews.

Project description:BackgroundComplete reporting is essential for clinical research. However, the endorsement of reporting guidelines in radiological journals is still unclear. Further, as a field extensively utilizing artificial intelligence (AI), the adoption of both general and AI reporting guidelines would be necessary for enhancing quality and transparency of radiological research. This study aims to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables.MethodsThis meta-research study screened journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports, and excluded journals not publishing original research, in non-English languages, and instructions for authors unavailable. The endorsement of fifteen general reporting guidelines and ten AI reporting guidelines was rated using a five-level tool: "active strong", "active weak", "passive moderate", "passive weak", and "none". The association between endorsement and journal characteristic variables was evaluated by logistic regression analysis.ResultsWe included 117 journals. The top-five endorsed reporting guidelines were CONSORT (Consolidated Standards of Reporting Trials, 58.1%, 68/117), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 54.7%, 64/117), STROBE (STrengthening the Reporting of Observational Studies in Epidemiology, 51.3%, 60/117), STARD (Standards for Reporting of Diagnostic Accuracy, 50.4%, 59/117), and ARRIVE (Animal Research Reporting of In Vivo Experiments, 35.9%, 42/117). The most implemented AI reporting guideline was CLAIM (Checklist for Artificial Intelligence in Medical Imaging, 1.7%, 2/117), while other nine AI reporting guidelines were not mentioned. The Journal Impact Factor quartile and publisher were associated with endorsement of reporting guidelines in radiological journals.ConclusionsThe general reporting guideline endorsement was suboptimal in radiological journals. The implementation of reporting guidelines for AI applications in medical imaging was extremely low. Their adoption should be strengthened to facilitate quality and transparency of radiological study reporting.

Project description:BACKGROUND:The aim of this study was to evaluate the current state of two publication practices, reporting guidelines requirements and clinical trial registration requirements, by analyzing the "Instructions for Authors" of emergency medicine journals. METHODS:We performed a web-based data abstraction from the "Instructions for Authors" of the 27 Emergency Medicine journals catalogued in the Expanded Science Citation Index of the 2014 Journal Citation Reports and Google Scholar Metrics h5-index to identify whether each journal required, recommended, or made no mention of the following reporting guidelines: EQUATOR Network, ICMJE, ARRIVE, CARE, CONSORT, STARD, TRIPOD, CHEERS, MOOSE, STROBE, COREQ, SRQR, SQUIRE, PRISMA-P, SPIRIT, PRISMA, and QUOROM. We also extracted whether journals required or recommended trial registration. Authors were blinded to one another's ratings until completion of the data validation. Cross-tabulations and descriptive statistics were calculated using IBM SPSS 22. RESULTS:Of the 27 emergency medicine journals, 11 (11/27, 40.7%) did not mention a single guideline within their "Instructions for Authors," while the remaining 16 (16/27, 59.3%) mentioned one or more guidelines. The QUOROM statement and SRQR were not mentioned by any journals whereas the ICMJE guidelines (18/27, 66.7%) and CONSORT statement (15/27, 55.6%) were mentioned most often. Of the 27 emergency medicine journals, 15 (15/27, 55.6%) did not mention trial or review registration, while the remaining 12 (12/27, 44.4%) at least mentioned one of the two. Trial registration through ClinicalTrials.gov was mentioned by seven (7/27, 25.9%) journals while the WHO registry was mentioned by four (4/27, 14.8%). Twelve (12/27, 44.4%) journals mentioned trial registration through any registry platform. DISCUSSION:The aim of this study was to evaluate the current state of two publication practices, reporting guidelines requirements and clinical trial registration requirements, by analyzing the "Instructions for Authors" of emergency medicine journals. In this study, there was not a single reporting guideline mentioned in more than half of the journals. This undermines efforts of other journals to improve the completeness and transparency of research reporting. CONCLUSIONS:Reporting guidelines are infrequently required or recommended by emergency medicine journals. Furthermore, few require clinical trial registration. These two mechanisms may limit bias and should be considered for adoption by journal editors in emergency medicine. TRIAL REGISTRATION:UMIN000022486.

Project description:BackgroundThe peer review process has been questioned as it may fail to allow the publication of high-quality articles. This study aimed to evaluate the accuracy in identifying inadequate reporting in RCT reports by early career researchers (ECRs) using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process.MethodsWe performed a cross-sectional diagnostic study of 119 manuscripts, from BMC series medical journals, BMJ, BMJ Open, and Annals of Emergency Medicine reporting the results of two-arm parallel-group RCTs. One hundred and nineteen ECRs who had never reviewed an RCT manuscript were recruited from December 2017 to January 2018. Each ECR assessed one manuscript. To assess accuracy in identifying inadequate reporting, we used two tests: (1) ECRs assessing a manuscript using the COBPeer tool (after completing an online training module) and (2) the usual peer-review process. The reference standard was the assessment of the manuscript by two systematic reviewers. Inadequate reporting was defined as incomplete reporting or a switch in primary outcome and considered nine domains: the eight most important CONSORT domains and a switch in primary outcome(s). The primary outcome was the mean number of domains accurately classified (scale from 0 to 9).ResultsThe mean (SD) number of domains (0 to 9) accurately classified per manuscript was 6.39 (1.49) for ECRs using COBPeer versus 5.03 (1.84) for the journal's usual peer-review process, with a mean difference [95% CI] of 1.36 [0.88-1.84] (p < 0.001). Concerning secondary outcomes, the sensitivity of ECRs using COBPeer versus the usual peer-review process in detecting incompletely reported CONSORT items was 86% [95% CI 82-89] versus 20% [16-24] and in identifying a switch in primary outcome 61% [44-77] versus 11% [3-26]. The specificity of ECRs using COBPeer versus the usual process to detect incompletely reported CONSORT domains was 61% [57-65] versus 77% [74-81] and to identify a switch in primary outcome 77% [67-86] versus 98% [92-100].ConclusionsTrained ECRs using the COBPeer tool were more likely to detect inadequate reporting in RCTs than the usual peer review processes used by journals. Implementing a two-step peer-review process could help improve the quality of reporting.Trial registrationClinical.Trials.gov NCT03119376 (Registered April, 18, 2017).

Dataset Information

Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals.

Background

Methods

Findings

Conclusions

Publications

Are peer reviewers encouraged to use reporting guidelines? A survey of 116 health research journals.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets