Dataset Information

Impact of peer review on reports of randomised trials published in open peer review journals: retrospective before and after study.

ABSTRACT:

Objective

To investigate the effectiveness of open peer review as a mechanism to improve the reporting of randomised trials published in biomedical journals.

Design

Retrospective before and after study.

Setting

BioMed Central series medical journals.

Sample

93 primary reports of randomised trials published in BMC-series medical journals in 2012.

Main outcome measures

Changes to the reporting of methodological aspects of randomised trials in manuscripts after peer review, based on the CONSORT checklist, corresponding peer reviewer reports, the type of changes requested, and the extent to which authors adhered to these requests.

Results

Of the 93 trial reports, 38% (n=35) did not describe the method of random sequence generation, 54% (n=50) concealment of allocation sequence, 50% (n=46) whether the study was blinded, 34% (n=32) the sample size calculation, 35% (n=33) specification of primary and secondary outcomes, 55% (n=51) results for the primary outcome, and 90% (n=84) details of the trial protocol. The number of changes between manuscript versions was relatively small; most involved adding new information or altering existing information. Most changes requested by peer reviewers had a positive impact on the reporting of the final manuscript--for example, adding or clarifying randomisation and blinding (n=27), sample size (n=15), primary and secondary outcomes (n=16), results for primary or secondary outcomes (n=14), and toning down conclusions to reflect the results (n=27). Some changes requested by peer reviewers, however, had a negative impact, such as adding additional unplanned analyses (n=15).

Conclusion

Peer reviewers fail to detect important deficiencies in reporting of the methods and results of randomised trials. The number of these changes requested by peer reviewers was relatively small. Although most had a positive impact, some were inappropriate and could have a negative impact on reporting in the final publication.

SUBMITTER: Hopewell S

PROVIDER: S-EPMC4077234 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectivesTo evaluate the methodological quality of randomised controlled trial (RCT) abstracts in leading critical care nursing journals against the Consolidated Standards of Reporting Trials-Abstracts (CONSORT-A) checklist and to identify variables related with abstract reporting quality.DesignDescriptive methodological quality review.Data sourceWe searched the PubMed database and the websites of each included journal.Eligibility criteriaWe included RCT abstracts published between 2011 and 2021 in the first 11 Scopus-ranking (2021) critical care nursing journals that reported the results of RCTs in English that referred to the care of adult patients with acute/critical illness or their relatives and/or caregivers and conducted in ICUs.Data extraction and synthesisTwo independent investigators extracted the data using a prespecified 17-item checklist directly derived from CONSORT-A. For the primary outcome, each item was evaluated whether it was adequately reported or not and descriptive statistics were reported. An overall score was calculated by summing the results of all items and multivariate linear regression was conducted to detect potential predictors.ResultsSeventy-eight RCT abstracts were included in this review. The items with the highest CONSORT-A adherence were authors, objective, conclusion, participants, interventions and outcome. The randomisation item had the lowest CONSORT-A adherence, followed by trial registration, funding source, harms or side effects, recruitment, blinding and outcome results. The average CONSORT-A score was 8.5±1.5 points (of the maximum 17.5 points). Multivariate analysis indicated that the categorised word score and publication date were positively associated with the overall CONSORT-A score, while the first author's country being in Asia was negatively associated with the overall CONSORT-A score.ConclusionsThe 78 RCT abstracts showed poor overall adherence to CONSORT-A. The results indicate that the methodological reporting quality of RCT abstracts in critical care nursing journals requires improvement to facilitate assessment of the applicability and relevance of the results reported.

Project description:The field of law has retained its distinctiveness regarding peer review to this day, and reviews are often conducted without following standardized rules and principles. External and independent evaluation of submissions has recently become adopted by European law journals, and peer review procedures are still poorly defined, investigated, and attuned to the legal science publishing landscape. The aim of our study was to gain a better insight into current editorial policies on peer review in law journals by exploring editorial documents (instructions, guidelines, policies) issued by 119 Croatian, Italian, and Spanish law journals. We relied on automatic content analysis of 135 publicly available documents collected from the journal websites to analyze the basic features of the peer review processes, manuscript evaluation criteria, and related ethical issues using WordStat8. Differences in covered topics between the countries were compared using the chi-square test. Our findings reveal that most law journals have adopted a traditional approach, in which the editorial board manages mostly anonymized peer review (104, 77%) engaging independent/external reviewers (65, 48%). Submissions are evaluated according to their originality and relevance (113, 84%), quality of writing and presentation (94, 70%), comprehensiveness of literature references (93, 69%), and adequacy of methods (57, 42%). The main ethical issues related to peer review addressed by these journals are reviewer's competing interests (42, 31%), plagiarism (35, 26%), and biases (30, 22%). We observed statistically significant differences between countries in mentioning key concepts such as "Peer review ethics", "Reviewer", "Transparency of identities", "Publication type", and "Research misconduct". Spanish journals favor reviewers' "Independence" and "Competence" and "Anonymized" peer review process. Also, some manuscript types popular in one country are rarely mentioned in other countries. Even though peer review is equally conventional in all three countries, high transparency in Croatian law journals, respect for research integrity in Spanish ones, and diversity and inclusion in Italian are promising indicators of future development.

Project description:IntroductionThe role of clinical trials in medicine is expanding, particularly in surgery. Randomised controlled trials (RCTs) represent the gold standard evidence for high-quality assessment of healthcare interventions. The Consolidated Standards of Reporting Trials (CONSORT) guidance has been published to maximise RCT reporting transparency. This paper outlines the study protocol for a systematic review that will assess the current compliance of RCTs published within craniofacial surgery with the CONSORT criteria. The aims are to identify areas where reporting can be improved to ensure craniofacial surgery is guided by high-quality evidence.Methods and analysisThis protocol is compliant with the Preferred Reporting Items for Systematic Review and meta-Analysis protocols (PRISMA-P) guidelines. Craniofacial surgery RCTs will be identified by searching within craniofacial surgery journals. Five journals from the Thomson Reuters Impact Factor Report 2016 included 'cranio' in their title and were included. MEDLINE PubMed will be used to search all RCTs published in these journals. The search strategy is described within this protocol. It will be limited to articles written in English, conducted on humans, and published in the last five years. Two independent researchers will assess each study for inclusion and will perform the data extraction. The researchers will assess compliance of each RCT with the 25-item CONSORT Statement checklist as the primary outcome. Discrepancies will be resolved through consensus or third author arbitration. Secondary outcomes to be extracted include the pathology and interventions examined, and indices of RCT quality. The systematic review will be compliant with PRISMA guidelines. The review has been registered a priori with the Registry of Systematic Reviews/meta-analyses (UIN: reviewregistry219).Ethics and disseminationThis systematic review will be conducted in line with the Cochrane Handbook for Systematic Reviews and Interventions. The intent is to publish in a peer-reviewed journal and present the data at relevant conferences.