Dataset Information

Statistical reviewers improve reporting in biomedical articles: a randomized trial.

ABSTRACT:

Background

Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both.

Methodology and principal findings

Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with "no statistical expert" and "no checklist" as controls. The two interventions were crossed in a 2x2 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6-24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: -0.3-+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3-6.7), showing a significant improvement in quality.

Conclusions and significance

This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines.

SUBMITTER: Cobo E

PROVIDER: S-EPMC1824709 | biostudies-literature | 2007 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Statistical reviewers improve reporting in biomedical articles: a randomized trial.

Cobo Erik E Selva-O'Callagham Albert A Ribera Josep-Maria JM Cardellach Francesc F Dominguez Ruth R Vilardell Miquel M

PloS one 20070328 3

<h4>Background</h4>Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both.<h4>Methodology and principal findings</h4>Interventions were defined as 1) th ...[more]

PMID: 17389922

Similar Datasets

Project description:ImportanceNumerous studies have shown that adherence to reporting guidelines is suboptimal.ObjectiveTo evaluate whether asking peer reviewers to check if specific reporting guideline items were adequately reported would improve adherence to reporting guidelines in published articles.Design, setting, and participantsTwo parallel-group, superiority randomized trials were performed using manuscripts submitted to 7 biomedical journals (5 from the BMJ Publishing Group and 2 from the Public Library of Science) as the unit of randomization, with peer reviewers allocated to the intervention or control group.InterventionsThe first trial (CONSORT-PR) focused on manuscripts that presented randomized clinical trial (RCT) results and reported following the Consolidated Standards of Reporting Trials (CONSORT) guideline, and the second trial (SPIRIT-PR) focused on manuscripts that presented RCT protocols and reported following the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline. The CONSORT-PR trial included manuscripts that described RCT primary results (submitted July 2019 to July 2021). The SPIRIT-PR trial included manuscripts that contained RCT protocols (submitted June 2020 to May 2021). Manuscripts in both trials were randomized (1:1) to the intervention or control group; the control group received usual journal practice. In the intervention group of both trials, peer reviewers received an email from the journal that asked them to check whether the 10 most important and poorly reported CONSORT (for CONSORT-PR) or SPIRIT (for SPIRIT-PR) items were adequately reported in the manuscript. Peer reviewers and authors were not informed of the purpose of the study, and outcome assessors were blinded.Main outcomes and measuresThe difference in the mean proportion of adequately reported 10 CONSORT or SPIRIT items between the intervention and control groups in published articles.ResultsIn the CONSORT-PR trial, 510 manuscripts were randomized. Of those, 243 were published (122 in the intervention group and 121 in the control group). A mean proportion of 69.3% (95% CI, 66.0%-72.7%) of the 10 CONSORT items were adequately reported in the intervention group and 66.6% (95% CI, 62.5%-70.7%) in the control group (mean difference, 2.7%; 95% CI, -2.6% to 8.0%). In the SPIRIT-PR trial, of the 244 randomized manuscripts, 178 were published (90 in the intervention group and 88 in the control group). A mean proportion of 46.1% (95% CI, 41.8%-50.4%) of the 10 SPIRIT items were adequately reported in the intervention group and 45.6% (95% CI, 41.7% to 49.4%) in the control group (mean difference, 0.5%; 95% CI, -5.2% to 6.3%).Conclusions and relevanceThese 2 randomized trials found that it was not useful to implement the tested intervention to increase reporting completeness in published articles. Other interventions should be assessed and considered in the future.Trial registrationClinicalTrials.gov Identifiers: NCT05820971 (CONSORT-PR) and NCT05820984 (SPIRIT-PR).

Project description:OBJECTIVE:Many journals permit authors to submit supplementary material for publication alongside the article. We explore the value, use and role of this material in biomedical journal articles from the perspectives of authors, peer reviewers and readers. DESIGN AND SETTING:We conducted online surveys (November-December 2016) of corresponding authors and peer reviewers at 17 BMJ Publishing Group journals in a range of specialities. PARTICIPANTS:Participants were asked to respond to one of three surveys: as authors, peer reviewers or readers. RESULTS:We received 2872/20340 (14%) responses: authors 819/6892 (12%), peer reviewers 1142/6682 (17%) and readers 911/6766 (14%). Most authors submitted (711/819, 87%) and 80% (724/911) of readers reported reading supplementary material with their last article, while 95% (1086/1142) of reviewers reported seeing these materials sometimes. Additional data tables were the most common supplementary material reported (authors: 74%; reviewers: 89%; readers: 67%). A majority in each group indicated additional tables were most useful to readers (61%-77%); 20%-36%?and 3%-4% indicated they were most useful to peer reviewers and journal editors, respectively. Checklists and reporting guidelines showed the opposite: higher proportions of each group regarded these as most useful to journal editors. All three groups favoured the publication of additional tables and figures on the journal's website (80%-83%), with <4% of each group responding that these do not need to be available. Approximately one-fifth (16%-23%) responded that raw study data should be available on the journal's website, while 24%-33% said that these materials should not be made available anywhere. CONCLUSIONS:Authors, peer reviewers and readers agree that supplementary materials are useful. Supplementary tables and figures were favoured over reporting checklists or raw data for reading but not for study replication. Journals should consider the roles, resource costs and strategic placement of supplementary materials to ensure optimal usage and minimise waste. TRIAL REGISTRATION NUMBER:NCT02961036.

Dataset Information

Statistical reviewers improve reporting in biomedical articles: a randomized trial.

Background

Methodology and principal findings

Conclusions and significance

Publications

Statistical reviewers improve reporting in biomedical articles: a randomized trial.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets