Dataset Information

Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial.

ABSTRACT:

Objective

To investigate the effect of an additional review based on reporting guidelines such as STROBE and CONSORT on quality of manuscripts.

Design

Masked randomised trial. Population Original research manuscripts submitted to the Medicina Clínica journal from May 2008 to April 2009 and considered suitable for publication.

Intervention

Control group

conventional peer reviews alone. Intervention group: conventional review plus an additional review looking for missing items from reporting guidelines. Outcomes Manuscript quality, assessed with a 5 point Likert scale (primary: overall quality; secondary: average quality of specific items in paper). Main analysis compared groups as allocated, after adjustment for baseline factors (analysis of covariance); sensitivity analysis compared groups as reviewed. Adherence to reviewer suggestions assessed with Likert scale.

Results

Of 126 consecutive papers receiving conventional review, 34 were not suitable for publication. The remaining 92 papers were allocated to receive conventional reviews alone (n=41) or additional reviews (n=51). Four papers assigned to the conventional review group deviated from protocol; they received an additional review based on reporting guidelines. We saw an improvement in manuscript quality in favour of the additional review group (comparison as allocated, 0.25, 95% confidence interval -0.05 to 0.54; as reviewed, 0.33, 0.03 to 0.63). More papers with additional reviews than with conventional reviews alone improved from baseline (22 (43%) v eight (20%), difference 23.6% (3.2% to 44.0%), number needed to treat 4.2 (from 2.3 to 31.2), relative risk 2.21 (1.10 to 4.44)). Authors in the additional review group adhered more to suggestions from conventional reviews than to those from additional reviews (average increase 0.43 Likert points (0.19 to 0.67)).

Conclusions

Additional reviews based on reporting guidelines improve manuscript quality, although the observed effect was smaller than hypothesised and not definitively demonstrated. Authors adhere more to suggestions from conventional reviews than to those from additional reviews, showing difficulties in adhering to high methodological standards at the latest research phases. To boost paper quality and impact, authors should be aware of future requirements of reporting guidelines at the very beginning of their study. Trial registration and protocol Although registries do not include trials of peer review, the protocol design was submitted to sponsored research projects (Instituto de Salud Carlos III, PI081903).

SUBMITTER: Cobo E

PROVIDER: S-EPMC3222149 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ObjectiveTo describe prominent authorship positions held by women and the overall percentage of women co-authoring manuscripts submitted during the covid-19 pandemic compared with the previous two years.DesignCross sectional study.SettingNine specialist and two large general medical journals.PopulationAuthors of research manuscripts submitted between 1 January 2018 and 31 May 2021.Main outcome measuresPrimary outcome: first author's gender.Secondary outcomeslast and corresponding authors' gender; number (percentage) of women on authorship byline in "pre-pandemic" period (1 January 2018 to 31 December 2019) and in "covid-19" and "non-covid-19" manuscripts during pandemic.ResultsA total of 63 259 manuscripts were included. The number of female first, last, and corresponding authors respectively were 1313 (37.1%), 996 (27.9%), and 1119 (31.1%) for covid-19 manuscripts (lowest values in Jan-May 2020: 230 (29.4%), 165 (21.1%), and 185 (22.9%)), compared with 8583 (44.9%), 6118 (31.2%), and 7273 (37.3%) for pandemic non-covid-19 manuscripts and 12 724 (46.0%), 8923 (31.4%), and 10 981 (38.9%) for pre-pandemic manuscripts. The adjusted odds ratio of having a female first author in covid-19 manuscripts was <1.00 in all groups (P<0.001) compared with pre-pandemic (lowest in Jan-May 2020: 0.55, 98.75% confidence interval 0.43 to 0.70). The adjusted odds ratio of having a woman as last or corresponding author was significantly lower for covid-19 manuscripts in all time periods (except for the two most recent periods for last author) compared with pre-pandemic (lowest values in Jan-May 2020: 0.74 (0.57 to 0.97) for last and 0.61 (0.49 to 0.77) for corresponding author). The odds ratios for pandemic non-covid-19 manuscripts were not significantly different compared with pre-pandemic manuscripts. The median percentage of female authors on the byline was lower for covid-19 manuscripts (28.6% in Jan-May 2020) compared with pre-pandemic (36.4%) and non-covid-19 pandemic manuscripts (33.3% in Jan-May 2020). Gender disparities in all prominent authorship positions and the proportion of women authors on the byline narrowed in the most recent period (Feb-May 2021) compared with the early pandemic period (Jan-May 2020) and were very similar to values observed for pre-pandemic manuscripts.ConclusionsWomen have been underrepresented as co-authors and in prominent authorship positions in covid-19 research, and this gender disparity needs to be corrected by those involved in academic promotion and awarding of research grants. Women attained some prominent authorship positions equally or more frequently than before the pandemic on non-covid-19 related manuscripts submitted at some time points during the pandemic.

Project description:Background:Despite the widespread use of antibodies as a research tool, problems with specificity, lot-to-lot consistency and sensitivity commonly occur and may be important contributing factors to the 'replication crisis' in biomedical research. This makes the validation of antibodies and accurate reporting of this validation in the scientific literature extremely important. Therefore, some journals now require authors to comply with antibody reporting guidelines. Methods:We used a quasi-experimental approach to assess the effectiveness of such journal guidelines in improving antibody reporting in the scientific literature. In a sample of 120 publications, we compared the reporting of antibody validation and identification information in two journals with guidelines (Nature and the Journal of Comparative Neurology) with two journals without guidelines (Science and Neuroscience), before and after the introduction of these guidelines. Results:Our results suggest that the implementation of antibody reporting guidelines might have some influence on the reporting of antibody validation information. The percentage of validated antibodies per article slightly increased from 39% to 57% in journals with guidelines, whereas this percentage decreased from 23% to 14% in journals without guidelines. Furthermore, the reporting of validation information of all primary antibodies increased by 23 percentage points in the journals with guidelines (OR = 2.80, 95% CI = 0.96-INF; adjusted p = 1, one-tailed), compared to a decrease of 13 percentage points in journals without guidelines. Fortunately, the guidelines seem to be more effective in improving the reporting of antibody identification information. The reporting of identification information of all primary antibodies used in a study increased by 58 percentage points (OR = 17.8, 95% CI = 4.8-INF; adjusted p = 0.0003, one-tailed) in journals with guidelines. This percentage also slightly increased in journals without guidelines (by 18 percentage points), suggesting an overall increased awareness of the importance of antibody identifiability. Moreover, this suggests that reporting guidelines mostly have an influence on the reporting of information that is relatively easy to provide. A small increase in the reporting of validation by referencing the scientific literature or the manufacturer's data also indicates this. Conclusion:Combined with the results of previous studies on journal guidelines, our study suggests that the effect of journal antibody guidelines on validation practices by themselves may be limited, since they mostly seem to improve antibody identification instead of actual experimental validation. These guidelines, therefore, may require additional measures to ensure effective implementation. However, due to the explorative nature of our study and our small sample size, we must remain cautious towards other factors that might have played a role in the observed change in antibody reporting behaviour.

Project description:BackgroundReporting of health research is often inadequate and incomplete. Complete and transparent reporting is imperative to enable readers to assess the validity of research findings for use in healthcare and policy decision-making. To this end, many guidelines, aimed at improving the quality of health research reports, have been developed for reporting a variety of research types. Despite efforts, many reporting guidelines are underused. In order to increase their uptake, evidence of their effectiveness is important and will provide authors, peer reviewers and editors with an important resource for use and implementation of pertinent guidance. The objective of this study was to assess whether endorsement of reporting guidelines by journals influences the completeness of reporting of health studies.MethodsGuidelines providing a minimum set of items to guide authors in reporting a specific type of research, developed with explicit methodology, and using a consensus process will be identified from an earlier systematic review and from the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network's reporting guidelines library. MEDLINE, EMBASE, the Cochrane Methodology Register and Scopus will be searched for evaluations of those reporting guidelines; relevant evaluations from the recently conducted CONSORT systematic review will also be included. Single data extraction with 10% verification of study characteristics, 20% of outcomes and complete verification of aspects of study validity will be carried out. We will include evaluations of reporting guidelines that assess the completeness of reporting: (1) before and after journal endorsement, and/or (2) between endorsing and non-endorsing journals. For a given guideline, analyses will be conducted for individual and the total sum of items. When possible, standard, pooled effects with 99% confidence intervals using random effects models will be calculated.DiscussionEvidence on which guidelines have been evaluated and which are associated with improved completeness of reporting is important for various stakeholders, including editors who consider which guidelines to endorse in their journal editorial policies.