Ten considerations for open peer review.
Ontology highlight
ABSTRACT:
SUBMITTER: Schmidt B
PROVIDER: S-EPMC6073088 | biostudies-other | 2018
REPOSITORIES: biostudies-other
Publications
Ten considerations for open peer review.
Schmidt Birgit B Ross-Hellauer Tony T van Edig Xenia X Moylan Elizabeth C EC
F1000Research 20180629
Open peer review (OPR), as with other elements of open science and open research, is on the rise. It aims to bring greater transparency and participation to formal and informal peer review processes. But what is meant by `open peer review', and what advantages and disadvantages does it have over standard forms of review? How do authors or reviewers approach OPR? And what pitfalls and opportunities should you look out for? Here, we propose ten considerations for OPR, drawing on discussions with a ...[more]
Similar Datasets
Project description:Open peer review (OPR) is moving into the mainstream, but it is often poorly understood and surveys of researcher attitudes show important barriers to implementation. As more journals move to implement and experiment with the myriad of innovations covered by this term, there is a clear need for best practice guidelines to guide implementation. This brief article aims to address this knowledge gap, reporting work based on an interactive stakeholder workshop to create best-practice guidelines for editors and journals who wish to transition to OPR. Although the advice is aimed mainly at editors and publishers of scientific journals, since this is the area in which OPR is at its most mature, many of the principles may also be applicable for the implementation of OPR in other areas (e.g., books, conference submissions).
| S-EPMC6394088 | biostudies-literature
Project description:One of the foundations of the scientific method is to be able to reproduce experiments and corroborate the results of research that has been done before. However, with the increasing complexities of new technologies and techniques, coupled with the specialisation of experiments, reproducing research findings has become a growing challenge. Clearly, scientific methods must be conveyed succinctly, and with clarity and rigour, in order for research to be reproducible. Here, we propose steps to help increase the transparency of the scientific method and the reproducibility of research results: specifically, we introduce a peer-review oath and accompanying manifesto. These have been designed to offer guidelines to enable reviewers (with the minimum friction or bias) to follow and apply open science principles, and support the ideas of transparency, reproducibility and ultimately greater societal impact. Introducing the oath and manifesto at the stage of peer review will help to check that the research being published includes everything that other researchers would need to successfully repeat the work. Peer review is the lynchpin of the publishing system: encouraging the community to consciously (and conscientiously) uphold these principles should help to improve published papers, increase confidence in the reproducibility of the work and, ultimately, provide strategic benefits to authors and their institutions.
| S-EPMC4304228 | biostudies-literature
Project description:Background: "Open peer review" (OPR), despite being a major pillar of Open Science, has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with numerous overlapping and contradictory definitions. While for some the term refers to peer review where the identities of both author and reviewer are disclosed to each other, for others it signifies systems where reviewer reports are published alongside articles. For others it signifies both of these conditions, and for yet others it describes systems where not only "invited experts" are able to comment. For still others, it includes a variety of combinations of these and other novel methods. Methods: Recognising the absence of a consensus view on what open peer review is, this article undertakes a systematic review of definitions of "open peer review" or "open review", to create a corpus of 122 definitions. These definitions are systematically analysed to build a coherent typology of the various innovations in peer review signified by the term, and hence provide the precise technical definition currently lacking. Results: This quantifiable data yields rich information on the range and extent of differing definitions over time and by broad subject area. Quantifying definitions in this way allows us to accurately portray exactly how ambiguously the phrase "open peer review" has been used thus far, for the literature offers 22 distinct configurations of seven traits, effectively meaning that there are 22 different definitions of OPR in the literature reviewed. Conclusions: I propose a pragmatic definition of open peer review as an umbrella term for a number of overlapping ways that peer review models can be adapted in line with the aims of Open Science, including making reviewer and author identities open, publishing review reports and enabling greater participation in the peer review process.
| S-EPMC5437951 | biostudies-literature
Project description:CytoJournal organized its first Peer-Reviewer's Retreat of 2006 during the United States and Canadian Academy of Pathology Annual Meeting at Atlanta on Feb 12, 2006. The major topics discussed were open access, peer review, and impact factors. Representative participants volunteered to join the task force to prepare an instructional guide for peer-reviewing cytopathology manuscripts. Concern about the impact factor for CytoJournal was discussed. A feedback to its readers and authors was recommended. Impact factor calculation needs at least three years of journal statistics. It is only possible after two years from the time a journal is first accepted by Thomson-ISI for citation tracking. CytoJournal is still too new for an impact factor to be calculated. However, general progress of CytoJournal suggests an encouraging pattern for high impact factor.
| S-EPMC1421433 | biostudies-literature
Project description:Open peer review, peer review where authors' and reviewers' identities are disclosed to one another, is a growing trend in scholarly publishing. Through observation of four journals in STEM disciplines, PLOS One, Atmospheric Chemistry & Physics, PeerJ, and F1000Research, an observational overview is conducted. The overview relies on defined characteristics of open peer review. Results show that despite differing open peer review implementations, each journal retains editorial involvement in scholarly publishing. Further, the analysis shows that only one of these implementations is fully transparent in its peer review and decision making process. Finally, the overview contends that journals should clearly outline peer review and editorial processes in order to allow for open peer review to be better understood and adopted by authors, reviewers, editors, and readers of science communications.
| S-EPMC4350441 | biostudies-literature
Project description:ObjectiveTo investigate the effectiveness of open peer review as a mechanism to improve the reporting of randomised trials published in biomedical journals.DesignRetrospective before and after study.SettingBioMed Central series medical journals.Sample93 primary reports of randomised trials published in BMC-series medical journals in 2012.Main outcome measuresChanges to the reporting of methodological aspects of randomised trials in manuscripts after peer review, based on the CONSORT checklist, corresponding peer reviewer reports, the type of changes requested, and the extent to which authors adhered to these requests.ResultsOf the 93 trial reports, 38% (n=35) did not describe the method of random sequence generation, 54% (n=50) concealment of allocation sequence, 50% (n=46) whether the study was blinded, 34% (n=32) the sample size calculation, 35% (n=33) specification of primary and secondary outcomes, 55% (n=51) results for the primary outcome, and 90% (n=84) details of the trial protocol. The number of changes between manuscript versions was relatively small; most involved adding new information or altering existing information. Most changes requested by peer reviewers had a positive impact on the reporting of the final manuscript--for example, adding or clarifying randomisation and blinding (n=27), sample size (n=15), primary and secondary outcomes (n=16), results for primary or secondary outcomes (n=14), and toning down conclusions to reflect the results (n=27). Some changes requested by peer reviewers, however, had a negative impact, such as adding additional unplanned analyses (n=15).ConclusionPeer reviewers fail to detect important deficiencies in reporting of the methods and results of randomised trials. The number of these changes requested by peer reviewers was relatively small. Although most had a positive impact, some were inappropriate and could have a negative impact on reporting in the final publication.
| S-EPMC4077234 | biostudies-literature
Project description:Mounting evidence indicates that worldwide, innovation systems are increasing unsustainable. Equally, concerns about inequities in the science and innovation process, and in access to its benefits, continue. Against a backdrop of growing health, economic and scientific challenges global stakeholders are urgently seeking to spur innovation and maximize the just distribution of benefits for all. Open Science collaboration (OS) - comprising a variety of approaches to increase open, public, and rapid mobilization of scientific knowledge - is seen to be one of the most promising ways forward. Yet, many decision-makers hesitate to construct policy to support the adoption and implementation of OS without access to substantive, clear and reliable evidence. In October 2017, international thought-leaders gathered at an Open Science Leadership Forum in the Washington DC offices of the Bill and Melinda Gates Foundation to share their views on what successful Open Science looks like. Delegates from developed and developing nations, national governments, science agencies and funding bodies, philanthropy, researchers, patient organizations and the biotechnology, pharma and artificial intelligence (AI) industries discussed the outcomes that would rally them to invest in OS, as well as wider issues of policy and implementation. This first of two reports, summarizes delegates' views on what they believe OS will deliver in terms of research, innovation and social impact in the life sciences. Through open and collaborative process over the next months, we will translate these success outcomes into a toolkit of quantitative and qualitative indicators to assess when, where and how open science collaborations best advance research, innovation and social benefit. Ultimately, this work aims to develop and openly share tools to allow stakeholders to evaluate and re-invent their innovation ecosystems, to maximize value for the global public and patients, and address long-standing questions about the mechanics of innovation.
| S-EPMC7839829 | biostudies-literature
Project description:BMC Pharmacology and Toxicology was created from the merger of two journals within the BMC series published by BioMed Central: BMC Pharmacology and BMC Clinical Pharmacology. BMC Pharmacology operated anonymous peer review whereas BMC Clinical Pharmacology operated a fully open peer review policy where the identity of the reviewers was known to the editors, authors and readers. The merged journal also adopted a fully open peer review policy. Two years on we discuss the views and experiences of our Editorial Board Members towards open peer review on this biomedical journal.
| S-EPMC4191873 | biostudies-literature
Project description:BackgroundAn a priori design is essential to reduce the risk of bias in systematic reviews (SRs). To this end, authors can register their SR with PROSPERO, and/or publish a SR protocol in an academic journal. The latter has the advantage that the manuscript for the SR protocol is usually peer-reviewed. However, since authors ought not to begin/continue the SR before their protocol has been accepted for publication, it is crucial that SR protocols are processed in a timely manner. Our main aim was to descriptively analyse the peer review process of SR protocols published in 'BMC Systematic Reviews' from 2012 to 2017.MethodsWe systematically searched MEDLINE via PubMed for all SR protocols published in 'BMC Systematic Reviews' between 2012 and 2017, except for protocols for overviews, scoping reviews or realist reviews. Data were extracted from the SR protocols and Open Peer Review reports. For each round of peer review, two researchers judged the extent of revision (minor/major) based on the reviewer reports. Their content was further investigated by two researchers in a random 10%-sample using PRISMA-P as a guideline. All data were analysed descriptively.ResultsWe identified 544 eligible protocols published in 'BMC Systematic Reviews' between 2012 and 2017. Of those, 485 (89.2%) also registered the SR in PROSPERO, the majority (87.4%) before first submission of the manuscript for the SR protocol (median 49 days). The absolute number of published SR protocols increased from 2012 to 2017 (21 vs 145 protocols), as did the median processing time (61 vs 142 days from submission to acceptance) and the proportion of protocols requiring a major revision after first peer review (19.1% vs 52.4%). Reviewer comments most frequently addressed the PRISMA-P item 'Eligibility criteria'. Overall, 76.0% of the reviewer comments suggested more transparency.ConclusionsThe number of published SR protocols increased over the years, but so did the processing time. In 2017, it took several months from submission to acceptance, which is critical from an author's perspective. New models of peer review such as post publication peer review for SR protocols should be investigated. This could probably be realized with PROSPERO.
| S-EPMC6415341 | biostudies-literature
Project description:Open peer review (OPR) is a cornerstone of the emergent Open Science agenda. Yet to date no large-scale survey of attitudes towards OPR amongst academic editors, authors, reviewers and publishers has been undertaken. This paper presents the findings of an online survey, conducted for the OpenAIRE2020 project during September and October 2016, that sought to bridge this information gap in order to aid the development of appropriate OPR approaches by providing evidence about attitudes towards and levels of experience with OPR. The results of this cross-disciplinary survey, which received 3,062 full responses, show the majority (60.3%) of respondents to be believe that OPR as a general concept should be mainstream scholarly practice (although attitudes to individual traits varied, and open identities peer review was not generally favoured). Respondents were also in favour of other areas of Open Science, like Open Access (88.2%) and Open Data (80.3%). Among respondents we observed high levels of experience with OPR, with three out of four (76.2%) reporting having taken part in an OPR process as author, reviewer or editor. There were also high levels of support for most of the traits of OPR, particularly open interaction, open reports and final-version commenting. Respondents were against opening reviewer identities to authors, however, with more than half believing it would make peer review worse. Overall satisfaction with the peer review system used by scholarly journals seems to strongly vary across disciplines. Taken together, these findings are very encouraging for OPR's prospects for moving mainstream but indicate that due care must be taken to avoid a "one-size fits all" solution and to tailor such systems to differing (especially disciplinary) contexts. OPR is an evolving phenomenon and hence future studies are to be encouraged, especially to further explore differences between disciplines and monitor the evolution of attitudes.
| S-EPMC5728564 | biostudies-literature