Open peer review at four STEM journals: an observational overview.
Ontology highlight
ABSTRACT:
SUBMITTER: Ford E
PROVIDER: S-EPMC4350441 | biostudies-literature | 2015
REPOSITORIES: biostudies-literature
Publications
Open peer review at four STEM journals: an observational overview.
F1000Research 20150109
Open peer review, peer review where authors' and reviewers' identities are disclosed to one another, is a growing trend in scholarly publishing. Through observation of four journals in STEM disciplines, PLOS One, Atmospheric Chemistry & Physics, PeerJ, and F1000Research, an observational overview is conducted. The overview relies on defined characteristics of open peer review. Results show that despite differing open peer review implementations, each journal retains editorial involvement in scho ...[more]
Similar Datasets
Project description:The field of law has retained its distinctiveness regarding peer review to this day, and reviews are often conducted without following standardized rules and principles. External and independent evaluation of submissions has recently become adopted by European law journals, and peer review procedures are still poorly defined, investigated, and attuned to the legal science publishing landscape. The aim of our study was to gain a better insight into current editorial policies on peer review in law journals by exploring editorial documents (instructions, guidelines, policies) issued by 119 Croatian, Italian, and Spanish law journals. We relied on automatic content analysis of 135 publicly available documents collected from the journal websites to analyze the basic features of the peer review processes, manuscript evaluation criteria, and related ethical issues using WordStat8. Differences in covered topics between the countries were compared using the chi-square test. Our findings reveal that most law journals have adopted a traditional approach, in which the editorial board manages mostly anonymized peer review (104, 77%) engaging independent/external reviewers (65, 48%). Submissions are evaluated according to their originality and relevance (113, 84%), quality of writing and presentation (94, 70%), comprehensiveness of literature references (93, 69%), and adequacy of methods (57, 42%). The main ethical issues related to peer review addressed by these journals are reviewer's competing interests (42, 31%), plagiarism (35, 26%), and biases (30, 22%). We observed statistically significant differences between countries in mentioning key concepts such as "Peer review ethics", "Reviewer", "Transparency of identities", "Publication type", and "Research misconduct". Spanish journals favor reviewers' "Independence" and "Competence" and "Anonymized" peer review process. Also, some manuscript types popular in one country are rarely mentioned in other countries. Even though peer review is equally conventional in all three countries, high transparency in Croatian law journals, respect for research integrity in Spanish ones, and diversity and inclusion in Italian are promising indicators of future development.
| S-EPMC8692876 | biostudies-literature
Project description:ObjectiveTo investigate the effectiveness of open peer review as a mechanism to improve the reporting of randomised trials published in biomedical journals.DesignRetrospective before and after study.SettingBioMed Central series medical journals.Sample93 primary reports of randomised trials published in BMC-series medical journals in 2012.Main outcome measuresChanges to the reporting of methodological aspects of randomised trials in manuscripts after peer review, based on the CONSORT checklist, corresponding peer reviewer reports, the type of changes requested, and the extent to which authors adhered to these requests.ResultsOf the 93 trial reports, 38% (n=35) did not describe the method of random sequence generation, 54% (n=50) concealment of allocation sequence, 50% (n=46) whether the study was blinded, 34% (n=32) the sample size calculation, 35% (n=33) specification of primary and secondary outcomes, 55% (n=51) results for the primary outcome, and 90% (n=84) details of the trial protocol. The number of changes between manuscript versions was relatively small; most involved adding new information or altering existing information. Most changes requested by peer reviewers had a positive impact on the reporting of the final manuscript--for example, adding or clarifying randomisation and blinding (n=27), sample size (n=15), primary and secondary outcomes (n=16), results for primary or secondary outcomes (n=14), and toning down conclusions to reflect the results (n=27). Some changes requested by peer reviewers, however, had a negative impact, such as adding additional unplanned analyses (n=15).ConclusionPeer reviewers fail to detect important deficiencies in reporting of the methods and results of randomised trials. The number of these changes requested by peer reviewers was relatively small. Although most had a positive impact, some were inappropriate and could have a negative impact on reporting in the final publication.
| S-EPMC4077234 | biostudies-literature
Project description:Researchers contribute to the scientific peer review system by providing reviews, and "withdraw" from it by submitting manuscripts that are subsequently reviewed. So far as we are aware, there has been no quantification of the balance of individual's contributions and withdrawals. We compared the number of reviews provided by individual researchers (i.e., their contribution) to the number required by their submissions (i.e. their withdrawals) in a large and anonymised database provided by the British Ecological Society. The database covered the Journal of Ecology, Journal of Animal Ecology, Journal of Applied Ecology, and Functional Ecology from 2003-2010. The majority of researchers (64%) did not have balanced contributions and withdrawals. Depending on assumptions, 12% to 44% contributed more than twice as much as required; 20% to 52% contributed less than half as much as required. Balance, or lack thereof, varied little in relation to the number of years a researcher had been active (reviewing or submitting). Researchers who contributed less than required did not lack the opportunity to review. Researchers who submitted more were more likely to accept invitations to review. These finding suggest overall that peer review of the four analysed journals is not in crisis, but only due to the favourable balance of over- and under-contributing researchers. These findings are limited to the four journals analysed, and therefore cannot include researcher's other peer review activities, which if included might change the proportions reported. Relatively low effort was required to assemble, check, and analyse the data. Broader analyses of individual researcher's peer review activities would contribute to greater quality, efficiency, and fairness in the peer review system.
| S-EPMC3962470 | biostudies-literature
Project description:Improving the methodological rigour and the quality of data analysis in manuscripts submitted to journals is key to ensure the validity of scientific claims. However, there is scant knowledge of how manuscripts change throughout the review process in academic journals. Here, we examined 27 467 manuscripts submitted to four journals from the Royal Society (2006–2017) and analysed the effect of peer review on the amount of statistical content of manuscripts, i.e. one of the most important aspects to assess the methodological rigour of manuscripts. We found that manuscripts with both initial low or high levels of statistical content increased their statistical content during peer review. The availability of guidelines on statistics in the review forms of journals was associated with an initial similarity of statistical content of manuscripts but did not have any relevant implications on manuscript change during peer review. We found that when reports were more concentrated on statistical content, there was a higher probability that these manuscripts were eventually rejected by editors.
| S-EPMC9470276 | biostudies-literature
Project description:Open peer review (OPR), as with other elements of open science and open research, is on the rise. It aims to bring greater transparency and participation to formal and informal peer review processes. But what is meant by `open peer review', and what advantages and disadvantages does it have over standard forms of review? How do authors or reviewers approach OPR? And what pitfalls and opportunities should you look out for? Here, we propose ten considerations for OPR, drawing on discussions with authors, reviewers, editors, publishers and librarians, and provide a pragmatic, hands-on introduction to these issues. We cover basic principles and summarise best practices, indicating how to use OPR to achieve best value and mutual benefits for all stakeholders and the wider research community.
| S-EPMC6073088 | biostudies-other
Project description:BackgroundAwareness of patients' innovative capabilities is increasing, but there is limited knowledge regarding the extent and nature of patient-driven innovations in the peer-reviewed literature.ObjectivesThe objective of the review was to answer the question: what is the nature and extent of patient-driven innovations published in peer-reviewed scientific journals?Eligibility criteriaWe used a broad definition of innovation to allow for a comprehensive review of different types of innovations and a narrow definition of 'patient driven' to focus on the role of patients and/or family caregivers. The search was limited to years 2008-2020.Sources of evidenceFour electronic databases (Medline (Ovid), Web of Science Core Collection, PsycINFO (Ovid) and Cinahl (Ebsco)) were searched in December 2020 for publications describing patient-driven innovations and complemented with snowball strategies.Charting methodsData from the included articles were extracted and categorised inductively.ResultsA total of 96 articles on 20 patient-driven innovations were included. The number of publications increased over time, with 69% of the articles published between 2016 and 2020. Author affiliations were exclusively in high income countries with 56% of first authors in North America and 36% in European countries. Among the 20 innovations reported, 'Do-It-Yourself Artificial Pancreas System' and the online health network 'PatientsLikeMe', were the subject of half of the articles.ConclusionsPeer-reviewed publications on patient-driven innovations are increasing and we see an important opportunity for researchers and clinicians to support patient innovators' research while being mindful of taking over the work of the innovators themselves.
| S-EPMC8788234 | biostudies-literature
Project description:Clear and findable publishing policies are important for authors to choose appropriate journals for publication. We investigated the clarity of policies of 171 major academic journals across disciplines regarding peer review and preprinting. 31.6% of journals surveyed do not provide information on the type of peer review they use. Information on whether preprints can be posted or not is unclear in 39.2% of journals. 58.5% of journals offer no clear information on whether reviewer identities are revealed to authors. Around 75% of journals have no clear policy on co-reviewing, citation of preprints, and publication of reviewer identities. Information regarding practices of open peer review is even more scarce, with <20% of journals providing clear information. Having found a lack of clear information, we conclude by examining the implications this has for researchers (especially early career) and the spread of open research practices.
| S-EPMC7577440 | biostudies-literature
Project description:Scholarly journals are often blamed for a gender gap in publication rates, but it is unclear whether peer review and editorial processes contribute to it. This article examines gender bias in peer review with data for 145 journals in various fields of research, including about 1.7 million authors and 740,000 referees. We reconstructed three possible sources of bias, i.e., the editorial selection of referees, referee recommendations, and editorial decisions, and examined all their possible relationships. Results showed that manuscripts written by women as solo authors or coauthored by women were treated even more favorably by referees and editors. Although there were some differences between fields of research, our findings suggest that peer review and editorial processes do not penalize manuscripts by women. However, increasing gender diversity in editorial teams and referee pools could help journals inform potential authors about their attention to these factors and so stimulate participation by women.
| S-EPMC7787493 | biostudies-literature
Project description:BackgroundAs part of the Open Science movement, this study aims to analyze the current state of open access and open data policies concerning the availability of articles and raw data of the journals belonging to the category "Medicine, General & Internal" of the Science Citation Index Expanded.MethodsJournal data sharing policies were evaluated through the following variables: possibility of manuscript storage in repositories; reuse policy; publication on a website; and statement regarding complementary material. Subsequently, an analysis of the supplementary material associated with each article was performed through the PubMed Central repository. The study reported was assessed following the STROBE guidelines for observational studies.ResultsThis study shows that only one-third of the journals included in the category "Medicine, General & Internal" allow the depositing of their documents in repositories and its reuse, while approximately half of the journals agree to publish the document on a website as well as to deposit supplementary material along with the publication. However, the reality about this last variable is that only 9.5% of the articles analyzed contained supplementary material being the main journals involved, BMJ Open, JAMA Network Open, New England Journal of Medicine, Lancet and Plos Medicine.ConclusionsThe analysis of the opening policies of the journals concerning data availability in medical research reveals the unequal positioning of publishers towards the sharing of open data, the ambiguity regarding government policies about the obligation to deposit data and the need for ethical and standardization requirements in the typology/format of the data deposited without forgetting the important role that the researcher plays. Further studies based on journals indexed in medical databases other than Science Citation Index Expanded are needed.
| S-EPMC9154089 | biostudies-literature
Project description:One of the foundations of the scientific method is to be able to reproduce experiments and corroborate the results of research that has been done before. However, with the increasing complexities of new technologies and techniques, coupled with the specialisation of experiments, reproducing research findings has become a growing challenge. Clearly, scientific methods must be conveyed succinctly, and with clarity and rigour, in order for research to be reproducible. Here, we propose steps to help increase the transparency of the scientific method and the reproducibility of research results: specifically, we introduce a peer-review oath and accompanying manifesto. These have been designed to offer guidelines to enable reviewers (with the minimum friction or bias) to follow and apply open science principles, and support the ideas of transparency, reproducibility and ultimately greater societal impact. Introducing the oath and manifesto at the stage of peer review will help to check that the research being published includes everything that other researchers would need to successfully repeat the work. Peer review is the lynchpin of the publishing system: encouraging the community to consciously (and conscientiously) uphold these principles should help to improve published papers, increase confidence in the reproducibility of the work and, ultimately, provide strategic benefits to authors and their institutions.
| S-EPMC4304228 | biostudies-literature