Perspectives on involvement in the peer-review process: surveys of patient and public reviewers at two journals.
Ontology highlight
ABSTRACT: Objective
In 2014/2015, The BMJ and Research Involvement and Engagement (RIE) became the first journals to routinely include patients and the public in the peer review process of journal articles. This survey explores the perspectives and early experiences of these reviewers.Design
A cross-sectional survey.Setting and participants
Patient and public reviewers for The BMJ and RIE who have been invited to review.Results
The response rate was 69% (157/227) for those who had previously reviewed and 31% (67/217) for those who had not yet reviewed. Reviewers described being motivated to review by the opportunity to include the patient voice in the research process, influence the quality of the biomedical literature and ensure it meets the needs of patients. Of the 157 who had reviewed, 127 (81%) would recommend being a reviewer to other patients and carers. 144 (92%) thought more journals should adopt patient and public review. Few reviewers (16/224, 7%) reported concerns about doing open review. Annual acknowledgement on the journals' websites was welcomed as was free access to journal information. Participants were keen to have access to more online resources and training to improve their reviewing skills. Suggestions on how to improve the reviewing experience included: allowing more time to review; better and more frequent communication; a more user-friendly process; improving guidance on how to review including videos; improving the matching of papers to reviewers' experience; providing more varied sample reviews and brief feedback on the usefulness of reviews; developing a sense of community among reviewers; and publicising of the contribution that patient and public review brings.Conclusions
Patient and public reviewers shared practical ideas to improve the reviewing experience and these will be reviewed to enhance the guidance and support given to them.
SUBMITTER: Schroter S
PROVIDER: S-EPMC6129098 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:OBJECTIVE:Peer reviewers of biomedical journals are expected to perform a large number of roles and tasks, some of which are seemingly contradictory or demonstrate incongruities between the respective positions of peer reviewers and journal editors. Our aim was to explore the perspectives, expectations and understanding of the roles and tasks of peer reviewers of journal editors from general and specialty biomedical journals. DESIGN:Qualitative study. SETTING:Worldwide. PARTICIPANTS:56 journal editors from biomedical journals, most of whom were editors-in-chief (n=39), male (n=40) and worked part-time (n=50) at journals from 22 different publishers. METHODS:Semistructured interviews with journal editors were conducted. Recruitment was based on purposive maximum variation sampling. Data were analysed thematically following the methodology by Braun and Clarke. RESULTS:Journal editors' understanding of the roles and partly of tasks of peer reviewers are profoundly shaped by each journal's unique context and characteristics, including financial and human resources and journal reputation or prestige. There was a broad agreement among journal editors on expected technical tasks of peer reviewers related to scientific aspects, but there were different expectations in the level of depth. We also found that most journal editors support the perspective that authorship experience is key to high-quality reviews, while formal training in peer reviewing is not. CONCLUSION:These journal editors' accounts reveal issues of a social nature within the peer-review process related to missed opportunities for journal editors to engage with peer reviewers to clarify the expected roles and tasks.Further research is needed on actual performance of peer reviewers looking into the content of peer-reviewer reports to inform meaningful training interventions, journal policies and guidelines.
| S-EPMC6886905 | biostudies-literature
Project description:BACKGROUND:Although peer reviewers play a key role in the manuscript review process, their roles and tasks are poorly defined. Clarity around this issue is important as it may influence the quality of peer reviewer reports. This scoping review explored the roles and tasks of peer reviewers of biomedical journals. METHODS:Comprehensive literature searches were conducted in Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science from inception up to May 2017. There were no date and language restrictions. We also searched for grey literature. Studies with statements mentioning roles, tasks and competencies pertaining to the role of peer reviewers in biomedical journals were eligible for inclusion. Two reviewers independently performed study screening and selection. Relevant statements were extracted, collated and classified into themes. RESULTS:After screening 2763 citations and 600 full-text papers, 209 articles and 13 grey literature sources were included. A total of 1426 statements related to roles were extracted, resulting in 76 unique statements. These were grouped into 13 emergent themes: proficient experts in their field (3 items), dutiful/altruistic towards scientific community (7 items), familiar with journal (2 items), unbiased and ethical professionals (18 items), self-critical professionals (4 items), reliable professionals (7 items), skilled critics (15 items), respectful communicators (6 items), gatekeepers (2 items), educators (2 items), advocates for author/editor/reader (3 items) and advisors to editors (2 items). Roles that do not fall within the remit of peer reviewers were also identified (5 items). We also extracted 2026 statements related to peer reviewers' tasks, resulting in 73 unique statements. These were grouped under six themes: organisation and approach to reviewing (10 items), make general comments (10 items), assess and address content for each section of the manuscript (36 items), address ethical aspects (5 items), assess manuscript presentation (8 items) and provide recommendations (4 items). CONCLUSIONS:Peer reviewers are expected to perform a large number of roles and tasks for biomedical journals. These warrant further discussion and clarification in order not to overburden these key actors.
| S-EPMC6585141 | biostudies-literature
Project description:INTRODUCTION:Despite dealing with scientific output and potentially having an impact on the quality of research published, the manuscript peer-review process itself has at times been criticised for being 'unscientific'. Research indicates that there are social and subjective dimensions of the peer-review process that contribute to this perception, including how key stakeholders-namely authors, editors and peer reviewers-communicate. In particular, it has been suggested that the expected roles and tasks of stakeholders need to be more clearly defined and communicated if the manuscript review process is to be improved. Disentangling current communication practices, and outlining the specific roles and tasks of the main actors, might be a first step towards establishing the design of interventions that counterbalance social influences on the peer-review process.The purpose of this article is to present a methodological design for a qualitative study exploring the communication practices within the manuscript review process of biomedical journals from the journal editors' point of view. METHODS AND ANALYSIS:Semi-structured interviews will be carried out with editors of biomedical journals between October 2017 and February 2018. A heterogeneous sample of participants representing a wide range of biomedical journals will be sought through purposive maximum variation sampling, drawing from a professional network of contacts, publishers, conference participants and snowballing.Interviews will be thematically analysed following the method outlined by Braun and Clarke. The qualitative data analysis software NVivo V.11 will be used to aid data management and analysis. ETHICS AND DISSEMINATION:This research project was evaluated and approved by the University of Split, Medical School Ethics Committee (2181-198-03-04-17-0029) in May 2017. Findings will be disseminated through a publication in a peer-reviewed journal and presentations during conferences.
| S-EPMC6196803 | biostudies-other
Project description:BACKGROUND:In biomedical research, there have been numerous scandals highlighting conflicts of interest (COIs) leading to significant bias in judgment and questionable practices. Academic institutions, journals, and funding agencies have developed and enforced policies to mitigate issues related to COI, especially surrounding financial interests. After a case of editorial COI in a prominent bioethics journal, there is concern that the same level of oversight regarding COIs in the biomedical sciences may not apply to the field of bioethics. In this study, we examined the availability and comprehensiveness of COI policies for authors, peer reviewers, and editors of bioethics journals. METHODS:After developing a codebook, we analyzed the content of online COI policies of 63 bioethics journals, along with policy information provided by journal editors that was not publicly available. RESULTS:Just over half of the bioethics journals had COI policies for authors (57%), and only 25% for peer reviewers and 19% for editors. There was significant variation among policies regarding definitions, the types of COIs described, the management mechanisms, and the consequences for noncompliance. Definitions and descriptions centered on financial COIs, followed by personal and professional relationships. Almost all COI policies required disclosure of interests for authors as the primary management mechanism. Very few journals outlined consequences for noncompliance with COI policies or provided additional resources. CONCLUSION:Compared to other studies of biomedical journals, a much lower percentage of bioethics journals have COI policies and these vary substantially in content. The bioethics publishing community needs to develop robust policies for authors, peer reviewers, and editors and these should be made publicly available to enhance academic and public trust in bioethics scholarship.
| S-EPMC6310149 | biostudies-literature
Project description:INTRODUCTION:The primary functions of peer reviewers are poorly defined. Thus far no body of literature has systematically identified the roles and tasks of peer reviewers of biomedical journals. A clear establishment of these can lead to improvements in the peer review process. The purpose of this scoping review is to determine what is known on the roles and tasks of peer reviewers. METHODS:We will use the methodological framework first proposed by Arksey and O'Malley and subsequently adapted by Levac et al and the Joanna Briggs Institute. The scoping review will include all study designs, as well as editorials, commentaries and grey literature. The following eight electronic databases will be searched (from inception to May 2017): Cochrane Library, Cumulative Index to Nursing and Allied Health Literature, Educational Resources Information Center, EMBASE, MEDLINE, PsycINFO, Scopus and Web of Science. Two reviewers will use inclusion and exclusion criteria based on the 'Population-Concept-Context' framework to independently screen titles and abstracts of articles considered for inclusion. Full-text screening of relevant eligible articles will also be carried out by two reviewers. The search strategy for grey literature will include searching in websites of existing networks, biomedical journal publishers and organisations that offer resources for peer reviewers. In addition we will review journal guidelines to peer reviewers on how to perform the manuscript review. Journals will be selected using the 2016 journal impact factor. We will identify and assess the top five, middle five and lowest-ranking five journals across all medical specialties. ETHICS AND DISSEMINATION:This scoping review will undertake a secondary analysis of data already collected and does not require ethical approval. The results will be disseminated through journals and conferences targeting stakeholders involved in peer review in biomedical research.
| S-EPMC5665269 | biostudies-other
Project description:BackgroundPre-publication peer review of manuscripts should enhance the value of research publications to readers who may wish to utilize findings in clinical care or health policy-making. Much published research across all medical specialties is not useful, may be misleading, wasteful and even harmful. Reporting guidelines are tools that in addition to helping authors prepare better manuscripts may help peer reviewers in assessing them. We examined journals' instructions to peer reviewers to see if and how reviewers are encouraged to use them.MethodsWe surveyed websites of 116 journals from the McMaster list. Main outcomes were 1) identification of online instructions to peer reviewers and 2) presence or absence of key domains within instructions: on journal logistics, reviewer etiquette and addressing manuscript content (11 domains).FindingsOnly 41/116 journals (35%) provided online instructions. All 41 guided reviewers about the logistics of their review processes, 38 (93%) outlined standards of behaviour expected and 39 (95%) contained instruction about evaluating the manuscript content. There was great variation in explicit instruction for reviewers about how to evaluate manuscript content. Almost half of the online instructions 19/41 (46%) mentioned reporting guidelines usually as general statements suggesting they may be useful or asking whether authors had followed them rather than clear instructions about how to use them. All 19 named CONSORT for reporting randomized trials but there was little mention of CONSORT extensions. PRISMA, QUOROM (forerunner of PRISMA), STARD, STROBE and MOOSE were mentioned by several journals. No other reporting guideline was mentioned by more than two journals.ConclusionsAlthough almost half of instructions mentioned reporting guidelines, their value in improving research publications is not being fully realised. Journals have a responsibility to support peer reviewers. We make several recommendations including wider reference to the EQUATOR Network online library (www.equator-network.org/).
| S-EPMC3338712 | biostudies-literature
Project description:BackgroundSmartphones are being increasingly used for research owing to their multifunctionality and flexibility, and crowdsourced research using smartphone applications (apps) is effective in the early detection and management of chronic diseases. We developed the AllerSearch app to gather real-world data on individual subjective symptoms and lifestyle factors related to hay fever. This study established a foundation for interactive research by adopting novel, diverse perspectives accrued through implementing the principles of patient and public involvement (PPI) in the development of our app.MethodsPatients and members of the public with a history or family history of hay fever were recruited from November 2019 to December 2021 through a dedicated website, social networking services, and web briefing according to the PPI Guidebook 2019 by the Japan Agency for Medical Research and Development. Nine opinion exchange meetings were held from February 2020 to December 2021 to collect opinions and suggestions for updating the app. After each meeting, interactive evaluations from PPI contributors and researchers were collected. The compiled suggestions were then incorporated into the app, establishing an active feedback loop fed by the consistently interactive infrastructure.ResultsFour PPI contributors (one man and three women) were recruited, and 93 items were added/changed in the in-app survey questionnaire in accordance with discussions from the exchange meetings. The exchange meetings emphasized an atmosphere and opportunity for participants to speak up, ensuring frequent opportunities for them to contribute to the research. In March 2020, a public website was created to display real-time outcomes of the number of participants and users' hay-fever-preventative behaviors. In August 2020, a new PPI-implemented AllerSearch app was released.ConclusionsThis study marks the first research on clinical smartphone apps for hay fever in Japan that implements PPI throughout its timeline from research and development to the publication of research results. Taking advantage of the distinct perspectives offered by PPI contributors, a step was taken toward actualizing a foundation for an interactive research environment. These results should promote future PPI research and foster the establishment of a social construct that enables PPI efforts in various fields.
| S-EPMC9437402 | biostudies-literature
Project description:Background:There is concern in the academic publishing community that it is becoming more difficult to secure reviews for peer-reviewed manuscripts, but much of this concern stems from anecdotal and rhetorical evidence. Methods:We examined the proportion of review requests that led to a completed review over a 6-year period (2009-2015) in a mid-tier biology journal (Molecular Ecology). We also re-analyzed previously published data from four other mid-tier ecology journals (Functional Ecology, Journal of Ecology, Journal of Animal Ecology, and Journal of Applied Ecology), looking at the same proportion over the period 2003 to 2010. Results:The data from Molecular Ecology showed no significant decrease through time in the proportion of requests that led to a review (proportion in 2009?=?0.47 (95 % CI?=?0.43 to 0.52), proportion in 2015?=?0.44 (95 % CI?=?0.40 to 0.48)). This proportion did decrease for three of the other ecology journals (changes in proportions from 2003 to 2010?=?-0.10, -0.18, and -0.09), while the proportion for the fourth (Functional Ecology) stayed roughly constant (change in proportion?=?-0.04). Conclusions:Overall, our data suggest that reviewer agreement rates have probably declined slightly but not to the extent suggested by the anecdotal and rhetorical evidence.
| S-EPMC5803631 | biostudies-literature
Project description:Peer review is the gold standard for scientific communication, but its ability to guarantee the quality of published research remains difficult to verify. Recent modeling studies suggest that peer review is sensitive to reviewer misbehavior, and it has been claimed that referees who sabotage work they perceive as competition may severely undermine the quality of publications. Here we examine which aspects of suboptimal reviewing practices most strongly impact quality, and test different mitigating strategies that editors may employ to counter them. We find that the biggest hazard to the quality of published literature is not selfish rejection of high-quality manuscripts but indifferent acceptance of low-quality ones. Bypassing or blacklisting bad reviewers and consulting additional reviewers to settle disagreements can reduce but not eliminate the impact. The other editorial strategies we tested do not significantly improve quality, but pairing manuscripts to reviewers unlikely to selfishly reject them and allowing revision of rejected manuscripts minimize rejection of above-average manuscripts. In its current form, peer review offers few incentives for impartial reviewing efforts. Editors can help, but structural changes are more likely to have a stronger impact.
| S-EPMC5633172 | biostudies-other
Project description:Background Publishing in scientific journals is one of the most important ways in which scientists disseminate research to their peers and to the wider public. Pre-publication peer review underpins this process, but peer review is subject to various criticisms and is under pressure from growth in the number of scientific publications. Methods Here we examine an element of the editorial process at eLife, in which the Reviewing Editor usually serves as one of the referees, to see what effect this has on decision times, decision type, and the number of citations. We analysed a dataset of 8,905 research submissions to eLife since June 2012, of which 2,747 were sent for peer review. This subset of 2747 papers was then analysed in detail. Results The Reviewing Editor serving as one of the peer reviewers results in faster decision times on average, with the time to final decision ten days faster for accepted submissions (n=1,405) and five days faster for papers that were rejected after peer review (n=1,099). Moreover, editors acting as reviewers had no effect on whether submissions were accepted or rejected, and a very small (but significant) effect on citation rates. Conclusions An important aspect of eLife's peer-review process is shown to be effective, given that decision times are faster when the Reviewing Editor serves as a reviewer. Other journals hoping to improve decision times could consider adopting a similar approach.
| S-EPMC4962294 | biostudies-literature