Gender bias in scholarly peer review.
Ontology highlight
ABSTRACT:
SUBMITTER: Helmer M
PROVIDER: S-EPMC5360442 | biostudies-literature | 2017 Mar
REPOSITORIES: biostudies-literature
Publications
Gender bias in scholarly peer review.
Helmer Markus M Schottdorf Manuel M Neef Andreas A Battaglia Demian D
eLife 20170321
Peer review is the cornerstone of scholarly publishing and it is essential that peer reviewers are appointed on the basis of their expertise alone. However, it is difficult to check for any bias in the peer-review process because the identity of peer reviewers generally remains confidential. Here, using public information about the identities of 9000 editors and 43000 reviewers from the Frontiers series of journals, we show that women are underrepresented in the peer-review process, that editors ...[more]
Similar Datasets
Project description:Scholarly journals are often blamed for a gender gap in publication rates, but it is unclear whether peer review and editorial processes contribute to it. This article examines gender bias in peer review with data for 145 journals in various fields of research, including about 1.7 million authors and 740,000 referees. We reconstructed three possible sources of bias, i.e., the editorial selection of referees, referee recommendations, and editorial decisions, and examined all their possible relationships. Results showed that manuscripts written by women as solo authors or coauthored by women were treated even more favorably by referees and editors. Although there were some differences between fields of research, our findings suggest that peer review and editorial processes do not penalize manuscripts by women. However, increasing gender diversity in editorial teams and referee pools could help journals inform potential authors about their attention to these factors and so stimulate participation by women.
| S-EPMC7787493 | biostudies-literature
Project description:IntroductionEducation scholarship requires peer reviewers. For novice scholars, reviewing is an important developmental activity that cultivates deeper participation in the scholarship community. Yet getting started with reviewing is challenging for those not involved with the educational scholarship community. Beyond scientific expertise, reviewers must have a mentoring mindset, skills in providing constructive feedback, and knowledge of common ethical challenges associated with producing and evaluating scholarship.MethodsOur workshop introduced novice health professions educators to peer reviewing. It included four stimulus presentations about the peer reviewer's mindset and skills, followed by reinforcement activities. The workshop could be adapted to variously sized groups. An 8:1 ratio of participants to facilitators was ideal for activities. Topics covered included considerations before accepting an invitation, the review process, the good citizen approach to reviewing, and reviewer ethics. The session concluded with suggestions for continued development of peer reviewer competencies. The workshop was evaluated using a custom survey.ResultsThroughout 2019 and early 2020, 58 health professions educators and trainees participated in the workshop across varied venues. Evaluations were obtained from 33 participants (57%). Nearly all rated the workshop as high quality and valuable to peer reviewer preparation. Most (26 of 33; 75%) gained confidence about their qualifications to serve as reviewers. Eighty percent (28 of 33) believed they could recognize ethical dilemmas.DiscussionThis workshop provided a springboard for peer reviewing health professions education scholarship. Participants generally praised the experience for introducing them to the world of peer review and preparing them for it.
| S-EPMC8335801 | biostudies-literature
Project description:ObjectiveTo review the literature on strategies implemented or identified to prevent or reduce gender bias in peer review of research grants.MethodsStudies of any type of qualitative or quantitative design examining interventions to reduce or prevent gender bias during the peer review of health-related research grants were included. Electronic databases including MEDLINE, EMBASE, Education Resources Information Center (ERIC), PsycINFO, Joanna Briggs, the Cochrane Library, Evidence Based Medicine (EBM) Reviews, and the Campbell Library were searched from 2005 to April 2016. A search for grey (i.e., difficult to locate or unpublished) literature was conducted and experts in the field were consulted to identify additional potentially relevant articles. Two individuals screened titles and abstracts, full-text articles, and abstracted data with discrepancies resolved by a third person consistently.ResultsAfter screening 5524 citations and 170 full-text articles, one article evaluating gender-blinding of grant applications using an uncontrolled before-after study design was included. In this study, 891 applications for long-term fellowships in 2006 were included and 47% of the applicants were women. These were scored by 13 peer reviewers (38% were women). The intervention included eliminating references to gender from the applications, letters of recommendations, and interview reports that were sent to the committee members for evaluation. The proportion of successful applications led by women did not change with gender-blinding, although the number of successful applications that were led by men increased slightly.ConclusionsThere is limited research on interventions to mitigate gender bias in the peer review of grants. Only one study was identified and no difference in the proportion of women who were successful in receiving grant funding was observed. Our results suggest that interventions to prevent gender bias should be adapted and tested in the context of grant peer review to determine if they will have an impact.
| S-EPMC5218731 | biostudies-literature
Project description:BackgroundThe emergence of systems based on large language models (LLMs) such as OpenAI's ChatGPT has created a range of discussions in scholarly circles. Since LLMs generate grammatically correct and mostly relevant (yet sometimes outright wrong, irrelevant or biased) outputs in response to provided prompts, using them in various writing tasks including writing peer review reports could result in improved productivity. Given the significance of peer reviews in the existing scholarly publication landscape, exploring challenges and opportunities of using LLMs in peer review seems urgent. After the generation of the first scholarly outputs with LLMs, we anticipate that peer review reports too would be generated with the help of these systems. However, there are currently no guidelines on how these systems should be used in review tasks.MethodsTo investigate the potential impact of using LLMs on the peer review process, we used five core themes within discussions about peer review suggested by Tennant and Ross-Hellauer. These include 1) reviewers' role, 2) editors' role, 3) functions and quality of peer reviews, 4) reproducibility, and 5) the social and epistemic functions of peer reviews. We provide a small-scale exploration of ChatGPT's performance regarding identified issues.ResultsLLMs have the potential to substantially alter the role of both peer reviewers and editors. Through supporting both actors in efficiently writing constructive reports or decision letters, LLMs can facilitate higher quality review and address issues of review shortage. However, the fundamental opacity of LLMs' training data, inner workings, data handling, and development processes raise concerns about potential biases, confidentiality and the reproducibility of review reports. Additionally, as editorial work has a prominent function in defining and shaping epistemic communities, as well as negotiating normative frameworks within such communities, partly outsourcing this work to LLMs might have unforeseen consequences for social and epistemic relations within academia. Regarding performance, we identified major enhancements in a short period and expect LLMs to continue developing.ConclusionsWe believe that LLMs are likely to have a profound impact on academia and scholarly communication. While potentially beneficial to the scholarly communication system, many uncertainties remain and their use is not without risks. In particular, concerns about the amplification of existing biases and inequalities in access to appropriate infrastructure warrant further attention. For the moment, we recommend that if LLMs are used to write scholarly reviews and decision letters, reviewers and editors should disclose their use and accept full responsibility for data security and confidentiality, and their reports' accuracy, tone, reasoning and originality.
| S-EPMC10191680 | biostudies-literature
Project description: Not available
| S-EPMC10257577 | biostudies-literature
Project description:For decades, the supra-inflation increase of subscription prices for scholarly journals has concerned scholarly institutions. After years of fruitless efforts to solve this “serials crisis”, open access has been proposed as the latest potential solution. However, also the prices for open access publishing are high and are rising well beyond inflation. What has been missing from the public discussion so far is a quantitative approach to determine the actual
| S-EPMC8276192 | biostudies-literature
Project description:To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier. By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees' willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.
| S-EPMC6338763 | biostudies-literature
Project description:Peer review may be "single-blind," in which reviewers are aware of the names and affiliations of paper authors, or "double-blind," in which this information is hidden. Noting that computer science research often appears first or exclusively in peer-reviewed conferences rather than journals, we study these two reviewing models in the context of the 10th Association for Computing Machinery International Conference on Web Search and Data Mining, a highly selective venue (15.6% acceptance rate) in which expert committee members review full-length submissions for acceptance. We present a controlled experiment in which four committee members review each paper. Two of these four reviewers are drawn from a pool of committee members with access to author information; the other two are drawn from a disjoint pool without such access. This information asymmetry persists through the process of bidding for papers, reviewing papers, and entering scores. Reviewers in the single-blind condition typically bid for 22% fewer papers and preferentially bid for papers from top universities and companies. Once papers are allocated to reviewers, single-blind reviewers are significantly more likely than their double-blind counterparts to recommend for acceptance papers from famous authors, top universities, and top companies. The estimated odds multipliers are tangible, at 1.63, 1.58, and 2.10, respectively.
| S-EPMC5715744 | biostudies-literature
Project description:ObjectivesTo examine whether the gender of applicants and peer reviewers and other factors influence peer review of grant proposals submitted to a national funding agency.SettingSwiss National Science Foundation (SNSF).DesignCross-sectional analysis of peer review reports submitted from 2009 to 2016 using linear mixed effects regression models adjusted for research topic, applicant's age, nationality, affiliation and calendar period.ParticipantsExternal peer reviewers.Primary outcome measureOverall score on a scale from 1 (worst) to 6 (best).ResultsAnalyses included 38 250 reports on 12 294 grant applications from medicine, architecture, biology, chemistry, economics, engineering, geology, history, linguistics, mathematics, physics, psychology and sociology submitted by 26 829 unique peer reviewers. In univariable analysis, male applicants received more favourable evaluation scores than female applicants (+0.18 points; 95% CI 0.14 to 0.23), and male reviewers awarded higher scores than female reviewers (+0.11; 95% CI 0.08 to 0.15). Applicant-nominated reviewers awarded higher scores than reviewers nominated by the SNSF (+0.53; 95% CI 0.50 to 0.56), and reviewers from outside of Switzerland more favourable scores than reviewers affiliated with Swiss institutions (+0.53; 95% CI 0.49 to 0.56). In multivariable analysis, differences between male and female applicants were attenuated (+0.08; 95% CI 0.04 to 0.13) whereas results changed little for source of nomination and affiliation of reviewers. The gender difference increased after September 2011, when new evaluation forms were introduced (p=0.033 from test of interaction).ConclusionsPeer review of grant applications at SNSF might be prone to biases stemming from different applicant and reviewer characteristics. The SNSF abandoned the nomination of peer reviewers by applicants. The new form introduced in 2011 may inadvertently have given more emphasis to the applicant's track record. We encourage other funders to conduct similar studies, in order to improve the evidence base for rational and fair research funding.
| S-EPMC7440717 | biostudies-literature
Project description:PurposeTo systematically review experimental evidence for interventions mitigating gender bias in employment. Unconscious endorsement of gender stereotypes can undermine academic medicine's commitment to gender equity.MethodThe authors performed electronic and hand searches for randomized controlled studies since 1973 of interventions that affect gender differences in evaluation of job applicants. Twenty-seven studies met all inclusion criteria. Interventions fell into three categories: application information, applicant features, and rating conditions.ResultsThe studies identified gender bias as the difference in ratings or perceptions of men and women with identical qualifications. Studies reaffirmed negative bias against women being evaluated for positions traditionally or predominantly held by men (male sex-typed jobs). The assessments of male and female raters rarely differed. Interventions that provided raters with clear evidence of job-relevant competencies were effective. However, clearly competent women were rated lower than equivalent men for male sex-typed jobs unless evidence of communal qualities was also provided. A commitment to the value of credentials before review of applicants and women's presence at above 25% of the applicant pool eliminated bias against women. Two studies found unconscious resistance to "antibias" training, which could be overcome with distraction or an intervening task. Explicit employment equity policies and an attractive appearance benefited men more than women, whereas repeated employment gaps were more detrimental to men. Masculine-scented perfume favored the hiring of both sexes. Negative bias occurred against women who expressed anger or who were perceived as self-promoting.ConclusionsHigh-level evidence exists for strategies to mitigate gender bias in hiring.
| S-EPMC4554714 | biostudies-literature