Dataset Information

Unprofessional peer reviews disproportionately harm underrepresented groups in STEM.

ABSTRACT: Background:Peer reviewed research is paramount to the advancement of science. Ideally, the peer review process is an unbiased, fair assessment of the scientific merit and credibility of a study; however, well-documented biases arise in all methods of peer review. Systemic biases have been shown to directly impact the outcomes of peer review, yet little is known about the downstream impacts of unprofessional reviewer comments that are shared with authors. Methods:In an anonymous survey of international participants in science, technology, engineering, and mathematics (STEM) fields, we investigated the pervasiveness and author perceptions of long-term implications of receiving of unprofessional comments. Specifically, we assessed authors' perceptions of scientific aptitude, productivity, and career trajectory after receiving an unprofessional peer review. Results:We show that survey respondents across four intersecting categories of gender and race/ethnicity received unprofessional peer review comments equally. However, traditionally underrepresented groups in STEM fields were most likely to perceive negative impacts on scientific aptitude, productivity, and career advancement after receiving an unprofessional peer review. Discussion:Studies show that a negative perception of aptitude leads to lowered self-confidence, short-term disruptions in success and productivity and delays in career advancement. Therefore, our results indicate that unprofessional reviews likely have and will continue to perpetuate the gap in STEM fields for traditionally underrepresented groups in the sciences.

SUBMITTER: Silbiger NJ

PROVIDER: S-EPMC6911688 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Unprofessional peer reviews disproportionately harm underrepresented groups in STEM.

Silbiger Nyssa J NJ Stubler Amber D AD

PeerJ 20191212

<h4>Background</h4>Peer reviewed research is paramount to the advancement of science. Ideally, the peer review process is an unbiased, fair assessment of the scientific merit and credibility of a study; however, well-documented biases arise in all methods of peer review. Systemic biases have been shown to directly impact the outcomes of peer review, yet little is known about the downstream impacts of unprofessional reviewer comments that are shared with authors.<h4>Methods</h4>In an anonymous su ...[more]

PMID: 31844596

Similar Datasets

Project description:Peer Review is at the heart of scholarly communications and the cornerstone of scientific publishing. However, academia often criticizes the peer review system as non-transparent, biased, arbitrary, a flawed process at the heart of science, leading to researchers arguing with its reliability and quality. These problems could also be due to the lack of studies with the peer-review texts for various proprietary and confidentiality clauses. Peer review texts could serve as a rich source of Natural Language Processing (NLP) research on understanding the scholarly communication landscape, and thereby build systems towards mitigating those pertinent problems. In this work, we present a first of its kind multi-layered dataset of 1199 open peer review texts manually annotated at the sentence level (∼ 17k sentences) across the four layers, viz. Paper Section Correspondence, Paper Aspect Category, Review Functionality, and Review Significance. Given a text written by the reviewer, we annotate: to which sections (e.g., Methodology, Experiments, etc.), what aspects (e.g., Originality/Novelty, Empirical/Theoretical Soundness, etc.) of the paper does the review text correspond to, what is the role played by the review text (e.g., appreciation, criticism, summary, etc.), and the importance of the review statement (major, minor, general) within the review. We also annotate the sentiment of the reviewer (positive, negative, neutral) for the first two layers to judge the reviewer's perspective on the different sections and aspects of the paper. We further introduce four novel tasks with this dataset, which could serve as an indicator of the exhaustiveness of a peer review and can be a step towards the automatic judgment of review quality. We also present baseline experiments and results for the different tasks for further investigations. We believe our dataset would provide a benchmark experimental testbed for automated systems to leverage on current NLP state-of-the-art techniques to address different issues with peer review quality, thereby ushering increased transparency and trust on the holy grail of scientific research validation. Our dataset and associated codes are available at https://www.iitp.ac.in/~ai-nlp-ml/resources.html#Peer-Review-Analyze.

Project description:Internationally, studies show that similar levels of alcohol consumption in deprived communities (vs. more affluent) result in higher levels of alcohol-related ill health. Hypotheses to explain this alcohol harm paradox include deprived drinkers: suffering greater combined health challenges (e.g. smoking, obesity) which exacerbate effects of alcohol harms; exhibiting more harmful consumption patterns (e.g. bingeing); having a history of more harmful consumption; and disproportionately under-reporting consumption. We use a bespoke national survey to assess each of these hypotheses.A national telephone survey designed to test this alcohol harm paradox was undertaken (May 2013 to April 2014) with English adults (n = 6015). Deprivation was assigned by area of residence. Questions examined factors including: current and historic drinking patterns; combined health challenges (smoking, diet, exercise and body mass); and under-reported consumption (enhanced questioning on atypical/special occasion drinking). For each factor, analyses examined differences between deprived and more affluent individuals controlled for total alcohol consumption.Independent of total consumption, deprived drinkers were more likely to smoke, be overweight and report poor diet and exercise. Consequently, deprived increased risk drinkers (male >168-400 g, female >112-280 g alcohol/week) were >10 times more likely than non-deprived counterparts to drink in a behavioural syndrome combining smoking, excess weight and poor diet/exercise. Differences by deprivation were significant but less marked in higher risk drinkers (male >400 g, female >280 g alcohol/week). Current binge drinking was associated with deprivation independently of total consumption and a history of bingeing was also associated with deprivation in lower and increased risk drinkers.Deprived increased/higher drinkers are more likely than affluent counterparts to consume alcohol as part of a suite of health challenging behaviours including smoking, excess weight and poor diet/exercise. Together these can have multiplicative effects on risks of wholly (e.g. alcoholic liver disease) and partly (e.g. cancers) alcohol-related conditions. More binge drinking in deprived individuals will also increase risks of injury and heart disease despite total alcohol consumption not differing from affluent counterparts. Public health messages on how smoking, poor diet/exercise and bingeing escalate health risks associated with alcohol are needed, especially in deprived communities, as their absence will contribute to health inequalities.

Project description:BACKGROUND:Adjuvant chemotherapy in early stage breast cancer has been shown to reduce mortality in a large meta-analysis of over 100 randomised trials. However, these trials largely excluded patients aged 70 years and over or with higher levels of comorbidity. There is therefore uncertainty about whether the effectiveness of adjuvant chemotherapy generalises to these groups, hindering patient and clinician decision-making. This study utilises administrative healthcare data-real world data (RWD)-and econometric methods for causal analysis to estimate treatment effectiveness in these trial-underrepresented groups. METHODS AND FINDINGS:Women with early breast cancer aged 70 years and over and those under 70 years with a high level of comorbidity were identified and their records extracted from Scottish Cancer Registry (2001-2015) data linked to other routine health records. A high level of comorbidity was defined as scoring 1 or more on the Charlson comorbidity index, being in the top decile of inpatient stays, and/or having 5 or more visits to specific outpatient clinics, all within the 5 years preceding breast cancer diagnosis. Propensity score matching (PSM) and instrumental variable (IV) analysis, previously identified as feasible and valid in this setting, were used in conjunction with Cox regression to estimate hazard ratios for death from breast cancer and death from all causes. The analysis adjusts for age, clinical prognostic factors, and socioeconomic deprivation; the IV method may also adjust for unmeasured confounding factors. Cohorts of 9,653 and 7,965 were identified for women aged 70 years and over and those with high comorbidity, respectively. In the ?70/high comorbidity cohorts, median follow-up was 5.17/6.53 years and there were 1,935/740 deaths from breast cancer. For women aged 70 years and over, the PSM-estimated HR was 0.73 (95% CI 0.64-0.95), while for women with high comorbidity it was 0.67 (95% CI 0.51-0.86). This translates to a mean predicted benefit in terms of overall survival at 10 years of approximately3% (percentage points) and 4%, respectively. A limitation of this analysis is that use of observational data means uncertainty remains both from sampling uncertainty and from potential bias from residual confounding. CONCLUSIONS:The results of this study, as RWD, should be interpreted with caution and in the context of existing and emerging randomised data. The relative effectiveness of adjuvant chemotherapy in reducing mortality in patients with early stage breast cancer appears to be generalisable to the selected trial-underrepresented groups.

Dataset Information

Unprofessional peer reviews disproportionately harm underrepresented groups in STEM.

Publications

Unprofessional peer reviews disproportionately harm underrepresented groups in STEM.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets