Dataset Information

Imbalance in individual researcher's peer review activities quantified for four British Ecological Society journals, 2003-2010.

ABSTRACT: Researchers contribute to the scientific peer review system by providing reviews, and "withdraw" from it by submitting manuscripts that are subsequently reviewed. So far as we are aware, there has been no quantification of the balance of individual's contributions and withdrawals. We compared the number of reviews provided by individual researchers (i.e., their contribution) to the number required by their submissions (i.e. their withdrawals) in a large and anonymised database provided by the British Ecological Society. The database covered the Journal of Ecology, Journal of Animal Ecology, Journal of Applied Ecology, and Functional Ecology from 2003-2010. The majority of researchers (64%) did not have balanced contributions and withdrawals. Depending on assumptions, 12% to 44% contributed more than twice as much as required; 20% to 52% contributed less than half as much as required. Balance, or lack thereof, varied little in relation to the number of years a researcher had been active (reviewing or submitting). Researchers who contributed less than required did not lack the opportunity to review. Researchers who submitted more were more likely to accept invitations to review. These finding suggest overall that peer review of the four analysed journals is not in crisis, but only due to the favourable balance of over- and under-contributing researchers. These findings are limited to the four journals analysed, and therefore cannot include researcher's other peer review activities, which if included might change the proportions reported. Relatively low effort was required to assemble, check, and analyse the data. Broader analyses of individual researcher's peer review activities would contribute to greater quality, efficiency, and fairness in the peer review system.

SUBMITTER: Petchey OL

PROVIDER: S-EPMC3962470 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Imbalance in individual researcher's peer review activities quantified for four British Ecological Society journals, 2003-2010.

Petchey Owen L OL Fox Jeremy W JW Haddon Lindsay L

PloS one 20140321 3

Researchers contribute to the scientific peer review system by providing reviews, and "withdraw" from it by submitting manuscripts that are subsequently reviewed. So far as we are aware, there has been no quantification of the balance of individual's contributions and withdrawals. We compared the number of reviews provided by individual researchers (i.e., their contribution) to the number required by their submissions (i.e. their withdrawals) in a large and anonymised database provided by the Br ...[more]

PMID: 24658631

Similar Datasets

Project description:The field of law has retained its distinctiveness regarding peer review to this day, and reviews are often conducted without following standardized rules and principles. External and independent evaluation of submissions has recently become adopted by European law journals, and peer review procedures are still poorly defined, investigated, and attuned to the legal science publishing landscape. The aim of our study was to gain a better insight into current editorial policies on peer review in law journals by exploring editorial documents (instructions, guidelines, policies) issued by 119 Croatian, Italian, and Spanish law journals. We relied on automatic content analysis of 135 publicly available documents collected from the journal websites to analyze the basic features of the peer review processes, manuscript evaluation criteria, and related ethical issues using WordStat8. Differences in covered topics between the countries were compared using the chi-square test. Our findings reveal that most law journals have adopted a traditional approach, in which the editorial board manages mostly anonymized peer review (104, 77%) engaging independent/external reviewers (65, 48%). Submissions are evaluated according to their originality and relevance (113, 84%), quality of writing and presentation (94, 70%), comprehensiveness of literature references (93, 69%), and adequacy of methods (57, 42%). The main ethical issues related to peer review addressed by these journals are reviewer's competing interests (42, 31%), plagiarism (35, 26%), and biases (30, 22%). We observed statistically significant differences between countries in mentioning key concepts such as "Peer review ethics", "Reviewer", "Transparency of identities", "Publication type", and "Research misconduct". Spanish journals favor reviewers' "Independence" and "Competence" and "Anonymized" peer review process. Also, some manuscript types popular in one country are rarely mentioned in other countries. Even though peer review is equally conventional in all three countries, high transparency in Croatian law journals, respect for research integrity in Spanish ones, and diversity and inclusion in Italian are promising indicators of future development.

Project description:ImportanceHigh-quality peer reviews are often thought to be essential to ensuring the integrity of the scientific publication process, but measuring peer review quality is challenging. Although imperfect, review word count could potentially serve as a simple, objective metric of review quality.ObjectiveTo determine the prevalence of very short peer reviews and how often they inform editorial decisions on research articles in 3 leading general medical journals.Design, setting, and participantsThis cross-sectional study compiled a data set of peer reviews from published, full-length original research articles from 3 general medical journals (The BMJ, PLOS Medicine, and BMC Medicine) between 2003 and 2022. Eligible articles were those with peer review data; all peer reviews used to make the first editorial decision (ie, accept vs revise and resubmit) were included.Main outcomes and measuresPrevalence of very short reviews was the primary outcome, which was defined as a review of fewer than 200 words. In secondary analyses, thresholds of fewer than 100 words and fewer than 300 words were used. Results were disaggregated by journal and year. The proportion of articles for which the first editorial decision was made based on a set of peer reviews in which very short reviews constituted 100%, 50% or more, 33% or more, and 20% or more of the reviews was calculated.ResultsIn this sample of 11 466 reviews (including 6086 in BMC Medicine, 3816 in The BMJ, and 1564 in PLOS Medicine) corresponding to 4038 published articles, the median (IQR) word count per review was 425 (253-575) words, and the mean (SD) word count was 520.0 (401.0) words. The overall prevalence of very short (<200 words) peer reviews was 1958 of 11 466 reviews (17.1%). Across the 3 journals, 843 of 4038 initial editorial decisions (20.9%) were based on review sets containing 50% or more very short reviews. The prevalence of very short reviews and share of editorial decisions based on review sets containing 50% or more very short reviews was highest for BMC Medicine (693 of 2585 editorial decisions [26.8%]) and lowest for The BMJ (76 of 1040 editorial decisions [7.3%]).Conclusion and relevanceIn this study of 3 leading general medical journals, one-fifth of initial editorial decisions for published articles were likely based at least partially on reviews of such short length that they were unlikely to be of high quality. Future research could determine whether monitoring peer review length improves the quality of peer reviews and which interventions, such as incentives and norm-based interventions, may elicit more detailed reviews.

Project description:ObjectiveWe undertook this investigation to characterize conflict of interest (COI) policies of biomedical journals with respect to authors, peer-reviewers, and editors, and to ascertain what information about COI disclosures is publicly available.MethodsWe performed a cross-sectional survey of a convenience sample of 135 editors of peer-reviewed biomedical journals that publish original research. We chose an international selection of general and specialty medical journals that publish in English. Selection was based on journal impact factor, and the recommendations of experts in the field. We developed and pilot tested a 3-part web-based survey. The survey included questions about the presence of specific policies for authors, peer-reviewers, and editors, specific restrictions on authors, peer-reviewers, and editors based on COI, and the public availability of these disclosures. Editors were contacted a minimum of 3 times.ResultsThe response rate for the survey was 91 (67%) of 135, and 85 (93%) of 91 journals reported having an author COI policy. Ten (11%) journals reported that they restrict author submissions based on COI (e.g., drug company authors' papers on their products are not accepted). While 77% report collecting COI information on all author submissions, only 57% publish all author disclosures. A minority of journals report having a specific policy on peer-reviewer 46% (42/91) or editor COI 40% (36/91); among these, 25% and 31% of journals state that they require recusal of peer-reviewers and editors if they report a COI. Only 3% of respondents publish COI disclosures of peer-reviewers, and 12% publish editor COI disclosures, while 11% and 24%, respectively, reported that this information is available upon request.ConclusionMany more journals have a policy regarding COI for authors than they do for peer-reviewers or editors. Even author COI policies are variable, depending on the type of manuscript submitted. The COI information that is collected by journals is often not published; the extent to which such "secret disclosure" may impact the integrity of the journal or the published work is not known.

Dataset Information

Imbalance in individual researcher's peer review activities quantified for four British Ecological Society journals, 2003-2010.

Publications

Imbalance in individual researcher's peer review activities quantified for four British Ecological Society journals, 2003-2010.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets