Dataset Information

High-throughput analysis suggests differences in journal false discovery rate by subject area and impact factor but not open access status.

ABSTRACT:

Background

A low replication rate has been reported in some scientific areas motivating the creation of resource intensive collaborations to estimate the replication rate by repeating individual studies. The substantial resources required by these projects limits the number of studies that can be repeated and consequently the generalizability of the findings. We extend the use of a method from Jager and Leek to estimate the false discovery rate for 94 journals over a 5-year period using p values from over 30,000 abstracts enabling the study of how the false discovery rate varies by journal characteristics.

Results

We find that the empirical false discovery rate is higher for cancer versus general medicine journals (p?=?9.801E-07, 95% CI: 0.045, 0.097; adjusted mean false discovery rate cancer?=?0.264 vs. general medicine?=?0.194). We also find that false discovery rate is negatively associated with log journal impact factor. A two-fold decrease in journal impact factor is associated with an average increase of 0.020 in FDR (p?=?2.545E-04). Conversely, we find no statistically significant evidence of a higher false discovery rate, on average, for Open Access versus closed access journals (p?=?0.320, 95% CI -?0.015, 0.046, adjusted mean false discovery rate Open Access?=?0.241 vs. closed access?=?0.225).

Conclusions

Our results identify areas of research that may need additional scrutiny and support to facilitate replicable science. Given our publicly available R code and data, others can complete a broad assessment of the empirical false discovery rate across other subject areas and characteristics of published research.

SUBMITTER: Hall LM

PROVIDER: S-EPMC7724881 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

High-throughput analysis suggests differences in journal false discovery rate by subject area and impact factor but not open access status.

Hall L M LM Hendricks A E AE

BMC bioinformatics 20201209 1

<h4>Background</h4>A low replication rate has been reported in some scientific areas motivating the creation of resource intensive collaborations to estimate the replication rate by repeating individual studies. The substantial resources required by these projects limits the number of studies that can be repeated and consequently the generalizability of the findings. We extend the use of a method from Jager and Leek to estimate the false discovery rate for 94 journals over a 5-year period using ...[more]

PMID: 33297936

Similar Datasets

Project description:BackgroundHead & Face Medicine (HFM) was launched in August 2005 to provide multidisciplinary science in the field of head and face disorders with an open access and open peer review publication platform. The objective of this study is to evaluate the characteristics of submissions, the effectiveness of open peer reviewing, and factors biasing the acceptance or rejection of submitted manuscripts.MethodsA 1-year period of submissions and all concomitant journal operations were retrospectively analyzed. The analysis included submission rate, reviewer rate, acceptance rate, article type, and differences in duration for peer reviewing, final decision, publishing, and PubMed inclusion. Statistical analysis included Mann-Whitney U test, Chi-square test, regression analysis, and binary logistic regression.ResultsHFM received 126 articles (10.5 articles/month) for consideration in the first year. Submissions have been increasing, but not significantly over time. Peer reviewing was completed for 82 articles and resulted in an acceptance rate of 48.8%. In total, 431 peer reviewers were invited (5.3/manuscript), of which 40.4% agreed to review. The mean peer review time was 37.8 days. The mean time between submission and acceptance (including time for revision) was 95.9 days. Accepted papers were published on average 99.3 days after submission. The mean time between manuscript submission and PubMed inclusion was 101.3 days. The main article types submitted to HFM were original research, reviews, and case reports. The article type had no influence on rejection or acceptance. The variable 'number of invited reviewers' was the only significant (p < 0.05) predictor for rejection of manuscripts.ConclusionThe positive trend in submissions confirms the need for publication platforms for multidisciplinary science. HFM's peer review time comes in shorter than the 6-weeks turnaround time the Editors set themselves as the maximum. Rejection of manuscripts was associated with the number of invited reviewers. None of the other parameters tested had any effect on the final decision. Thus, HFM's ethical policy, which is based on Open Access, Open Peer, and transparency of journal operations, is free of 'editorial bias' in accepting manuscripts. ORIGINAL DATA: Provided as a downloadable tab-delimited text file (URL and variable code available under section 'additional files').

Dataset Information

High-throughput analysis suggests differences in journal false discovery rate by subject area and impact factor but not open access status.

Background

Results

Conclusions

Publications

High-throughput analysis suggests differences in journal false discovery rate by subject area and impact factor but not open access status.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets