Dataset Information

Fiddle: a tool to combat publication bias by getting research out of the file drawer and into the scientific community.

ABSTRACT: Statistically significant findings are more likely to be published than non-significant or null findings, leaving scientists and healthcare personnel to make decisions based on distorted scientific evidence. Continuously expanding ´file drawers' of unpublished data from well-designed experiments waste resources creates problems for researchers, the scientific community and the public. There is limited awareness of the negative impact that publication bias and selective reporting have on the scientific literature. Alternative publication formats have recently been introduced that make it easier to publish research that is difficult to publish in traditional peer reviewed journals. These include micropublications, data repositories, data journals, preprints, publishing platforms, and journals focusing on null or neutral results. While these alternative formats have the potential to reduce publication bias, many scientists are unaware that these formats exist and don't know how to use them. Our open source file drawer data liberation effort (fiddle) tool (RRID:SCR_017327 available at: http://s-quest.bihealth.org/fiddle/) is a match-making Shiny app designed to help biomedical researchers to identify the most appropriate publication format for their data. Users can search for a publication format that meets their needs, compare and contrast different publication formats, and find links to publishing platforms. This tool will assist scientists in getting otherwise inaccessible, hidden data out of the file drawer into the scientific community and literature. We briefly highlight essential details that should be included to ensure reporting quality, which will allow others to use and benefit from research published in these new formats.

SUBMITTER: Bernard R

PROVIDER: S-EPMC7593522 | biostudies-literature | 2020 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

fiddle: a tool to combat publication bias by getting research out of the file drawer and into the scientific community.

Bernard René R Weissgerber Tracey L TL Bobrov Evgeny E Winham Stacey J SJ Dirnagl Ulrich U Riedel Nico N

Clinical science (London, England : 1979) 20201001 20

Statistically significant findings are more likely to be published than non-significant or null findings, leaving scientists and healthcare personnel to make decisions based on distorted scientific evidence. Continuously expanding ´file drawers' of unpublished data from well-designed experiments waste resources creates problems for researchers, the scientific community and the public. There is limited awareness of the negative impact that publication bias and selective reporting have on the scie ...[more]

PMID: 33111948

Similar Datasets

Project description:Infection experiments are critical for understanding wildlife disease dynamics. Although infection experiments are typically designed to reduce complexity, disease outcomes still result from complex interactions between host, pathogen, and environmental factors. Cryptic variation across factors can lead to decreased repeatability of infection experiments within and between research groups and hinder research progress. Furthermore, studies with unexpected results are often relegated to the "file drawer" and potential insights gained from these experimental outcomes are lost. Here, we report unexpected results from an infection experiment studying the response of two differentially-susceptible but related frogs (American Bullfrog Rana catesbeiana and the Mountain yellow-legged frog Rana muscosa) to the amphibian-killing chytrid fungus (Batrachochytrium dendrobatidis, Bd). Despite well-documented differences in susceptibility between species, we found no evidence for antibody-mediated immune response and no Bd-related mortality in either species. Additionally, during the study, the sham-inoculated R. catesbeiana control group became unexpectedly Bd-positive. We used a custom genotyping assay to demonstrate that the aberrantly-infected R. catesbeiana carried a Bd genotype distinct from the inoculation genotype. Thus R. catesbeiana individuals were acquired with low-intensity infections that could not be detected with qPCR. In the Bd-inoculated R. catesbeiana treatment group, the inoculated genotype appeared to out-compete the cryptic infection. Thus, our results provide insight into Bd coinfection dynamics, a phenomenon that is increasingly relevant as different pathogen strains are moved around the globe. Our experiment highlights how unexpected experimental outcomes can serve as both cautionary tales and opportunities to explore unanswered research questions. We use our results as a case study to highlight common sources of anomalous results for infection experiments. We argue that understanding these factors will aid researchers in the design, execution, and interpretation of experiments to understand wildlife disease processes.

Project description:BackgroundSystematic reviews and meta-analyses of pre-clinical studies, in vivo animal experiments in particular, can influence clinical care. Publication bias is one of the major threats of validity in systematic reviews and meta-analyses. Previous empirical studies suggested that systematic reviews and meta-analyses have become more prevalent until 2010 and found evidence for compromised methodological rigor with a trend towards improvement. We aim to comprehensively summarize and update the evidence base on systematic reviews and meta-analyses of animal studies, their methodological quality and assessment of publication bias in particular.Methods/designThe objectives of this systematic review are as follows: •To investigate the epidemiology of published systematic reviews of animal studies until present. •To examine methodological features of systematic reviews and meta-analyses of animal studies with special attention to the assessment of publication bias. •To investigate the influence of systematic reviews of animal studies on clinical research by examining citations of the systematic reviews by clinical studies. Eligible studies for this systematic review constitute systematic reviews and meta-analyses that summarize in vivo animal experiments with the purpose of reviewing animal evidence to inform human health. We will exclude genome-wide association studies and animal experiments with the main purpose to learn more about fundamental biology, physical functioning or behavior. In addition to the inclusion of systematic reviews and meta-analyses identified by other empirical studies, we will systematically search Ovid Medline, Embase, ToxNet, and ScienceDirect from 2009 to January 2013 for further eligible studies without language restrictions. Two reviewers working independently will assess titles, abstracts, and full texts for eligibility and extract relevant data from included studies. Data reporting will involve a descriptive summary of meta-analyses and systematic reviews.DiscussionResults are expected to be publicly available later in 2013 and may form the basis for recommendations to improve the quality of systematic reviews and meta-analyses of animal studies and their use with respect to clinical care.

Project description:Background:Publication bias is a form of scientific misconduct. It threatens the validity of research results and the credibility of science. Although several tests on publication bias exist, no in-depth evaluations are available that examine which test performs best for different research settings. Methods:Four tests on publication bias, Egger's test (FAT), p-uniform, the test of excess significance (TES), as well as the caliper test, were evaluated in a Monte Carlo simulation. Two different types of publication bias and its degree (0%, 50%, 100%) were simulated. The type of publication bias was defined either as file-drawer, meaning the repeated analysis of new datasets, or p-hacking, meaning the inclusion of covariates in order to obtain a significant result. In addition, the underlying effect (β = 0, 0.5, 1, 1.5), effect heterogeneity, the number of observations in the simulated primary studies (N = 100, 500), and the number of observations for the publication bias tests (K = 100, 1,000) were varied. Results:All tests evaluated were able to identify publication bias both in the file-drawer and p-hacking condition. The false positive rates were, with the exception of the 15%- and 20%-caliper test, unbiased. The FAT had the largest statistical power in the file-drawer conditions, whereas under p-hacking the TES was, except under effect heterogeneity, slightly better. The CTs were, however, inferior to the other tests under effect homogeneity and had a decent statistical power only in conditions with 1,000 primary studies. Discussion:The FAT is recommended as a test for publication bias in standard meta-analyses with no or only small effect heterogeneity. If two-sided publication bias is suspected as well as under p-hacking the TES is the first alternative to the FAT. The 5%-caliper test is recommended under conditions of effect heterogeneity and a large number of primary studies, which may be found if publication bias is examined in a discipline-wide setting when primary studies cover different research problems.

Dataset Information

Fiddle: a tool to combat publication bias by getting research out of the file drawer and into the scientific community.

Publications

fiddle: a tool to combat publication bias by getting research out of the file drawer and into the scientific community.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets