Dataset Information

Publication bias in laboratory animal research: a survey on magnitude, drivers, consequences and potential solutions.

ABSTRACT:

Context

Publication bias jeopardizes evidence-based medicine, mainly through biased literature syntheses. Publication bias may also affect laboratory animal research, but evidence is scarce.

Objectives

To assess the opinion of laboratory animal researchers on the magnitude, drivers, consequences and potential solutions for publication bias. And to explore the impact of size of the animals used, seniority of the respondent, working in a for-profit organization and type of research (fundamental, pre-clinical, or both) on those opinions.

Design

Internet-based survey.

Setting

All animal laboratories in The Netherlands.

Participants

Laboratory animal researchers.

Main outcome measure(s)

Median (interquartile ranges) strengths of beliefs on 5 and 10-point scales (1: totally unimportant to 5 or 10: extremely important).

Results

Overall, 454 researchers participated. They considered publication bias a problem in animal research (7 (5 to 8)) and thought that about 50% (32-70) of animal experiments are published. Employees (n = 21) of for-profit organizations estimated that 10% (5 to 50) are published. Lack of statistical significance (4 (4 to 5)), technical problems (4 (3 to 4)), supervisors (4 (3 to 5)) and peer reviewers (4 (3 to 5)) were considered important reasons for non-publication (all on 5-point scales). Respondents thought that mandatory publication of study protocols and results, or the reasons why no results were obtained, may increase scientific progress but expected increased bureaucracy. These opinions did not depend on size of the animal used, seniority of the respondent or type of research.

Conclusions

Non-publication of "negative" results appears to be prevalent in laboratory animal research. If statistical significance is indeed a main driver of publication, the collective literature on animal experimentation will be biased. This will impede the performance of valid literature syntheses. Effective, yet efficient systems should be explored to counteract selective reporting of laboratory animal research.

SUBMITTER: ter Riet G

PROVIDER: S-EPMC3434185 | biostudies-literature | 2012

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Publication bias in laboratory animal research: a survey on magnitude, drivers, consequences and potential solutions.

ter Riet Gerben G Korevaar Daniel A DA Leenaars Marlies M Sterk Peter J PJ Van Noorden Cornelis J F CJ Bouter Lex M LM Lutter René R Elferink Ronald P Oude RP Hooft Lotty L

PloS one 20120905 9

<h4>Context</h4>Publication bias jeopardizes evidence-based medicine, mainly through biased literature syntheses. Publication bias may also affect laboratory animal research, but evidence is scarce.<h4>Objectives</h4>To assess the opinion of laboratory animal researchers on the magnitude, drivers, consequences and potential solutions for publication bias. And to explore the impact of size of the animals used, seniority of the respondent, working in a for-profit organization and type of research ...[more]

PMID: 22957028

Similar Datasets

Project description:BackgroundSystematic reviews and meta-analyses of pre-clinical studies, in vivo animal experiments in particular, can influence clinical care. Publication bias is one of the major threats of validity in systematic reviews and meta-analyses. Previous empirical studies suggested that systematic reviews and meta-analyses have become more prevalent until 2010 and found evidence for compromised methodological rigor with a trend towards improvement. We aim to comprehensively summarize and update the evidence base on systematic reviews and meta-analyses of animal studies, their methodological quality and assessment of publication bias in particular.Methods/designThe objectives of this systematic review are as follows: •To investigate the epidemiology of published systematic reviews of animal studies until present. •To examine methodological features of systematic reviews and meta-analyses of animal studies with special attention to the assessment of publication bias. •To investigate the influence of systematic reviews of animal studies on clinical research by examining citations of the systematic reviews by clinical studies. Eligible studies for this systematic review constitute systematic reviews and meta-analyses that summarize in vivo animal experiments with the purpose of reviewing animal evidence to inform human health. We will exclude genome-wide association studies and animal experiments with the main purpose to learn more about fundamental biology, physical functioning or behavior. In addition to the inclusion of systematic reviews and meta-analyses identified by other empirical studies, we will systematically search Ovid Medline, Embase, ToxNet, and ScienceDirect from 2009 to January 2013 for further eligible studies without language restrictions. Two reviewers working independently will assess titles, abstracts, and full texts for eligibility and extract relevant data from included studies. Data reporting will involve a descriptive summary of meta-analyses and systematic reviews.DiscussionResults are expected to be publicly available later in 2013 and may form the basis for recommendations to improve the quality of systematic reviews and meta-analyses of animal studies and their use with respect to clinical care.

Project description:To determine a direct measure of publication bias by determining subsequent full-paper publication (P) of studies reported in animal research abstracts presented at an international conference (A).We selected 100 random (using a random-number generator) A from the 2008 Society of Critical Care Medicine Conference. Using a data collection form and study manual, we recorded methodology and result variables from A. We searched PubMed and EMBASE to June 2015, and DOAJ and Google Scholar to May 2017 to screen for subsequent P. Methodology and result variables were recorded from P to determine changes in reporting from A. Predictors of P were examined using Fisher's Exact Test.62% (95% CI 52-71%) of studies described in A were subsequently P after a median 19 [IQR 9-33.3] months from conference presentation. Reporting of studies in A was of low quality: randomized 27% (the method of randomization and allocation concealment not described), blinded 0%, sample-size calculation stated 0%, specifying the primary outcome 26%, numbers given with denominators 6%, and stating number of animals used 47%. Only being an orally presented (vs. poster presented) A (14/16 vs. 48/84, p = 0.025) predicted P. Reporting of studies in P was of poor quality: randomized 39% (the method of randomization and allocation concealment not described), likely blinded 6%, primary outcome specified 5%, sample size calculation stated 0%, numbers given with denominators 34%, and number of animals used stated 56%. Changes in reporting from A to P occurred: from non-randomized to randomized 19%, from non-blinded to blinded 6%, from negative to positive outcomes 8%, from having to not having a stated primary outcome 16%, and from non-statistically to statistically significant findings 37%. Post-hoc, using publication data, P was predicted by having positive outcomes (published 62/62, unpublished 33/38; p = 0.003), or statistically significant results (published 58/62, unpublished 20/38; p < 0.001).Only 62% (95% CI 52-71%) of animal research A are subsequently P; this was predicted by oral presentation of the A, finally having positive outcomes, and finally having statistically significant results. Publication bias is prevalent in critical care animal research.

Project description:If a laboratory animal survives an experiment without lasting compromised welfare, its future must be negotiated. Rehoming may be a consideration. This paper reports on research findings that provide an indication of the uptake of animal rehoming by UK facilities and the associated moral, ethical, practical and regulatory considerations that inform decisions to rehome or not. This research addresses a widely acknowledged gap in the literature to understand both the numbers, and types of animals rehomed from UK research facilities, as well as the main motivations for engaging in the practice, and the barriers for those facilities not currently rehoming. From the ~160 UK research facilities in the UK, 41 facilities completed the questionnaire, giving a response rate of approximately 25%. Results suggest rehoming occurs routinely, yet the numbers are small; just 2322 animals are known to have been rehomed between 2015-2017. At least 1 in 10 facilities are rehoming. There exists a clear preference for the rehoming of some species (mainly cats, dogs and horses) over others (rodents, agricultural animals and primates). Indeed, although 94.15% of species kept in laboratories are rodents, they make up under a fifth (19.14%) of all animals known to be rehomed between 2015-2017. The primary motivation for rehoming is to boost staff morale and promote a positive ethical profile for the facility. Barriers include concern for the animal's welfare following rehoming, high scientific demand for animals that leaves few to be rehomed, and, finally, certain animals (mainly those genetically modified) are simply unsuited to rehoming. The findings of this research will support facilities choosing to rehome, as well as those that are not currently engaging in the practice. By promoting the practice, the benefits to rehoming in terms of improving laboratory animal's quality of life, helping facility staff to overcome the moral stress of killing, and addressing public concern regarding the fate of laboratory animals, can be attained. It is only once an understanding of rehoming from the perspective of UK research facilities has been ascertained, that appropriate policy and support can be provided.

Project description:ObjectivesPublication bias, non-publication, and selective reporting of animal studies limit progress toward the 3Rs (Replacement, Reduction, and Refinement) that guide ethical animal testing, waste public resources, and result in redundant research, which collectively undermine the public's trust in scientific reliability. In this study, we aimed to 1) validate findings from a previous follow-up study by our team that examined the publication rates of animal studies from protocol to publication and 2) identify incentives for improving publication rates in animal research.MethodsThe researchers responsible for the animal proposals (n = 210) from our previous study were contacted as participants for a Web-based survey between October 2019 and April 2020. Question types varied between free text questions, answer options based on a 5-point Likert scale and closed yes/no questions.ResultsIn total, 78 researchers responsible for 101 of 210 animal study proposals participated, yielding a response rate of 48.1%. Results showed that the publication rate increased from 67% in our follow-up study to 70%. According to a 5-point Likert scale (from 1 = "not relevant" to 5 = "extremely relevant"), the most widely accepted suggestions for increasing publication rates were "Publication costs for open access journals are fully covered by funders or universities" (mean 4.02, SD 1.01), "Performance-based allocation of intramural funds for results reporting of animal research not supporting the initial hypothesis (including preprints and repositories)" (mean 3.37, SD 1.05), and "Researchers receive more information from scientific journals that also publish non-significant results" (mean 3.30, SD 1.02).ConclusionWhile the extent of publication and publication practices have been thoroughly investigated for clinical trials, less data is available for animal research to date. Therefore, the study contributes in complementing the picture of publication practice in animal research. Suggestions from our survey may help improve the publication rates of animal studies.

Project description:Scientists who perform major survival surgery on laboratory animals face a dual welfare and methodological challenge: how to choose surgical anesthetics and post-operative analgesics that will best control animal suffering, knowing that both pain and the drugs that manage pain can all affect research outcomes. Scientists who publish full descriptions of animal procedures allow critical and systematic reviews of data, demonstrate their adherence to animal welfare norms, and guide other scientists on how to conduct their own studies in the field. We investigated what information on animal pain management a reasonably diligent scientist might find in planning for a successful experiment. To explore how scientists in a range of fields describe their management of this ethical and methodological concern, we scored 400 scientific articles that included major animal survival surgeries as part of their experimental methods, for the completeness of information on anesthesia and analgesia. The 400 articles (250 accepted for publication pre-2011, and 150 in 2014-15, along with 174 articles they reference) included thoracotomies, craniotomies, gonadectomies, organ transplants, peripheral nerve injuries, spinal laminectomies and orthopedic procedures in dogs, primates, swine, mice, rats and other rodents. We scored articles for Publication Completeness (PC), which was any mention of use of anesthetics or analgesics; Analgesia Use (AU) which was any use of post-surgical analgesics, and Analgesia Completeness (a composite score comprising intra-operative analgesia, extended post-surgical analgesia, and use of multimodal analgesia). 338 of 400 articles were PC. 98 of these 338 were AU, with some mention of analgesia, while 240 of 338 mentioned anesthesia only but not post-surgical analgesia. Journals' caliber, as measured by their 2013 Impact Factor, had no effect on PC or AU. We found no effect of whether a journal instructs authors to consult the ARRIVE publishing guidelines published in 2010 on PC or AC for the 150 mouse and rat articles in our 2014-15 dataset. None of the 302 articles that were silent about analgesic use included an explicit statement that analgesics were withheld, or a discussion of how pain management or untreated pain might affect results. We conclude that current scientific literature cannot be trusted to present full detail on use of animal anesthetics and analgesics. We report that publication guidelines focus more on other potential sources of bias in experimental results, under-appreciate the potential for pain and pain drugs to skew data, and thus mostly treat pain management as solely an animal welfare concern, in the jurisdiction of animal care and use committees. At the same time, animal welfare regulations do not include guidance on publishing animal data, even though publication is an integral part of the cycle of research and can affect the welfare of animals in studies building on published work, leaving it to journals and authors to voluntarily decide what details of animal use to publish. We suggest that journals, scientists and animal welfare regulators should revise current guidelines and regulations, on treatment of pain and on transparent reporting of treatment of pain, to improve this dual welfare and data-quality deficiency.

Project description:Background:Publication bias is a form of scientific misconduct. It threatens the validity of research results and the credibility of science. Although several tests on publication bias exist, no in-depth evaluations are available that examine which test performs best for different research settings. Methods:Four tests on publication bias, Egger's test (FAT), p-uniform, the test of excess significance (TES), as well as the caliper test, were evaluated in a Monte Carlo simulation. Two different types of publication bias and its degree (0%, 50%, 100%) were simulated. The type of publication bias was defined either as file-drawer, meaning the repeated analysis of new datasets, or p-hacking, meaning the inclusion of covariates in order to obtain a significant result. In addition, the underlying effect (β = 0, 0.5, 1, 1.5), effect heterogeneity, the number of observations in the simulated primary studies (N = 100, 500), and the number of observations for the publication bias tests (K = 100, 1,000) were varied. Results:All tests evaluated were able to identify publication bias both in the file-drawer and p-hacking condition. The false positive rates were, with the exception of the 15%- and 20%-caliper test, unbiased. The FAT had the largest statistical power in the file-drawer conditions, whereas under p-hacking the TES was, except under effect heterogeneity, slightly better. The CTs were, however, inferior to the other tests under effect homogeneity and had a decent statistical power only in conditions with 1,000 primary studies. Discussion:The FAT is recommended as a test for publication bias in standard meta-analyses with no or only small effect heterogeneity. If two-sided publication bias is suspected as well as under p-hacking the TES is the first alternative to the FAT. The 5%-caliper test is recommended under conditions of effect heterogeneity and a large number of primary studies, which may be found if publication bias is examined in a discipline-wide setting when primary studies cover different research problems.

Dataset Information

Publication bias in laboratory animal research: a survey on magnitude, drivers, consequences and potential solutions.

Context

Objectives

Design

Setting

Participants

Main outcome measure(s)

Results

Conclusions

Publications

Publication bias in laboratory animal research: a survey on magnitude, drivers, consequences and potential solutions.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure