Inadequate use and regulation of interventions against publication bias decreases their effectiveness: a systematic review.
Ontology highlight
ABSTRACT: Objectives
To determine the effectiveness of interventions designed to prevent or reduce publication and related biases.Study design and setting
We searched multiple databases and performed manual searches using terms related to publication bias and known interventions against publication bias. We dually reviewed citations and assessed risk of bias. We synthesized results by intervention and outcomes measured and graded the quality of the evidence (QoE).Results
We located 38 eligible studies. The use of prospective trial registries (PTR) has increased since 2005 (seven studies, moderate QoE); however, positive outcome-reporting bias is prevalent (14 studies, low QoE), and information in nonmandatory fields is vague (10 studies, low QoE). Disclosure of financial conflict of interest (CoI) is inadequate (five studies, low QoE). Blinding peer reviewers may reduce geographical bias (two studies, very low QoE), and open-access publishing does not discriminate against authors from low-income countries (two studies, very low QoE).Conclusion
The use of PTR and CoI disclosures is increasing; however, the adequacy of their use requires improvement. The effect of open-access publication and blinding of peer reviewers on publication bias is unclear, as is the effect of other interventions such as electronic publication and authors' rights to publish their results.
SUBMITTER: Thaler K
PROVIDER: S-EPMC4459964 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundSystematic reviews and meta-analyses of pre-clinical studies, in vivo animal experiments in particular, can influence clinical care. Publication bias is one of the major threats of validity in systematic reviews and meta-analyses. Previous empirical studies suggested that systematic reviews and meta-analyses have become more prevalent until 2010 and found evidence for compromised methodological rigor with a trend towards improvement. We aim to comprehensively summarize and update the evidence base on systematic reviews and meta-analyses of animal studies, their methodological quality and assessment of publication bias in particular.Methods/designThe objectives of this systematic review are as follows: •To investigate the epidemiology of published systematic reviews of animal studies until present. •To examine methodological features of systematic reviews and meta-analyses of animal studies with special attention to the assessment of publication bias. •To investigate the influence of systematic reviews of animal studies on clinical research by examining citations of the systematic reviews by clinical studies. Eligible studies for this systematic review constitute systematic reviews and meta-analyses that summarize in vivo animal experiments with the purpose of reviewing animal evidence to inform human health. We will exclude genome-wide association studies and animal experiments with the main purpose to learn more about fundamental biology, physical functioning or behavior. In addition to the inclusion of systematic reviews and meta-analyses identified by other empirical studies, we will systematically search Ovid Medline, Embase, ToxNet, and ScienceDirect from 2009 to January 2013 for further eligible studies without language restrictions. Two reviewers working independently will assess titles, abstracts, and full texts for eligibility and extract relevant data from included studies. Data reporting will involve a descriptive summary of meta-analyses and systematic reviews.DiscussionResults are expected to be publicly available later in 2013 and may form the basis for recommendations to improve the quality of systematic reviews and meta-analyses of animal studies and their use with respect to clinical care.
| S-EPMC3651300 | biostudies-literature
Project description:The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention.We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
| S-EPMC2518111 | biostudies-literature
Project description:Background:Publication bias is a form of scientific misconduct. It threatens the validity of research results and the credibility of science. Although several tests on publication bias exist, no in-depth evaluations are available that examine which test performs best for different research settings. Methods:Four tests on publication bias, Egger's test (FAT), p-uniform, the test of excess significance (TES), as well as the caliper test, were evaluated in a Monte Carlo simulation. Two different types of publication bias and its degree (0%, 50%, 100%) were simulated. The type of publication bias was defined either as file-drawer, meaning the repeated analysis of new datasets, or p-hacking, meaning the inclusion of covariates in order to obtain a significant result. In addition, the underlying effect (β = 0, 0.5, 1, 1.5), effect heterogeneity, the number of observations in the simulated primary studies (N = 100, 500), and the number of observations for the publication bias tests (K = 100, 1,000) were varied. Results:All tests evaluated were able to identify publication bias both in the file-drawer and p-hacking condition. The false positive rates were, with the exception of the 15%- and 20%-caliper test, unbiased. The FAT had the largest statistical power in the file-drawer conditions, whereas under p-hacking the TES was, except under effect heterogeneity, slightly better. The CTs were, however, inferior to the other tests under effect homogeneity and had a decent statistical power only in conditions with 1,000 primary studies. Discussion:The FAT is recommended as a test for publication bias in standard meta-analyses with no or only small effect heterogeneity. If two-sided publication bias is suspected as well as under p-hacking the TES is the first alternative to the FAT. The 5%-caliper test is recommended under conditions of effect heterogeneity and a large number of primary studies, which may be found if publication bias is examined in a discipline-wide setting when primary studies cover different research problems.
| S-EPMC5712469 | biostudies-literature
Project description:BACKGROUND:The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. METHODOLOGY/PRINCIPAL FINDINGS:In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS:This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.
| S-EPMC3702538 | biostudies-literature
Project description:Publication bias is a serious problem in systematic reviews and meta-analyses, which can affect the validity and generalization of conclusions. Currently, approaches to dealing with publication bias can be distinguished into two classes: selection models and funnel-plot-based methods. Selection models use weight functions to adjust the overall effect size estimate and are usually employed as sensitivity analyses to assess the potential impact of publication bias. Funnel-plot-based methods include visual examination of a funnel plot, regression and rank tests, and the nonparametric trim and fill method. Although these approaches have been widely used in applications, measures for quantifying publication bias are seldom studied in the literature. Such measures can be used as a characteristic of a meta-analysis; also, they permit comparisons of publication biases between different meta-analyses. Egger's regression intercept may be considered as a candidate measure, but it lacks an intuitive interpretation. This article introduces a new measure, the skewness of the standardized deviates, to quantify publication bias. This measure describes the asymmetry of the collected studies' distribution. In addition, a new test for publication bias is derived based on the skewness. Large sample properties of the new measure are studied, and its performance is illustrated using simulations and three case studies.
| S-EPMC5953768 | biostudies-literature
Project description:IntroductionPositive results have a greater chance of being published and outcomes that are statistically significant have a greater chance of being fully reported. One consequence of research underreporting is that it may influence the sample of studies that is available for a meta-analysis. Smaller studies are often characterized by larger effects in published meta-analyses, which can be possibly explained by publication bias. We investigated the association between the statistical significance of the results and the probability of being included in recent meta-analyses.MethodsFor meta-analyses of clinical trials, we defined the relative risk as the ratio of the probability of including statistically significant results favoring the treatment to the probability of including other results. For meta-analyses of other studies, we defined the relative risk as the ratio of the probability of including biologically plausible statistically significant results to the probability of including other results. We applied a Bayesian selection model for meta-analyses that included at least 30 studies and were published in four major general medical journals (BMJ, JAMA, Lancet, and PLOS Medicine) between 2008 and 2012.ResultsWe identified 49 meta-analyses. The estimate of the relative risk was greater than one in 42 meta-analyses, greater than two in 16 meta-analyses, greater than three in eight meta-analyses, and greater than five in four meta-analyses. In 10 out of 28 meta-analyses of clinical trials, there was strong evidence that statistically significant results favoring the treatment were more likely to be included. In 4 out of 19 meta-analyses of observational studies, there was strong evidence that plausible statistically significant outcomes had a higher probability of being included.ConclusionsPublication bias was present in a substantial proportion of large meta-analyses that were recently published in four major medical journals.
| S-EPMC3868709 | biostudies-literature
Project description:BACKGROUND:Attentional bias modification (ABM) interventions have been developed to address addiction by reducing attentional bias for substance-related cues. This study provides a systematic review of the effectiveness of ABM interventions in decreasing symptoms of addictive behaviour, taking baseline levels of attentional bias and changes in attentional bias into account. METHODS:We included randomised and non-randomised studies that investigated the effectiveness of ABM interventions in heavy-using adults and treatment-seeking individuals with symptoms of substance use disorder to manipulate attentional bias and to reduce substance use-related symptoms. We searched for relevant English peer-reviewed articles without any restriction for the year of publication using PsycINFO, PubMed, and ISI Web in August 2016. Study quality was assessed regarding reporting, external validity, internal validity, and power of the study. RESULTS:Eighteen studies were included: nine studies reported on ABM intervention effects in alcohol use, six studies on nicotine use, and three studies on opiate use. The included studies differed with regard to type of ABM intervention (modified dot probe task n?=?14; Alcohol Attention Control Training Programme n?=?4), outcome measures, amount and length of provided sessions, and context (clinic versus laboratory versus home environment). The study quality mostly ranged from low average to high average (one study scored below the quality cut-off). Ten studies reported significant changes of symptoms of addictive behaviour, whereas eight studies found no effect of ABM interventions on symptoms. However, when restricted to multi-session ABM intervention studies, eight out of ten studies found effects on symptoms of addiction. Surprisingly, these effects on symptoms of addictive behaviour showed no straightforward relationship with baseline attentional bias and its change from baseline to post-test. CONCLUSIONS:Despite a number of negative findings and the diversity of studies, multi-session ABM interventions, especially in the case of alcohol and when the Alcohol Attention Control Training Programme was used, appear to have positive effects on symptoms of addictive behaviour. However, more rigorous well-powered future research in clinical samples is needed before firm conclusions regarding the effectiveness of ABM interventions can be drawn. SYSTEMATIC REVIEW REGISTRATION:Registration number PROSPERO: CRD42016046823.
| S-EPMC6186103 | biostudies-literature
Project description:ObjectivesTo evaluate which non-pharmaceutical interventions (NPIs) have been more and less effective in controlling the COVID-19 pandemic.MethodsWe performed a systematic review of published and unpublished empirical studies, either observational or interventional, analysing the comparative effectiveness of NPIs against the COVID-19 pandemic. We searched Embase/Medline and medRxiv to identify the relevant literature.ResultsWe identified 34 studies. During the first wave of the COVID-19 pandemic, school closing was the most effective NPI, followed by workplace closing, business and venue closing and public event bans. Public information campaigns and mask wearing requirements were also effective in controlling the pandemic while being less disruptive for the population than other NPIs. There was no evidence on the effectiveness of public transport closure, testing and contact tracing strategies and quarantining or isolation of individuals. Early implementation was associated with a higher effectiveness in reducing COVID-19 cases and deaths, while general stringency of the NPIs was not.ConclusionsIn this systematic review, we found that school closing, followed by workplace closing, business and venue closing and public event bans were the most effective NPIs in controlling the spread of COVID-19. An early response and a combination of specific social distancing measures are effective at reducing COVID-19 cases and deaths. Continuous monitoring of NPIs effectiveness is needed in order to adapt decision making.
| S-EPMC8214911 | biostudies-literature
Project description:The accuracy of a diagnostic test, which is often quantified by a pair of measures such as sensitivity and specificity, is critical for medical decision making. Separate studies of an investigational diagnostic test can be combined through meta-analysis; however, such an analysis can be threatened by publication bias. To the best of our knowledge, there is no existing method that accounts for publication bias in the meta-analysis of diagnostic tests involving bivariate outcomes. In this paper, we extend the Copas selection model from univariate outcomes to bivariate outcomes for the correction of publication bias when the probability of a study being published can depend on its sensitivity, specificity, and the associated standard errors. We develop an expectation-maximization algorithm for the maximum likelihood estimation under the proposed selection model. We investigate the finite sample performance of the proposed method through simulation studies and illustrate the method by assessing a meta-analysis of 17 published studies of a rapid diagnostic test for influenza.
| S-EPMC6443505 | biostudies-literature
Project description:BackgroundSince December 2019, the COVID-19 pandemic has changed the concept of medicine. This work aims to analyze the use of antibiotics in patients admitted to the hospital due to SARS-CoV-2 infection.MethodsThis work analyzes the use and effectiveness of antibiotics in hospitalized patients with COVID-19 based on data from the SEMI-COVID-19 registry, an initiative to generate knowledge about this disease using data from electronic medical records. Our primary endpoint was all-cause in-hospital mortality according to antibiotic use. The secondary endpoint was the effect of macrolides on mortality.ResultsOf 13,932 patients, antibiotics were used in 12,238. The overall death rate was 20.7% and higher among those taking antibiotics (87.8%). Higher mortality was observed with use of all antibiotics (OR 1.40, 95% CI 1.21-1.62; p < .001) except macrolides, which had a higher survival rate (OR 0.70, 95% CI 0.64-0.76; p < .001). The decision to start antibiotics was influenced by presence of increased inflammatory markers and any kind of infiltrate on an x-ray. Patients receiving antibiotics required respiratory support and were transferred to intensive care units more often.ConclusionsBacterial co-infection was uncommon among COVID-19 patients, yet use of antibiotics was high. There is insufficient evidence to support widespread use of empiric antibiotics in these patients. Most may not require empiric treatment and if they do, there is promising evidence regarding azithromycin as a potential COVID-19 treatment.
| S-EPMC8575150 | biostudies-literature