Dataset Information

The judgement of biases included in the category "other bias" in Cochrane systematic reviews of interventions: a systematic survey.

ABSTRACT:

Background

Clinical decisions are made based on Cochrane reviews, but the implementation of results of evidence syntheses such as Cochrane reviews is problematic if the evidence is not prepared consistently. All systematic reviews should assess the risk of bias (RoB) in included studies, and in Cochrane reviews, this is done by using Cochrane RoB tool. However, the tool is not necessarily applied according to the instructions. In this study, we aimed to determine the types of bias and their corresponding judgements noted in the 'other bias' domain of Cochrane RoB tool.

Methods

We analyzed Cochrane reviews that included randomized controlled trials (RCTs) and extracted data regarding 'other bias' from the RoB table and accompanying support for the judgment. We categorized different types of other bias.

Results

We analyzed 768 Cochrane reviews that included 11,369 RCTs. There were 602 (78%) Cochrane reviews that had 'other bias' domain in the RoB tool, and they included a total of 7811 RCTs. In the RoB table of 337 Cochrane reviews for at least one of the included trials it was indicated that no other bias was found and supporting explanations were inconsistently judged as low, unclear or high RoB. In the 524 Cochrane reviews that described various sources of other bias, there were 5762 individual types of explanations which we categorized into 31 groups. The judgments of the same supporting explanations were highly inconsistent. We found numerous other inconsistencies in reporting of sources of other bias in Cochrane reviews.

Conclusion

Cochrane authors mention a wide range of sources of other bias in the RoB tool and they inconsistently judge the same supporting explanations. Inconsistency in appraising risk of other bias hinders reliability and comparability of Cochrane systematic reviews. Discrepant and erroneous judgments of bias in evidence synthesis may hinder implementation of evidence in routine clinical practice and reduce confidence in otherwise trustworthy sources of information. These results can help authors of Cochrane and non-Cochrane reviews to gain insight into various sources of other bias that can be found in trials, and also to help them avoid mistakes that were recognized in published Cochrane reviews.

SUBMITTER: Babic A

PROVIDER: S-EPMC6458756 | biostudies-literature | 2019 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionThe validity of systematic reviews and meta-analysis depends on methodological quality and unbiased dissemination of trials. Our objective is to evaluate the association of estimates of treatment effects with different bias-related study characteristics in meta-analyses of interventions used for treating pain in osteoarthritis (OA). From the findings, we hope to consolidate guidance on interpreting OA trials in systematic reviews based on empirical evidence from Cochrane reviews.Methods and analysisOnly systematic reviews that compare experimental interventions with sham, placebo or no intervention control will be considered eligible. Bias will be assessed with the risk of bias tool, used according to the Cochrane Collaboration's recommendations. Furthermore, center status, trial size and funding will be assessed. The primary outcome (pain) will be abstracted from the first appearing forest plot for overall pain in the Cochrane review. Treatment effect sizes will be expressed as standardised mean differences (SMDs), where the difference in mean values available from the forest plots is divided by the pooled SD. To empirically assess the risk of bias in treatment benefits, we will perform stratified analyses of the trials from the included meta-analyses and assess the interaction between trial characteristics and treatment effect. A relevant study-level covariate is defined as one that decreases the between-study variance (τ(2), estimated as Tau-squared) as a consequence of inclusion in the mixed effects statistical model.Ethics and disseminationMeta-analyses and randomised controlled trials provide the most reliable basis for treatment of patients with OA, but the actual impact of bias is unclear. This study will systematically examine the methodological quality in OA Cochrane reviews and explore the effect estimates behind possible bias. Since our study does not collect primary data, no formal ethical assessment and informed consent are required.Trial registration numberPROSPERO (CRD42013006924).

Project description:BackgroundIn contemporary medical research, randomised controlled trials are seen as the gold standard for establishing treatment effects where it is ethical and practical to conduct them. In palliative care such trials are often impractical, unethical, or extremely difficult, with multiple methodological problems. We review the utility of Cochrane reviews in informing palliative care practice.MethodsPublished reviews in palliative care registered with the Cochrane Pain, Palliative and Supportive Care Group as of December 2007 were obtained from the Cochrane Database of Systematic Reviews, issue 1, 2008. We reviewed the quality and quantity of primary studies available for each review, assessed the quality of the review process, and judged the strength of the evidence presented. There was no prior intention to perform any statistical analyses.Results25 published systematic reviews were identified. Numbers of included trials ranged from none to 54. Within each review, included trials were heterogeneous with respect to patients, interventions, and outcomes, and the number of patients contributing to any single analysis was generally much lower than the total included in the review. A variety of tools were used to assess trial quality; seven reviews did not use this information to exclude low quality studies, weight analyses, or perform sensitivity analysis for effect of low quality. Authors indicated that there were frequently major problems with the primary studies, individually or in aggregate. Our judgment was that the reviewing process was generally good in these reviews, and that conclusions were limited by the number, size, quality and validity of the primary studies.We judged the evidence about 23 of the 25 interventions to be weak. Two reviews had stronger evidence, but with limitations due to methodological heterogeneity or definition of outcomes. No review provided strong evidence of no effect.ConclusionCochrane reviews in palliative care are well performed, but fail to provide good evidence for clinical practice because the primary studies are few in number, small, clinically heterogeneous, and of poor quality and external validity. They are useful in highlighting the weakness of the evidence base and problems in performing trials in palliative care.

Project description:BACKGROUND:Despite the growing reputation and subject coverage of the Cochrane Database of Systematic Reviews, many systematic reviews continue to be published solely in paper-based health care journals. This study was designed to determine why authors choose to publish their systematic reviews outside of the Cochrane Collaboration and if they might be interested in converting their reviews to Cochrane format for publication in the Cochrane Database of Systematic Reviews. METHODS:Cross-sectional survey of Australian primary authors of systematic reviews not published on the Cochrane Database of Systematic Reviews identified from the Database of Abstracts of Reviews of Effectiveness. RESULTS:We identified 88 systematic reviews from the Database of Abstracts of Reviews of Effectiveness with an Australian as the primary author. We surveyed 52 authors for whom valid contact information was available. The response rate was 88 per cent (46/52). Ten authors replied without completing the survey, leaving 36 valid surveys for analysis. The most frequently cited reasons for not undertaking a Cochrane review were: lack of time (78%), the need to undergo specific Cochrane training (46%), unwillingness to update reviews (36%), difficulties with the Cochrane process (26%) and the review topic already registered with the Cochrane Collaboration (21%). (Percentages based on completed responses to individual questions.) Nearly half the respondents would consider converting their review to Cochrane format. Dedicated time emerged as the most important factor in facilitating the potential conversion process. Other factors included navigating the Cochrane system, assistance with updating and financial support. Eighty-six per cent were willing to have their review converted to Cochrane format by another author. CONCLUSION:Time required to complete a Cochrane review and the need for specific training are the primary reasons why some authors publish systematic reviews outside of the Cochrane Collaboration. Encouragingly, almost half of the authors would consider converting their review to Cochrane format. Based on the current number of reviews in the Database of Abstracts of Reviews of Effectiveness, this could result in more than 700 additional Cochrane reviews. Ways of supporting these authors and how to provide dedicated time to convert systematic reviews needs further consideration.

Project description:BackgroundSeveral factors are associated with an increased risk of preterm birth (PTB); therefore, various interventions might have the potential to influence it. Due to the large number of interventions that address PTB, the objective of this overview is to summarise evidence from Cochrane reviews regarding the effects and safety of these different interventions.MethodsWe conducted a systematic literature search in the Cochrane Database of Systematic Reviews. Included reviews should be based on randomised controlled trials comparing antenatal non-pharmacological and pharmacological interventions that directly or indirectly address PTB with placebo/no treatment or routine care in pregnant women at less than 37 completed weeks of gestation without signs of threatened preterm labour. We considered PTB at less than 37 completed weeks of gestation as the primary outcome.ResultsWe included 56 Cochrane systematic reviews. Three interventions increased PTB risk significantly. Twelve interventions led to a statistically significant lower incidence of PTBs. However, this reduction was mostly observed in defined at-risk subgroups of pregnant women. The remaining antenatal interventions failed to prove a significant effect on PTB?<?37 weeks, but some of them at least showed a positive effect in secondary outcomes (e.g., reduction in early PTBs). As an unintended result of this review, we identified 28 additional Cochrane reviews which intended to report on PTB?<?37 weeks, but were not able to find any RCTs reporting appropriate data.ConclusionsThe possible effects of a diverse range of interventions on PTB have been evaluated in Cochrane systematic reviews. Few interventions have been demonstrated to be effective and a small number have been found to be harmful. For around half of the interventions evaluated, the Cochrane review concluded that there was insufficient evidence to provide sound recommendations for clinical practice. No RCT evidence is available for a number of potentially relevant interventions.

Project description:OBJECTIVES:Assess the frequency and reasons for disagreements in risk of bias assessments for randomised controlled trials (RCTs) included in more than one Cochrane review. DESIGN:Research on research study, using cross-sectional design. DATA SOURCES:2796 Cochrane reviews published between March 2011 and September 2014. DATA SELECTION:RCTs included in more than one review. DATA EXTRACTION:Risk of bias assessment and support for judgement for five key risk of bias items. DATA SYNTHESIS:For each item, we compared risk of bias assessment made in each review and calculated proportion of agreement. Two reviewers independently analysed 50% of all disagreements by comparing support for each judgement with information from study report to evaluate whether disagreements were related to a difference in information (eg, contact the study author) or a difference in interpretation (same support for judgement but different interpretation). They also identified main reasons for different interpretation. RESULTS:1604 RCTs were included in more than one review. Proportion of agreement ranged from 57% (770/1348 trials) for incomplete outcome data to 81% for random sequence generation (1193/1466). Most common source of disagreement was difference in interpretation of the same information, ranging from 65% (88/136) for random sequence generation to 90% (56/62) for blinding of participants and personnel. Access to different information explained 32/136 (24%) disagreements for random sequence generation and 38/205 (19%) for allocation concealment. Disagreements related to difference in interpretation were frequently related to incomplete or unclear reporting in the study report (83% of disagreements related to different interpretation for random sequence generation). CONCLUSIONS:Risk of bias judgements of RCTs included in more than one Cochrane review differed substantially. Most disagreements were related to a difference in interpretation of an incomplete or unclear description in the study report. A clearer guidance on common causes of incomplete information may improve agreement.

Project description:Eating disorders are psychiatric conditions originated from and perpetuated by individual, family and sociocultural factors. The psychosocial approach to treatment and prevention of relapse is crucial. To present an overview of the scientific evidence on effectiveness of psychosocial interventions in treatment of eating disorders. All systematic reviews published by the Cochrane Database of Systematic Reviews - Cochrane Library on the topic were included. Afterwards, as from the least recent date of these reviews (2001), an additional search was conducted at PubMed with sensitive search strategy and with the same keywords used. A total of 101 primary studies and 30 systematic reviews (5 Cochrane systematic reviews), meta-analysis, guidelines or narrative reviews of literature were included. The main outcomes were: symptomatic remission, body image, cognitive distortion, psychiatric comorbidity, psychosocial functioning and patient satisfaction. The cognitive behavioral approach was the most effective treatment, especially for bulimia nervosa, binge eating disorder and the night eating syndrome. For anorexia nervosa, the family approach showed greater effectiveness. Other effective approaches were interpersonal psychotherapy, dialectic behavioral therapy, support therapy and self-help manuals. Moreover, there was an increasing number of preventive and promotional approaches that addressed individual, family and social risk factors, being promising for the development of positive self-image and self-efficacy. Further studies are required to evaluate the impact of multidisciplinary approaches on all eating disorders, as well as the cost-effectiveness of some effective modalities, such as the cognitive behavioral therapy. RESUMO Transtornos alimentares são doenças psiquiátricas originadas de e perpetuadas por fatores individuais, familiares e socioculturais. A abordagem psicossocial é essencial para o tratamento e a prevenção de recaídas. Apresentar uma visão geral das evidências científicas sobre a efetividade das intervenções psicossociais no tratamento de transtornos alimentares. Foram incluídas todas as revisões sistemáticas publicadas no Banco de Dados de Revisões Sistemáticas da Cochrane Library. Posteriormente, a partir da data menos recente destas revisões (2001), realizou-se uma busca adicional no PubMed, com estratégia de busca sensibilizada e com os mesmos descritores utilizados antes. No total, foram incluídos 101 estudos primários e 30 revisões sistemáticas (5 revisões sistemáticas da Cochrane), metanálises, diretrizes ou revisões narrativas da literatura. Os principais desfechos foram remissão de sintomas, imagem corporal, distorção cognitiva, comorbidade psiquiátrica, funcionamento psicossocial e satisfação do paciente. A abordagem cognitivo-comportamental foi o tratamento mais efetivo, principalmente para bulimia nervosa, transtorno da compulsão alimentar periódica e síndrome do comer noturno. Para anorexia nervosa, a abordagem familiar demonstrou maior efetividade. Outras abordagens efetivas foram psicoterapia interpessoal, terapia comportamental dialética, terapia de apoio e manuais de autoajuda. Além disso, houve um número crescente de abordagens preventivas e promocionais que contemplaram fatores de risco individuais, familiares e sociais, sendo promissoras para o desenvolvimento da autoimagem positiva e autoeficácia. São necessários mais estudos que avaliem o impacto de abordagens multidisciplinares em todos transtornos alimentares, além da relação custo-efetividade de algumas modalidades efetivas, como a terapia cognitivo-comportamental.