Dataset Information

SYRCLE's risk of bias tool for animal studies.

ABSTRACT: BACKGROUND: Systematic Reviews (SRs) of experimental animal studies are not yet common practice, but awareness of the merits of conducting such SRs is steadily increasing. As animal intervention studies differ from randomized clinical trials (RCT) in many aspects, the methodology for SRs of clinical trials needs to be adapted and optimized for animal intervention studies. The Cochrane Collaboration developed a Risk of Bias (RoB) tool to establish consistency and avoid discrepancies in assessing the methodological quality of RCTs. A similar initiative is warranted in the field of animal experimentation. METHODS: We provide an RoB tool for animal intervention studies (SYRCLE's RoB tool). This tool is based on the Cochrane RoB tool and has been adjusted for aspects of bias that play a specific role in animal intervention studies. To enhance transparency and applicability, we formulated signalling questions to facilitate judgment. RESULTS: The resulting RoB tool for animal studies contains 10 entries. These entries are related to selection bias, performance bias, detection bias, attrition bias, reporting bias and other biases. Half these items are in agreement with the items in the Cochrane RoB tool. Most of the variations between the two tools are due to differences in design between RCTs and animal studies. Shortcomings in, or unfamiliarity with, specific aspects of experimental design of animal studies compared to clinical studies also play a role. CONCLUSIONS: SYRCLE's RoB tool is an adapted version of the Cochrane RoB tool. Widespread adoption and implementation of this tool will facilitate and improve critical appraisal of evidence from animal studies. This may subsequently enhance the efficiency of translating animal research into clinical practice and increase awareness of the necessity of improving the methodological quality of animal studies.

SUBMITTER: Hooijmans CR

PROVIDER: S-EPMC4230647 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

SYRCLE's risk of bias tool for animal studies.

Hooijmans Carlijn R CR Rovers Maroeska M MM de Vries Rob B M RB Leenaars Marlies M Ritskes-Hoitinga Merel M Langendam Miranda W MW

BMC medical research methodology 20140326

<h4>Background</h4>Systematic Reviews (SRs) of experimental animal studies are not yet common practice, but awareness of the merits of conducting such SRs is steadily increasing. As animal intervention studies differ from randomized clinical trials (RCT) in many aspects, the methodology for SRs of clinical trials needs to be adapted and optimized for animal intervention studies. The Cochrane Collaboration developed a Risk of Bias (RoB) tool to establish consistency and avoid discrepancies in ass ...[more]

PMID: 24667063

Similar Datasets

Project description:BackgroundSystematic reviews of the effects of healthcare interventions frequently include non-randomized studies. These are subject to confounding and a range of other biases that are seldom considered in detail when synthesizing and interpreting the results. Our aims were to assess the reliability and usability of a new Cochrane risk of bias (RoB) tool for non-randomized studies of interventions and to determine whether restricting analysis to studies with low or moderate RoB made a material difference to the results of the reviews.Methods and findingsWe selected two systematic reviews of population-based, controlled non-randomized studies of the relationship between the use of thiazolidinediones (TZDs) and cyclooxygenase-2 (COX-2) inhibitors and major cardiovascular events. Two epidemiologists applied the Cochrane RoB tool and made assessments across the seven specified domains of bias for each of 37 component studies. Inter-rater agreement was measured using the weighted Kappa statistic. We grouped studies according to overall RoB and performed statistical pooling for (a) all studies and (b) only studies with low or moderate RoB. Kappa scores across the seven bias domains ranged from 0.50 to 1.0. In the COX-2 inhibitor review, two studies had low overall RoB, 14 had moderate RoB, and five had serious RoB. In the TZD review, six studies had low RoB, four had moderate RoB, four had serious RoB, and two had critical RoB. The pooled odds ratios for myocardial infarction, heart failure, and death for rosiglitazone versus pioglitazone remained significantly elevated when analyses were confined to studies with low or moderate RoB. However, the estimate for myocardial infarction declined from 1.14 (95% CI 1.07-1.24) to 1.06 (95% CI 0.99-1.13) when analysis was confined to studies with low RoB. Estimates of pooled relative risks of cardiovascular events with COX-2 inhibitors compared with no nonsteroidal anti-inflammatory drug changed little when analyses were confined to studies with low or moderate RoB. The exception was a rise in the relative risk associated with ibuprofen from 1.07 (95% CI 0.97-1.18) to 1.14 (95% CI 1.03-1.26). The main limitation of our study was testing the instrument on a narrow range of pharmacoepidemiological studies; we cannot assume our findings extend to a broader range of interventions and settings.ConclusionsThe Cochrane RoB tool highlighted a wide range of risks of bias in studies included in two widely cited reviews and had the potential to change the conclusions of the reviews. Systematic reviews that incorporate non-randomized studies of medical interventions should include a detailed assessment of RoB for each included study.

Project description:IntroductionThere is little evidence regarding the influence of conflicts of interest on preclinical research. This study examines whether industry sponsorship is associated with increased risks of bias and/or effect sizes of outcomes in published preclinical thiazolidinedione (TZD) studies.MethodsWe identified preclinical TZD studies published between January 1, 1965 and November 14, 2012. Coders independently extracted information on study design criteria aimed at reducing bias, results for all relevant outcomes, sponsorship source, and investigator financial ties from the 112 studies meeting the inclusion criteria. The average standardized mean difference (SMD) across studies was calculated for plasma glucose (efficacy outcome) and weight gain (harm outcome). In subgroup analyses, TZD outcomes were assessed by sponsorship source and risk of bias criteria.ResultsSeven studies were funded by industry alone, 17 studies funded by both industry and non-industry, 49 studies funded by non-industry alone, and 39 studies had no disclosures. None of the studies used sample size calculations, intention-to-treat analyses, blinding of investigators, or concealment of allocation. Most studies reported favorable results (88 of 112) and conclusions (95 of 112) supporting TZD use. Efficacy estimates were significantly larger in 6 studies sponsored by industry alone (-3.41; 95% CI -5.21, -1.53; I2 = 93%) versus 42 studies sponsored by non-industry sources (-0.97; 95% CI -1.37, -0.56; I2 = 81%) (p value = 0.01). Harms estimates were significantly larger in 4 studies sponsored by industry alone (5.00; 95% CI 1.22, 8.77; I2 = 93%) versus 38 studies sponsored by non-industry sources (0.30; 95% CI -0.08, 0.68; I2 = 79%) (p value = 0.02). TZD efficacy and harms did not differ by disclosure of financial COIs or risks of bias.ConclusionsIndustry-sponsored TZD animal studies have exaggerated efficacy and harms outcomes compared to studies funded by non-industry sources. There was poor reporting of COIs.

Project description:BackgroundRandomized controlled trials on menopausal hormone therapy in humans have not confirmed the benefit of estrogens on cardiovascular disease found in animal studies. Flawed methodology or publication bias in animal studies may explain the dicrepancy.ObjectivesThe aim of this study was to investigate whether publication of the randomized controlled trials Heart and Estrogen/Progestin Replacement Study and Women's Health Initiative influenced study authors' assessment of research findings (confirmation bias) as well as to investigate publication bias and small-study effects in animal studies of estrogen effects on atherosclerosis.MethodsThe data source for this study was PubMed from inception to 2018. We selected animal studies with cardiovascular outcomes comparing 17-β-estradiol, its natural metabolites, or conjugated equine estrogen with control. Qualitative data were extracted on authors' conclusions about estrogen effects on cardiovascular disease, as well as quantitative data for atherosclerosis outcomes. Fixed- and random-effects meta-analyses were used. Publication bias/small-study effects were assessed using funnel plots and Egger's regression. Trim-and-fill plots and extrapolation from Egger's regression were used to adjust for publication bias. The main outcomes and measures were the primary study authors' interpretation of their own results, and estrogen effects on cardiovascular disease in general before and after publication of the Women's health Initiative study (2003). The effects of estrogens on atherosclerosis were measured as standardized mean difference between intervention and control.ResultsOf 1925 studies retrieved, 360 were eligible for analyses. Study-specific statements concluded that estrogens were protective against cardiovascular disease in 75% of studies before 2003 and 78% after, but the percentage of general statements about estrogens being cardioprotective changed from 70% to 40%. Meta-analyses showed less atherosclerosis in estrogen-treated animals. Extremely skewed funnel plots and P < .01 in Egger's regression suggested publication bias and/or exaggerated effects in small studies, which was more pronounced after 2002. There was substantial heterogeneity of effects (I 2 = 68%-86%) overall and in all subgroups except cynomolgus monkeys (I 2 = 9%), the only animal subgroup without clear bias. Adjusting for publication bias, overall estimates of estrogen effects on atherosclerosis were close to null effect.ConclusionsWe found substantial evidence of publication bias but not of confirmation bias. Publication bias and flawed small studies may partly explain why findings differ between animal studies and clinical trials on the effect of estrogens on cardiovascular disease.

Dataset Information

SYRCLE's risk of bias tool for animal studies.

Publications

SYRCLE's risk of bias tool for animal studies.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets