Dataset Information

Rating of Included Trials on the Efficacy-Effectiveness Spectrum: development of a new tool for systematic reviews.

ABSTRACT: BACKGROUND AND OBJECTIVE:Randomized trials may be designed to provide evidence more strongly related to efficacy or effectiveness of an intervention. When systematic reviews are used to inform clinical or policy decisions, it is important to know the efficacy-effectiveness nature of the included trials. The objective of this study was to develop a tool to characterize randomized trials included in a systematic review on an efficacy-effectiveness continuum. METHODS:We extracted rating domains and descriptors from existing tools and used a modified Delphi procedure to condense the domains and develop a new tool. The feasibility and interrater reliability of the tool was tested on trials from four systematic reviews. RESULTS:The Rating of Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool rates clinical trials on a five-point Likert scale in four domains: (1) participant characteristics, (2) trial setting, (3) flexibility of interventions, and (4) clinical relevance of interventions. When RITES was piloted on trials from three reviews by unaffiliated raters, ratings were variable (intraclass correlation coefficient [ICC] 0.25-0.66 for the four domains); but, when RITES was used on one review by the review authors with expertise on the topic, the ratings were consistent (ICCs > 0.80. CONCLUSION:RITES may help to characterize the efficacy-effectiveness nature of trials included in systematic reviews.

SUBMITTER: Wieland LS

PROVIDER: S-EPMC5441969 | biostudies-literature | 2017 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Rating of Included Trials on the Efficacy-Effectiveness Spectrum: development of a new tool for systematic reviews.

Wieland L Susan LS Berman Brian M BM Altman Douglas G DG Barth Jürgen J Bouter Lex M LM D'Adamo Christopher R CR Linde Klaus K Moher David D Mullins C Daniel CD Treweek Shaun S Tunis Sean S van der Windt Danielle A DA Zwarenstein Merrick M Witt Claudia C

Journal of clinical epidemiology 20170207

<h4>Background and objective</h4>Randomized trials may be designed to provide evidence more strongly related to efficacy or effectiveness of an intervention. When systematic reviews are used to inform clinical or policy decisions, it is important to know the efficacy-effectiveness nature of the included trials. The objective of this study was to develop a tool to characterize randomized trials included in a systematic review on an efficacy-effectiveness continuum.<h4>Methods</h4>We extracted rat ...[more]

PMID: 28188898

Similar Datasets

Project description:BACKGROUND:Nudge interventions are those that seek to modify the social and physical environment to enhance capacity for subconscious behaviours that align with the intrinsic values of an individual, without actively restricting options. This study sought to describe the application and effects of nudge strategies on clinician implementation of health-related guidelines, policies and practices within studies included in relevant Cochrane systematic reviews. METHODS:As there is varied terminology used to describe nudge, this study examined studies within relevant systematic reviews. A two-stage screening process was undertaken where, firstly, all systematic reviews published in the Cochrane Library between 2016 and 2018 were screened to identify reviews that included quantitative studies to improve implementation of guidelines among healthcare providers. Secondly, individual studies within relevant systematic reviews were included if they were (i) randomised controlled trials (RCTs), (ii) included a nudge strategy in at least one intervention arm, and (iii) explicitly aimed to improve clinician implementation behaviour. We categorised nudge strategies into priming, salience and affect, default, incentives, commitment and ego, and norms and messenger based on the Mindspace framework. SYNTHESIS:The number and percentage of trials using each nudge strategy was calculated. Due to substantial heterogeneity, we did not undertake a meta-analysis. Instead, we calculated within-study point estimates and 95% confidence intervals, and used a vote-counting approach to explore effects. RESULTS:Seven reviews including 42 trials reporting on 57 outcomes were included. The most common nudge strategy was priming (69%), then norms and messenger (40%). Of the 57 outcomes, 86% had an effect on clinician behaviour in the hypothesised direction, and 53% of those were statistically significant. For continuous outcomes, the median effect size was 0.39 (0.22, 0.45), while for dichotomous outcomes the median Odds Ratio was 1.62 (1.13, 2.76). CONCLUSIONS:This review of 42 RCTs included in Cochrane systematic reviews found that the impact of nudge strategies on clinician behaviour was at least comparable to other interventions targeting implementation of evidence-based guidelines. While uncertainty remains, the review provides justification for ongoing investigation of the evaluation and application of nudge interventions to support provider behaviour change. TRIAL REGISTRATION:This review was not prospectively registered.

Project description:IntroductionThe validity of systematic reviews and meta-analysis depends on methodological quality and unbiased dissemination of trials. Our objective is to evaluate the association of estimates of treatment effects with different bias-related study characteristics in meta-analyses of interventions used for treating pain in osteoarthritis (OA). From the findings, we hope to consolidate guidance on interpreting OA trials in systematic reviews based on empirical evidence from Cochrane reviews.Methods and analysisOnly systematic reviews that compare experimental interventions with sham, placebo or no intervention control will be considered eligible. Bias will be assessed with the risk of bias tool, used according to the Cochrane Collaboration's recommendations. Furthermore, center status, trial size and funding will be assessed. The primary outcome (pain) will be abstracted from the first appearing forest plot for overall pain in the Cochrane review. Treatment effect sizes will be expressed as standardised mean differences (SMDs), where the difference in mean values available from the forest plots is divided by the pooled SD. To empirically assess the risk of bias in treatment benefits, we will perform stratified analyses of the trials from the included meta-analyses and assess the interaction between trial characteristics and treatment effect. A relevant study-level covariate is defined as one that decreases the between-study variance (τ(2), estimated as Tau-squared) as a consequence of inclusion in the mixed effects statistical model.Ethics and disseminationMeta-analyses and randomised controlled trials provide the most reliable basis for treatment of patients with OA, but the actual impact of bias is unclear. This study will systematically examine the methodological quality in OA Cochrane reviews and explore the effect estimates behind possible bias. Since our study does not collect primary data, no formal ethical assessment and informed consent are required.Trial registration numberPROSPERO (CRD42013006924).

Project description:UnlabelledBackgroundCo-morbid symptoms (for example, chronic pain, depression, anxiety, and fatigue) are particularly common in military fighters returning from the current conflicts, who have experienced physical and/or psychological trauma. These overlapping conditions cut across the boundaries of mind, brain and body, resulting in a common symptomatic and functional spectrum of physical, cognitive, psychological and behavioral effects referred to as the 'Trauma Spectrum Response' (TSR). While acupuncture has been shown to treat some of these components effectively, the current literature is often difficult to interpret, inconsistent or of variable quality. Thus, to gauge comprehensively the effectiveness of acupuncture across TSR components, a systematic review of reviews was conducted using the Samueli Institute's Rapid Evidence Assessment of the Literature (REAL©) methodology.MethodsPubMed/MEDLINE, the Cochrane Database of Systematic Reviews, EMBASE, CINAHL, and PsycInfo were searched from inception to September 2011 for systematic reviews/meta-analyses. Quality assessment was rigorously performed using the Scottish Intercollegiate Guidelines Network (SIGN 50) checklist and the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) methodology. Adherence to the Standards for Reporting Interventions in Clinical Trials in Acupuncture (STRICTA) criteria was also assessed.ResultsOf the 1,480 citations identified by our searches, 52 systematic reviews/meta-analyses, all high quality except for one, met inclusion criteria for each TSR component except post-traumatic stress disorder (PTSD) and sexual function. The majority of reviews addressed most STRICTA components, but did not describe safety.ConclusionsBased on the results of our review, acupuncture appears to be effective for treating headaches and, although more research is needed, seems to be a promising treatment option for anxiety, sleep disturbances, depression and chronic pain. It does not, however, demonstrate any substantial treatment benefit for substance abuse. Because there were no reviews on PTSD or sexual function that met our pre-defined inclusion criteria, we cannot comment on acupuncture's effectiveness in treating these conditions. More quality data are also needed to determine whether acupuncture is appropriate for treating fatigue or cognitive difficulties. Further, while acupuncture has been shown to be generally safe, safety was not described in the majority of studies, making it difficult to provide any strong recommendations. Future research should address safety reporting in detail in order to increase our confidence in acupuncture's efficacy across the identified TSR components.

Project description:BackgroundClinical decisions are made based on Cochrane reviews, but the implementation of results of evidence syntheses such as Cochrane reviews is problematic if the evidence is not prepared consistently. All systematic reviews should assess the risk of bias (RoB) in included studies, and in Cochrane reviews, this is done by using Cochrane RoB tool. However, the tool is not necessarily applied according to the instructions. In this study, we aimed to determine the types of bias and their corresponding judgements noted in the 'other bias' domain of Cochrane RoB tool.MethodsWe analyzed Cochrane reviews that included randomized controlled trials (RCTs) and extracted data regarding 'other bias' from the RoB table and accompanying support for the judgment. We categorized different types of other bias.ResultsWe analyzed 768 Cochrane reviews that included 11,369 RCTs. There were 602 (78%) Cochrane reviews that had 'other bias' domain in the RoB tool, and they included a total of 7811 RCTs. In the RoB table of 337 Cochrane reviews for at least one of the included trials it was indicated that no other bias was found and supporting explanations were inconsistently judged as low, unclear or high RoB. In the 524 Cochrane reviews that described various sources of other bias, there were 5762 individual types of explanations which we categorized into 31 groups. The judgments of the same supporting explanations were highly inconsistent. We found numerous other inconsistencies in reporting of sources of other bias in Cochrane reviews.ConclusionCochrane authors mention a wide range of sources of other bias in the RoB tool and they inconsistently judge the same supporting explanations. Inconsistency in appraising risk of other bias hinders reliability and comparability of Cochrane systematic reviews. Discrepant and erroneous judgments of bias in evidence synthesis may hinder implementation of evidence in routine clinical practice and reduce confidence in otherwise trustworthy sources of information. These results can help authors of Cochrane and non-Cochrane reviews to gain insight into various sources of other bias that can be found in trials, and also to help them avoid mistakes that were recognized in published Cochrane reviews.

Dataset Information

Rating of Included Trials on the Efficacy-Effectiveness Spectrum: development of a new tool for systematic reviews.

Publications

Rating of Included Trials on the Efficacy-Effectiveness Spectrum: development of a new tool for systematic reviews.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure