Usefulness of Cochrane Reviews in Clinical Guideline Development-A Survey of 585 Recommendations.
Ontology highlight
ABSTRACT:
SUBMITTER: Korfitsen CB
PROVIDER: S-EPMC8775999 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:INTRODUCTION:Guidelines are systematically developed recommendations to assist practitioner and patient decisions about treatments for clinical conditions. High quality and comprehensive systematic reviews and 'overviews of systematic reviews' (overviews) represent the best available evidence. Many guideline developers, such as the WHO and the Australian National Health and Medical Research Council, recommend the use of these research syntheses to underpin guideline recommendations. We aim to evaluate the impact and use of systematic reviews with and without pairwise meta-analysis or network meta-analyses (NMAs) and overviews in clinical practice guideline (CPG) recommendations. METHODS AND ANALYSIS:CPGs will be retrieved from Turning Research Into Practice and Epistemonikos (2017-2018). The retrieved citations will be sorted randomly and then screened sequentially by two independent reviewers until 50 CPGs have been identified. We will include CPGs that provide at least two explicit recommendations for the management of any clinical condition. We will assess whether reviews or overviews were cited in a recommendation as part of the development process for guidelines. Data extraction will be done independently by two authors and compared. We will assess the risk of bias by examining how each guideline developed clinical recommendations. We will calculate the number and frequency of citations of reviews with or without pairwise meta-analysis, reviews with NMAs and overviews, and whether they were systematically or non-systematically developed. Results will be described, tabulated and categorised based on review type (reviews or overviews). CPGs reporting the use of the Grading of Recommendations, Assessment, Development and Evaluation approach will be compared with those using a different system, and pharmacological versus non-pharmacological CPGs will be compared. ETHICS AND DISSEMINATION:No ethics approval is required. We will present at the Cochrane Colloquium and the Guidelines International Network conference.
| S-EPMC7044835 | biostudies-literature
Project description:BACKGROUND:Despite the growing reputation and subject coverage of the Cochrane Database of Systematic Reviews, many systematic reviews continue to be published solely in paper-based health care journals. This study was designed to determine why authors choose to publish their systematic reviews outside of the Cochrane Collaboration and if they might be interested in converting their reviews to Cochrane format for publication in the Cochrane Database of Systematic Reviews. METHODS:Cross-sectional survey of Australian primary authors of systematic reviews not published on the Cochrane Database of Systematic Reviews identified from the Database of Abstracts of Reviews of Effectiveness. RESULTS:We identified 88 systematic reviews from the Database of Abstracts of Reviews of Effectiveness with an Australian as the primary author. We surveyed 52 authors for whom valid contact information was available. The response rate was 88 per cent (46/52). Ten authors replied without completing the survey, leaving 36 valid surveys for analysis. The most frequently cited reasons for not undertaking a Cochrane review were: lack of time (78%), the need to undergo specific Cochrane training (46%), unwillingness to update reviews (36%), difficulties with the Cochrane process (26%) and the review topic already registered with the Cochrane Collaboration (21%). (Percentages based on completed responses to individual questions.) Nearly half the respondents would consider converting their review to Cochrane format. Dedicated time emerged as the most important factor in facilitating the potential conversion process. Other factors included navigating the Cochrane system, assistance with updating and financial support. Eighty-six per cent were willing to have their review converted to Cochrane format by another author. CONCLUSION:Time required to complete a Cochrane review and the need for specific training are the primary reasons why some authors publish systematic reviews outside of the Cochrane Collaboration. Encouragingly, almost half of the authors would consider converting their review to Cochrane format. Based on the current number of reviews in the Database of Abstracts of Reviews of Effectiveness, this could result in more than 700 additional Cochrane reviews. Ways of supporting these authors and how to provide dedicated time to convert systematic reviews needs further consideration.
| S-EPMC149345 | biostudies-literature
Project description:BACKGROUND: In contemporary medical research, randomised controlled trials are seen as the gold standard for establishing treatment effects where it is ethical and practical to conduct them. In palliative care such trials are often impractical, unethical, or extremely difficult, with multiple methodological problems. We review the utility of Cochrane reviews in informing palliative care practice. METHODS: Published reviews in palliative care registered with the Cochrane Pain, Palliative and Supportive Care Group as of December 2007 were obtained from the Cochrane Database of Systematic Reviews, issue 1, 2008. We reviewed the quality and quantity of primary studies available for each review, assessed the quality of the review process, and judged the strength of the evidence presented. There was no prior intention to perform any statistical analyses. RESULTS: 25 published systematic reviews were identified. Numbers of included trials ranged from none to 54. Within each review, included trials were heterogeneous with respect to patients, interventions, and outcomes, and the number of patients contributing to any single analysis was generally much lower than the total included in the review. A variety of tools were used to assess trial quality; seven reviews did not use this information to exclude low quality studies, weight analyses, or perform sensitivity analysis for effect of low quality. Authors indicated that there were frequently major problems with the primary studies, individually or in aggregate. Our judgment was that the reviewing process was generally good in these reviews, and that conclusions were limited by the number, size, quality and validity of the primary studies.We judged the evidence about 23 of the 25 interventions to be weak. Two reviews had stronger evidence, but with limitations due to methodological heterogeneity or definition of outcomes. No review provided strong evidence of no effect. CONCLUSION: Cochrane reviews in palliative care are well performed, but fail to provide good evidence for clinical practice because the primary studies are few in number, small, clinically heterogeneous, and of poor quality and external validity. They are useful in highlighting the weakness of the evidence base and problems in performing trials in palliative care.
| S-EPMC2532992 | biostudies-literature
Project description:In 1996, shortly after the founding of The Cochrane Collaboration, leading figures in test evaluation research established a Methods Group to focus on the relatively new and rapidly evolving methods for the systematic review of studies of diagnostic tests. Seven years later, the Collaboration decided it was time to develop a publication format and methodology for Diagnostic Test Accuracy (DTA) reviews, as well as the software needed to implement these reviews in The Cochrane Library. A meeting hosted by the German Cochrane Centre in 2004 brought together key methodologists in the area, many of whom became closely involved in the subsequent development of the methodological framework for DTA reviews. DTA reviews first appeared in The Cochrane Library in 2008 and are now an integral part of the work of the Collaboration.
| S-EPMC3851548 | biostudies-literature
Project description:BackgroundThe American Thoracic Society (ATS)/Infectious Diseases Society of America (IDSA) Community-acquired Pneumonia (CAP) guidelines were developed using systematic reviews to inform every recommendation, as suggested by the Institute of Medicine Standards for Trustworthy Guidelines. Recent studies suggest that an expert consensus-based approach, called the Convergence of Opinion on Recommendations and Evidence (CORE) process, can produce recommendations that are concordant with recommendations informed by systematic reviews.PurposeThe goal of the study was to evaluate the efficacy of the CORE process had it been used to develop the ATS/IDSA CAP guidelines.MethodsExperts in CAP who were not on the guideline panel and had no knowledge of the guideline's systematic reviews or recommendations were recruited to participate in the CORE process, addressing the same questions asked by the guideline panel. Recommendations derived from the CORE process were compared to the guideline recommendations. Concordance of the course of action, strength of recommendation, and quality of evidence were determined.ResultsUsing a threshold of 70% of experts selecting the same course of action to make a recommendation, the CORE process yielded a recommendation for 20 of 31 (65%) questions. Among the 20 CORE-derived recommendations, 19 (95%) were concordant with the guideline recommendations (kappa agreement 0.88, 95% CI .64-1.00). There was less agreement among the strength of recommendations (58%) and quality of evidence (42%).ConclusionsIf the CORE process had been used, 11 systematic reviews would have been necessary rather than 31, with minimal impact on the recommended courses of action.
| S-EPMC8677595 | biostudies-literature
Project description:BackgroundSystematic reviews with meta-analyses often contain many statistical tests. This multiplicity may increase the risk of type I error. Few attempts have been made to address the problem of statistical multiplicity in systematic reviews. Before the implications are properly considered, the size of the issue deserves clarification. Because of the emphasis on bias evaluation and because of the editorial processes involved, Cochrane reviews may contain more multiplicity than their non-Cochrane counterparts. This study measured the quantity of statistical multiplicity present in a population of systematic reviews and aimed to assess whether this quantity is different in Cochrane and non-Cochrane reviews.Methods/principal findingsWe selected all the systematic reviews published by the Cochrane Anaesthesia Review Group containing a meta-analysis and matched them with comparable non-Cochrane reviews. We counted the number of statistical tests done in each systematic review. The median number of tests overall was 10 (interquartile range (IQR) 6 to 18). The median was 12 in Cochrane and 8 in non-Cochrane reviews (difference in medians 4 (95% confidence interval (CI) 2.0-19.0). The proportion that used an assessment of risk of bias as a reason for doing extra analyses was 42% in Cochrane and 28% in non-Cochrane reviews (difference in proportions 14% (95% CI -8 to 36). The issue of multiplicity was addressed in 6% of all the reviews.Conclusion/significanceStatistical multiplicity in systematic reviews requires attention. We found more multiplicity in Cochrane reviews than in non-Cochrane reviews. Many of the reasons for the increase in multiplicity may well represent improved methodological approaches and greater transparency, but multiplicity may also cause an increased risk of spurious conclusions. Few systematic reviews, whether Cochrane or non-Cochrane, address the issue of multiplicity.
| S-EPMC3229598 | biostudies-literature
Project description:Importance:Clinical practice guidelines (CPGs) may lack rigor and suitability to the setting in which they are to be applied. Methods to yield clinical practice guideline recommendations that are credible and implementable remain to be determined. Objective:To describe the development of AGREE-REX (Appraisal of Guidelines Research and Evaluation-Recommendations Excellence), a tool designed to evaluate the quality of clinical practice guideline recommendations. Design, Setting, and Participants:A cross-sectional study of 322 international stakeholders representing CPG developers, users, and researchers was conducted between December 2015 and March 2019. Advertisements to participate were distributed through professional organizations as well as through the AGREE Enterprise social media accounts and their registered users. Exposures:Between 2015 and 2017, participants appraised 1 of 161 CPGs using the Draft AGREE-REX tool and completed the AGREE-REX Usability Survey. Main Outcomes and Measures:Usability and measurement properties of the tool were assessed with 7-point scales (1 indicating strong disagreement and 7 indicating strong agreement). Internal consistency of items was assessed with the Cronbach ?, and the Spearman-Brown reliability adjustment was used to calculate reliability for 2 to 5 raters. Results:A total of 322 participants (202 female participants [62.7%]; 83 aged 40-49 years [25.8%]) rated the survey items (on a 7-point scale). All 11 items were rated as easy to understand (with a mean [SD] ranging from 5.2 [1.38] for the alignment of values item to 6.3 [0.87] for the evidence item) and easy to apply (with a mean [SD] ranging from 4.8 [1.49] for the alignment of values item to 6.1 [1.07] for the evidence item). Participants provided favorable feedback on the tool's instructions, which were considered clear (mean [SD], 5.8 [1.06]), helpful (mean [SD], 5.9 [1.00]), and complete (mean [SD], 5.8 [1.11]). Participants considered the tool easy to use (mean [SD], 5.4 [1.32]) and thought that it added value to the guideline enterprise (mean [SD], 5.9 [1.13]). Internal consistency of the items was high (Cronbach ??=?0.94). Positive correlations were found between the overall AGREE-REX score and the implementability score (r?=?0.81) and the clinical credibility score (r?=?0.76). Conclusions and Relevance:This cross-sectional study found that the AGREE-REX tool can be useful in evaluating CPG recommendations, differentiating among them, and identifying those that are clinically credible and implementable for practicing health professionals and decision makers who use recommendations to inform clinical policy.
| S-EPMC7254179 | biostudies-literature
Project description:BACKGROUND:Research turnover in the HIV field is rapid, and as a result, maintaining high-quality, up-to-date, and relevant systematic reviews is a challenge. One approach is to frequently update published reviews. METHODS:We evaluated the methods and relevance of all HIV systematic reviews and protocols published in the Cochrane Library over a 16-year period (2000-2016) to determine the need to update published reviews or complete of reviews in progress. RESULTS:Of 148 published reviews and protocols, 129 (87%) were identified as not for updating or progression to publication, mostly due to research questions which were either entirely outdated or addressed questions in an outdated manner (N?=?89; 60%); this was anticipated for older reviews, but was found also to be the case for recent publications. Some research questions were also inadequately conceptualized, particularly when complex pragmatic trials or behavioral interventions were included. CONCLUSIONS:We suggest that authors clearly characterize interventions and synthesis approaches in their review protocols. In research fields, such as HIV, where questions change frequently, systematic reviews and protocols should be regularly re-evaluated to ensure relevance to current questions. This process of re-evaluation should be incorporated into the methods of living systematic reviews.
| S-EPMC6366015 | biostudies-literature
Project description:BACKGROUND: Eczema is the most common inflammatory skin disease of childhood, characterized by an itchy red rash that usually involves the face and skin folds. There is currently no curative treatment for eczema, so the reduction of eczema incidence through disease prevention is a desirable goal. Potential interventions for preventing eczema include exclusive breastfeeding, hydrolysed protein formulas and soy formulas when bottle feeding, maternal antigen avoidance, omega oil supplementation, prebiotics and probiotics. OBJECTIVES: This overview of reviews aims to present the current body of data from Cochrane and non-Cochrane reviews to provide the most up-to-date evidence on the efficacy and safety of interventions to prevent eczema in infants and children at different risk levels for developing allergic disease. METHODS: Our pool of Cochrane and non-Cochrane reviews came from the 2010 United Kingdom National Health Service (NHS) Evidence Skin Disorders Annual Evidence Updates Mapping Exercise on Atopic Eczema. This group used a comprehensive search strategy last conducted in August 2010 to identify all systematic reviews on eczema prevention. We identified all reviews that met our pre-specified inclusion criteria, and data were extracted, analysed, compiled into tables and synthesized using quantitative and qualitative methods. MAIN RESULTS: Seven systematic reviews containing 39 relevant trials with 11 897 participants were included in this overview. Overall, there was no clear evidence that any of the main interventions reviewed reduced eczema incidence. In subgroup analyses of infants at high risk of allergic disease, an observational study found that exclusive breastfeeding for at least six months compared with introduction of solids at three to six months decreased the incidence of eczema by 60% (risk ratio (RR): 0.40; 95% confidence interval (CI): 0.21, 0.78), and a randomized controlled trial found that prebiotics compared with no prebiotics decreased incidence by 58% (RR: 0.42; 95% CI: 0.21, 0.84). However, each of these findings was based on the results of a single small trial, and no intervention reduced eczema incidence beyond the first two years of life. Although we pre-specified incidence of atopic eczema (i.e. eczema associated with immunoglobulin E (IgE) sensitization) as a primary outcome, data on whether participants diagnosed with eczema were truly atopic were largely lacking from systematic reviews. Similarly, data on atopy, measured using skin prick tests or specific IgE tests to allergens, were not reported in many reviews. No interventions were found to decrease atopy when reported. Adverse events data were generally lacking, but data from a trial of probiotics versus no probiotics showed significantly more spitting up in the first one (RR: 1.88; 95% CI: 1.03, 3.45) and two (RR: 1.69; 95% CI: 1.02, 2.80) months of life, but no overall increase in risk of gastrointestinal symptoms in the first year. AUTHORS#ENTITYSTARTX02019; CONCLUSIONS: Although there is currently no clear evidence showing that any of the interventions examined in this overview prevent eczema in participants not selected for risk of allergic disease, there is some evidence that exclusive breastfeeding for at least six months and prebiotics might reduce eczema incidence in high-risk participants. However, these conclusions are based on limited evidence with methodological shortcomings. Future research on prevention of eczema is needed and should examine different types of hydrolysed formulas, prebiotics and probiotics, as well as enhancement of the skin barrier and other novel approaches in infants at different risk levels for developing allergic disease.
| S-EPMC3399595 | biostudies-other
Project description:BackgroundCritics of systematic reviews have argued that these studies often fail to inform clinical decision making because their results are far too general, that the data are sparse, such that findings cannot be applied to individual patients or for other decision making. While there is some consensus on methods for investigating statistical and methodological heterogeneity, little attention has been paid to clinical aspects of heterogeneity. Clinical heterogeneity, true effect heterogeneity, can be defined as variability among studies in the participants, the types or timing of outcome measurements, and the intervention characteristics. The objective of this project was to develop recommendations for investigating clinical heterogeneity in systematic reviews.MethodsWe used a modified Delphi technique with three phases: (1) pre-meeting item generation; (2) face-to-face consensus meeting in the form of a modified Delphi process; and (3) post-meeting feedback. We identified and invited potential participants with expertise in systematic review methodology, systematic review reporting, or statistical aspects of meta-analyses, or those who published papers on clinical heterogeneity.ResultsBetween April and June of 2011, we conducted phone calls with participants. In June 2011 we held the face-to-face focus group meeting in Ann Arbor, Michigan. First, we agreed upon a definition of clinical heterogeneity: Variations in the treatment effect that are due to differences in clinically related characteristics. Next, we discussed and generated recommendations in the following 12 categories related to investigating clinical heterogeneity: the systematic review team, planning investigations, rationale for choice of variables, types of clinical variables, the role of statistical heterogeneity, the use of plotting and visual aids, dealing with outlier studies, the number of investigations or variables, the role of the best evidence synthesis, types of statistical methods, the interpretation of findings, and reporting.ConclusionsClinical heterogeneity is common in systematic reviews. Our recommendations can help guide systematic reviewers in conducting valid and reliable investigations of clinical heterogeneity. Findings of these investigations may allow for increased applicability of findings of systematic reviews to the management of individual patients.
| S-EPMC3847163 | biostudies-literature