Dataset Information

Over half of clinical practice guidelines use non-systematic methods to inform recommendations: A methods study.

ABSTRACT:

Introduction

Assessing the process used to synthesize the evidence in clinical practice guidelines enables users to determine the trustworthiness of the recommendations. Clinicians are increasingly dependent on guidelines to keep up with vast quantities of medical literature, and guidelines are followed to avoid malpractice suits. We aimed to assess whether systematic methods were used when synthesizing the evidence for guidelines; and to determine the type of review cited in support of recommendations.

Methods

Guidelines published in 2017 and 2018 were retrieved from the TRIP and Epistemonikos databases. We randomly sorted and sequentially screened clinical guidelines on all topics to select the first 50 that met our inclusion criteria. Our primary outcomes were the number of guidelines using either a systematic or non-systematic process to gather, assess, and synthesise evidence; and the numbers of recommendations within guidelines based on different types of evidence synthesis (systematic or non-systematic reviews). If a review was cited, we looked for evidence that it was critically appraised, and recorded which quality assessment tool was used. Finally, we examined the relation between the use of the GRADE approach, systematic review process, and type of funder.

Results

Of the 50 guidelines, 17 (34%) systematically synthesised the evidence to inform recommendations. These 17 guidelines clearly reported their objectives and eligibility criteria, conducted comprehensive search strategies, and assessed the quality of the studies. Of the 29/50 guidelines that included reviews, 6 (21%) assessed the risk of bias of the review. The quality of primary studies was reported in 30/50 (60%) guidelines.

Conclusions

High quality, systematic review products provide the best available evidence to inform guideline recommendations. Using non-systematic methods compromises the validity and reliability of the evidence used to inform guideline recommendations, leading to potentially misleading and untrustworthy results.

SUBMITTER: Lunny C

PROVIDER: S-EPMC8062080 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:INTRODUCTION:Guidelines are systematically developed recommendations to assist practitioner and patient decisions about treatments for clinical conditions. High quality and comprehensive systematic reviews and 'overviews of systematic reviews' (overviews) represent the best available evidence. Many guideline developers, such as the WHO and the Australian National Health and Medical Research Council, recommend the use of these research syntheses to underpin guideline recommendations. We aim to evaluate the impact and use of systematic reviews with and without pairwise meta-analysis or network meta-analyses (NMAs) and overviews in clinical practice guideline (CPG) recommendations. METHODS AND ANALYSIS:CPGs will be retrieved from Turning Research Into Practice and Epistemonikos (2017-2018). The retrieved citations will be sorted randomly and then screened sequentially by two independent reviewers until 50 CPGs have been identified. We will include CPGs that provide at least two explicit recommendations for the management of any clinical condition. We will assess whether reviews or overviews were cited in a recommendation as part of the development process for guidelines. Data extraction will be done independently by two authors and compared. We will assess the risk of bias by examining how each guideline developed clinical recommendations. We will calculate the number and frequency of citations of reviews with or without pairwise meta-analysis, reviews with NMAs and overviews, and whether they were systematically or non-systematically developed. Results will be described, tabulated and categorised based on review type (reviews or overviews). CPGs reporting the use of the Grading of Recommendations, Assessment, Development and Evaluation approach will be compared with those using a different system, and pharmacological versus non-pharmacological CPGs will be compared. ETHICS AND DISSEMINATION:No ethics approval is required. We will present at the Cochrane Colloquium and the Guidelines International Network conference.

Project description:ObjectiveTo identify and summarize clinical practice guidelines for autism spectrum disorder (ASD) and intellectual disability (ID) for the Package of Interventions for Rehabilitation for the World Health Organization (WHO).Data sourcesAcademic databases, Google Scholar, guideline databases, and professional society websites were searched using the general criteria "ASD/ID" AND "rehabilitation" AND "guideline," restricted to English-only guidelines.Study selectionWork group members independently screened titles and abstracts (1952 ASD; 1027 ID) and excluded articles if not (1) a guideline; (2) about rehabilitation; (3) published since 2008; or (4) about ASD/ID. Full-text screening (29 ASD; 5 ID) involved 3 additional exclusion criteria: (1) contained conflict of interest; (2) lacked information on strength of recommendation; or (3) failed the Appraisal of Guidelines for Research and Evaluation II instrument. Six guidelines (4 ASD: 2 on youth, 1 on adults, 1 on all ages; 2 ID: 1 on challenging behaviors, 1 on mental health) resulted.Data extractionWork group members extracted 524 recommendations (386 ASD; 138 ID) from the guidelines including the level of evidence, diagnostic and age group, recommendation type (assessment, intervention, service), target, and valence.Data synthesisOf the 270 intervention recommendations (212 ASD; 58 ID), only 36 for ASD and 47 for ID were empirically based. Most comprised biomedical (23%), pharmacologic (29%), and psychosocial (21%) interventions for ASD and behavioral (14%), pharmacologic (29%), and psychological (14%) interventions for ID. Intervention recommendations primarily targeted coexisting conditions (56% ASD; 93% ID), whereas core symptoms received much less attention (26% ASD).ConclusionsClinical practice guidelines reviewed for ASD and ID primarily contained recommendations based on expert opinion, with the plurality of recommendations relating to pharmacologic treatment. Vital next steps include identifying relevant interventions for inclusion in the WHO Package and continuing to conduct rigorous intervention research, particularly on core symptoms of these conditions, to extend recommendations for high-quality guidelines.

Project description:IntroductionClinical practice guidelines(CPGs) are important tools for medical decision-making. Given the high prevalence and financial burden associated with tobacco use disorder(TUD), it is critical that recommendations within CPGs are based on robust evidence. Systematic reviews(SRs) are considered the highest level of evidence, thus, we evaluated the quality of SRs underpinning CPG recommendations for TUD.MethodsWe used PubMed to search for CPGs relating to TUD published between January 1, 2010 and May 21, 2021. SRs were extracted from CPG references and evaluated using Preferred Reporting Items for Systematic Reviews and Meta-Analyses(PRISMA) and A MeaSurement Tool to Assess Systematic Reviews(AMSTAR-2) tools. We then compared SRs conducted by the Cochrane Collaboration with non-Cochrane SRs using a Mann-Whitney U test and determined associations between PRISMA and AMSTAR-2 extracted characteristics using multiple regression.ResultsOur search generated 10 CPGs with 98 SRs extracted. Mean PRISMA completion was 74.7%(SD = 16.7) and mean AMSTAR-2 completion was 53.8%(SD = 22.0) across all guidelines. Cochrane SRs were more complete than non-Cochrane studies in the PRISMA and AMSTAR-2 assessments. The regression model showed a statistically significant association between PRISMA completion and AMSTAR-2 rating, with those classified as "low" or "moderate" quality having higher PRISMA completion than those with "critically low" ratings.ConclusionWe found substandard adherence to PRISMA and AMSTAR-2 checklists across SRs cited in TUD CPGs. A lack of recent SRs in CPGs could lead to outdated recommendations. Therefore, frequent guideline updates with recently published evidence may ensure more accurate clinical recommendations and improve patient care.ImplicationsSystematic reviews used to underpin clinical practice guideline recommendations influence treatment decisions and, ultimately, patient outcomes. We found that many systematic reviews underpinning tobacco use disorder guideline recommendations were out of date and unsatisfactory in reporting and quality. Thus, including newer systematic reviews containing more recently conducted trials and better reporting could alter recommendations and improve the rate of successful tobacco cessation attempts.

Project description:ObjectiveTo systematically summarise acupuncture-related Clinical Practice Guidelines (CPGs)'s clinical and methodological characteristics and critically appraise their methodology quality.DesignWe summarised the characteristics of the guidelines and recommendations and evaluated their methodological quality using the Appraisal of Guidelines Research and Evaluation II (AGREE II) instrument.Data sourcesNine databases were searched from 1 January 2010 to 20 September 2020.Eligibility criteria for selecting studiesWe included the latest version of acupuncture CPGs, which must have used at least one systematic review addressing the benefits and harms of alternative care options to inform acupuncture recommendations.Data extraction and synthesisReviewers, working in pairs, independently screened and extracted data. When there are statistical differences among types of CPGs, we reported the data by type in the text, but when not, we reported the overall data.ResultsOf the 133 eligible guidelines, musculoskeletal and connective tissue diseases proved the most commonly addressed therapeutic areas. According to the AGREE II instrument, the CPG was moderate quality in the domain of clarity of scope and purpose, clarity of presentation, the rigour of development, stakeholder involvement and low quality in editorial independence, and applicability. The study identified 433 acupuncture-related recommendations; 380 recommended the use of acupuncture, 28 recommended against the use of acupuncture and 25 considered acupuncture but did not make recommendations. Of the 303 recommendations that used Grading of Recommendations Assessment, Development and Evaluation to determine the strength of recommendations, 152 were weak recommendations, 131 were strong recommendations, of which 104 were supported by low or very low certainty evidence (discordant recommendations).ConclusionIn the past 10 years, a large number of CPGs addressing acupuncture interventions exist. Although these guidelines may be as or more rigorous than many others, considerable room for improvement remains.

Project description:Background Up to 50% of individuals diagnosed with depression are known to use complementary and alternative medicine (CAM). The aim of this study was to identify the quantity and assess the quality of CAM recommendations in clinical practice guidelines for the treatment and/or management of depression in adults using the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument. Methods A systematic review was conducted to identify depression guidelines. MEDLINE, EMBASE and CINAHL were searched from 2008 to 2018. The Guidelines International Network and the National Center for Complementary and Integrative Health websites were also searched. Eligible guidelines published by non-profit agencies on treatment of depression for adults were assessed with the AGREE II instrument twice, once for the overall guideline and once for the CAM sections. Results From 931 unique search results, 19 guidelines mentioned CAM, of which 16 made CAM recommendations. Scaled domain percentages from highest to lowest were as follows (overall, CAM section): clarity of presentation (87.0, 66.1%), scope and purpose (80.9, 77.6%), stakeholder involvement (62.0, 44.3%), editorial independence (61.6, 61.6%), rigour of development (58.0, 52.0%), and applicability (42.2, 25.4%). Quality varied within and across guidelines. Only 1 of 16 guidelines was recommended without modifications for both its overall and CAM sections by both appraisers. Conclusions There are multiple depression guidelines containing CAM recommendations available and there are a comprehensive set of CAM therapy options for depression. The quality of guidelines varied within and across guidelines and the quality of CAM recommendations was generally lower than the overall recommendations in the guidelines for all domains except editorial independence. Generally, characteristics of guidelines, including the year of publication and region of development varied across the guidelines irrespective of quality. Guidelines with higher AGREE II scores can serve as a guide to facilitate communication between patients and medical professionals regarding CAM use for depression, while guidelines with lower scores could be improved in future updates using the AGREE II instrument as a guide.

Project description:BACKGROUND:Bladder cancer (BC) has become a major worldwide public health issue, especially non-muscle-invasive bladder cancer (NMIBC). A flood of related clinical practice guidelines (CPGs) have emerged; however, the quality and recommendations of the guidelines are controversial. We aimed to appraise the quality of the CPGs for NMIBC within the past 5?years and compare the similarities and differences between recommendations for therapies. METHODS:A systematic search to identify CPGs for NMIBC was performed using electronic databases (including PubMed, Embase, Web of Science), guideline development organizations, and professional societies from January 12, 2014 to January 12, 2019. The Appraisal of Guidelines Research & Evaluation (AGREE) II instrument was used to evaluate the quality of the guidelines. Intraclass correlation coefficient (ICC) analysis was performed to assess the overall agreement among reviewers. RESULTS:Nine CPGs were included. The overall agreement among reviewers was excellent. The interquartile range (IQR) of scores for each domain were as follows: scope and purpose 69.44% (35.42, 85.42%); stakeholder involvement 41.67% (30.56, 75.00%); rigour of development 48.96% (27.08, 65.63%); clarity and presentation 80.56% (75.00, 86.11%); applicability 34.38% (22.92, 40.63%) and editorial independence 70.83% (35.42, 85.42%). The NICE, AUA, EAU and CRHA/CPAM clinical practice guidelines consistently scored well in most domains. It was generally accepted that the transurethral resection of bladder tumour (TURBT) and intravesical chemotherapy should be performed in the management of bladder cancer. The application of chemotherapy was highly controversial in high risk NMIBC. The courses of BCG maintenance were similar and included 3?years of therapy at full maintenance doses. CONCLUSIONS:The quality of NMIBC guidelines within the past 5?years varied, especially regarding stakeholders, rigour and applicability. Despite many similarities, the recommendations had some inconsistencies in the details.

Project description:Quality of healthcare can be improved when the best external evidence available is integrated in clinical decision-making in a systematic explicit manner. With the rapid expansion of clinical evidence, the opportunities for evidence-based high-quality healthcare increase. Paradoxically, the likelihood of any one person to get a complete and balanced picture of the evidence available decreases. This is especially true for rehabilitation interventions that are complex in nature and where clinical research is rather diverse. Given the complex nature of the evidence, there is a substantial risk of misinterpreting the complex information both at the level of individual sources (e.g., reports of clinical trials) and for aggregated data syntheses (e.g., systematic reviews and meta-analyses). These risks are inherent in these sources themselves and are in addition related to the methodological expertise necessary to make valid use of the evidence for clinical decision-making. Taken together, there is a great demand for systematic structured guidance from evidence to clinical decision. This methodology paper describes a structured process for the development and report of evidence-based clinical practice recommendations that uses systematic reviews and meta-analyses as evidence source. It provides a comprehensive framework with specific requirements for the development group, the formulation of the healthcare question addressed, the systematic search for the evidence, its critical appraisal, the extraction and the outcome-centered presentation of the evidence, the rating of its quality, strengths and weaknesses, any further considerations relevant for decision-making, and an explicit recommendation statement along with its justification, implementation, and resource aspects. The suggested methodology uses international standards in evidence synthesis, critical appraisal of systematic reviews, rating the quality of evidence, characteristics of recommendations, and guideline development as developed by Cochrane, GRADE (Grading of Recommendations Assessment, Development and Evaluation), AMSTAR (A MeaSurement Tool to Assess systematic Reviews), and AGREE (Appraisal of Guidelines for REsearch & Evaluation). An added distinctive feature of the methodology is to focus on the most up-to-date, most valid evidence and hence to support the development of valid practice recommendations in an efficient way. Practice recommendations generated by such a valid methodology would be generally applicable and promote evidence-based clinical practice globally.

Project description:BackgroundIt is not known whether there are differences in the quality and recommendations between evidence-based (EB) and consensus-based (CB) guidelines. We used breast cancer guidelines as a case study to assess for these differences.MethodsFive different instruments to evaluate the quality of guidelines were identified by a literature search. We also searched MEDLINE and the Internet to locate 8 breast cancer guidelines. These guidelines were classified in three categories: evidence based, consensus based and consensus based with no explicit consideration of evidence (CB-EB). Each guideline was evaluated by three of the authors using each of the instruments. For each guideline we assessed the agreement among 14 decision points which were selected from the NCCN (National Cancer Comprehensive Network) guidelines algorithm. For each decision point we recorded the level of the quality of the information used to support it. A regression analysis was performed to assess if the percentage of high quality evidence used in the guidelines development was related to the overall quality of the guidelines.ResultsThree guidelines were classified as EB, three as CB-EB and two as CB. The EB guidelines scored better than CB, with the CB-EB scoring in the middle among all instruments for guidelines quality assessment. No major disagreement in recommendations was detected among the guidelines regardless of the method used for development, but the EB guidelines had a better agreement with the benchmark guideline for any decision point. When the source of evidence used to support decision were of high quality, we found a higher level of full agreement among the guidelines' recommendations. Up to 94% of variation in the quality score among guidelines could be explained by the quality of evidence used for guidelines development.ConclusionEB guidelines have a better quality than CB guidelines and CB-EB guidelines. Explicit use of high quality evidence can lead to a better agreement among recommendations. However, no major disagreement among guidelines was noted regardless of the method for their development.