Dataset Information

Over half of clinical practice guidelines use non-systematic methods to inform recommendations: A methods study.

ABSTRACT:

Introduction

Assessing the process used to synthesize the evidence in clinical practice guidelines enables users to determine the trustworthiness of the recommendations. Clinicians are increasingly dependent on guidelines to keep up with vast quantities of medical literature, and guidelines are followed to avoid malpractice suits. We aimed to assess whether systematic methods were used when synthesizing the evidence for guidelines; and to determine the type of review cited in support of recommendations.

Methods

Guidelines published in 2017 and 2018 were retrieved from the TRIP and Epistemonikos databases. We randomly sorted and sequentially screened clinical guidelines on all topics to select the first 50 that met our inclusion criteria. Our primary outcomes were the number of guidelines using either a systematic or non-systematic process to gather, assess, and synthesise evidence; and the numbers of recommendations within guidelines based on different types of evidence synthesis (systematic or non-systematic reviews). If a review was cited, we looked for evidence that it was critically appraised, and recorded which quality assessment tool was used. Finally, we examined the relation between the use of the GRADE approach, systematic review process, and type of funder.

Results

Of the 50 guidelines, 17 (34%) systematically synthesised the evidence to inform recommendations. These 17 guidelines clearly reported their objectives and eligibility criteria, conducted comprehensive search strategies, and assessed the quality of the studies. Of the 29/50 guidelines that included reviews, 6 (21%) assessed the risk of bias of the review. The quality of primary studies was reported in 30/50 (60%) guidelines.

Conclusions

High quality, systematic review products provide the best available evidence to inform guideline recommendations. Using non-systematic methods compromises the validity and reliability of the evidence used to inform guideline recommendations, leading to potentially misleading and untrustworthy results.

SUBMITTER: Lunny C

PROVIDER: S-EPMC8062080 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:INTRODUCTION:Guidelines are systematically developed recommendations to assist practitioner and patient decisions about treatments for clinical conditions. High quality and comprehensive systematic reviews and 'overviews of systematic reviews' (overviews) represent the best available evidence. Many guideline developers, such as the WHO and the Australian National Health and Medical Research Council, recommend the use of these research syntheses to underpin guideline recommendations. We aim to evaluate the impact and use of systematic reviews with and without pairwise meta-analysis or network meta-analyses (NMAs) and overviews in clinical practice guideline (CPG) recommendations. METHODS AND ANALYSIS:CPGs will be retrieved from Turning Research Into Practice and Epistemonikos (2017-2018). The retrieved citations will be sorted randomly and then screened sequentially by two independent reviewers until 50 CPGs have been identified. We will include CPGs that provide at least two explicit recommendations for the management of any clinical condition. We will assess whether reviews or overviews were cited in a recommendation as part of the development process for guidelines. Data extraction will be done independently by two authors and compared. We will assess the risk of bias by examining how each guideline developed clinical recommendations. We will calculate the number and frequency of citations of reviews with or without pairwise meta-analysis, reviews with NMAs and overviews, and whether they were systematically or non-systematically developed. Results will be described, tabulated and categorised based on review type (reviews or overviews). CPGs reporting the use of the Grading of Recommendations, Assessment, Development and Evaluation approach will be compared with those using a different system, and pharmacological versus non-pharmacological CPGs will be compared. ETHICS AND DISSEMINATION:No ethics approval is required. We will present at the Cochrane Colloquium and the Guidelines International Network conference.

Project description:ObjectiveTo identify and summarize clinical practice guidelines for autism spectrum disorder (ASD) and intellectual disability (ID) for the Package of Interventions for Rehabilitation for the World Health Organization (WHO).Data sourcesAcademic databases, Google Scholar, guideline databases, and professional society websites were searched using the general criteria "ASD/ID" AND "rehabilitation" AND "guideline," restricted to English-only guidelines.Study selectionWork group members independently screened titles and abstracts (1952 ASD; 1027 ID) and excluded articles if not (1) a guideline; (2) about rehabilitation; (3) published since 2008; or (4) about ASD/ID. Full-text screening (29 ASD; 5 ID) involved 3 additional exclusion criteria: (1) contained conflict of interest; (2) lacked information on strength of recommendation; or (3) failed the Appraisal of Guidelines for Research and Evaluation II instrument. Six guidelines (4 ASD: 2 on youth, 1 on adults, 1 on all ages; 2 ID: 1 on challenging behaviors, 1 on mental health) resulted.Data extractionWork group members extracted 524 recommendations (386 ASD; 138 ID) from the guidelines including the level of evidence, diagnostic and age group, recommendation type (assessment, intervention, service), target, and valence.Data synthesisOf the 270 intervention recommendations (212 ASD; 58 ID), only 36 for ASD and 47 for ID were empirically based. Most comprised biomedical (23%), pharmacologic (29%), and psychosocial (21%) interventions for ASD and behavioral (14%), pharmacologic (29%), and psychological (14%) interventions for ID. Intervention recommendations primarily targeted coexisting conditions (56% ASD; 93% ID), whereas core symptoms received much less attention (26% ASD).ConclusionsClinical practice guidelines reviewed for ASD and ID primarily contained recommendations based on expert opinion, with the plurality of recommendations relating to pharmacologic treatment. Vital next steps include identifying relevant interventions for inclusion in the WHO Package and continuing to conduct rigorous intervention research, particularly on core symptoms of these conditions, to extend recommendations for high-quality guidelines.

Project description:IntroductionClinical practice guidelines(CPGs) are important tools for medical decision-making. Given the high prevalence and financial burden associated with tobacco use disorder(TUD), it is critical that recommendations within CPGs are based on robust evidence. Systematic reviews(SRs) are considered the highest level of evidence, thus, we evaluated the quality of SRs underpinning CPG recommendations for TUD.MethodsWe used PubMed to search for CPGs relating to TUD published between January 1, 2010 and May 21, 2021. SRs were extracted from CPG references and evaluated using Preferred Reporting Items for Systematic Reviews and Meta-Analyses(PRISMA) and A MeaSurement Tool to Assess Systematic Reviews(AMSTAR-2) tools. We then compared SRs conducted by the Cochrane Collaboration with non-Cochrane SRs using a Mann-Whitney U test and determined associations between PRISMA and AMSTAR-2 extracted characteristics using multiple regression.ResultsOur search generated 10 CPGs with 98 SRs extracted. Mean PRISMA completion was 74.7%(SD = 16.7) and mean AMSTAR-2 completion was 53.8%(SD = 22.0) across all guidelines. Cochrane SRs were more complete than non-Cochrane studies in the PRISMA and AMSTAR-2 assessments. The regression model showed a statistically significant association between PRISMA completion and AMSTAR-2 rating, with those classified as "low" or "moderate" quality having higher PRISMA completion than those with "critically low" ratings.ConclusionWe found substandard adherence to PRISMA and AMSTAR-2 checklists across SRs cited in TUD CPGs. A lack of recent SRs in CPGs could lead to outdated recommendations. Therefore, frequent guideline updates with recently published evidence may ensure more accurate clinical recommendations and improve patient care.ImplicationsSystematic reviews used to underpin clinical practice guideline recommendations influence treatment decisions and, ultimately, patient outcomes. We found that many systematic reviews underpinning tobacco use disorder guideline recommendations were out of date and unsatisfactory in reporting and quality. Thus, including newer systematic reviews containing more recently conducted trials and better reporting could alter recommendations and improve the rate of successful tobacco cessation attempts.

Project description:ObjectiveTo systematically summarise acupuncture-related Clinical Practice Guidelines (CPGs)'s clinical and methodological characteristics and critically appraise their methodology quality.DesignWe summarised the characteristics of the guidelines and recommendations and evaluated their methodological quality using the Appraisal of Guidelines Research and Evaluation II (AGREE II) instrument.Data sourcesNine databases were searched from 1 January 2010 to 20 September 2020.Eligibility criteria for selecting studiesWe included the latest version of acupuncture CPGs, which must have used at least one systematic review addressing the benefits and harms of alternative care options to inform acupuncture recommendations.Data extraction and synthesisReviewers, working in pairs, independently screened and extracted data. When there are statistical differences among types of CPGs, we reported the data by type in the text, but when not, we reported the overall data.ResultsOf the 133 eligible guidelines, musculoskeletal and connective tissue diseases proved the most commonly addressed therapeutic areas. According to the AGREE II instrument, the CPG was moderate quality in the domain of clarity of scope and purpose, clarity of presentation, the rigour of development, stakeholder involvement and low quality in editorial independence, and applicability. The study identified 433 acupuncture-related recommendations; 380 recommended the use of acupuncture, 28 recommended against the use of acupuncture and 25 considered acupuncture but did not make recommendations. Of the 303 recommendations that used Grading of Recommendations Assessment, Development and Evaluation to determine the strength of recommendations, 152 were weak recommendations, 131 were strong recommendations, of which 104 were supported by low or very low certainty evidence (discordant recommendations).ConclusionIn the past 10 years, a large number of CPGs addressing acupuncture interventions exist. Although these guidelines may be as or more rigorous than many others, considerable room for improvement remains.

Project description:BACKGROUND:Bladder cancer (BC) has become a major worldwide public health issue, especially non-muscle-invasive bladder cancer (NMIBC). A flood of related clinical practice guidelines (CPGs) have emerged; however, the quality and recommendations of the guidelines are controversial. We aimed to appraise the quality of the CPGs for NMIBC within the past 5 years and compare the similarities and differences between recommendations for therapies. METHODS:A systematic search to identify CPGs for NMIBC was performed using electronic databases (including PubMed, Embase, Web of Science), guideline development organizations, and professional societies from January 12, 2014 to January 12, 2019. The Appraisal of Guidelines Research & Evaluation (AGREE) II instrument was used to evaluate the quality of the guidelines. Intraclass correlation coefficient (ICC) analysis was performed to assess the overall agreement among reviewers. RESULTS:Nine CPGs were included. The overall agreement among reviewers was excellent. The interquartile range (IQR) of scores for each domain were as follows: scope and purpose 69.44% (35.42, 85.42%); stakeholder involvement 41.67% (30.56, 75.00%); rigour of development 48.96% (27.08, 65.63%); clarity and presentation 80.56% (75.00, 86.11%); applicability 34.38% (22.92, 40.63%) and editorial independence 70.83% (35.42, 85.42%). The NICE, AUA, EAU and CRHA/CPAM clinical practice guidelines consistently scored well in most domains. It was generally accepted that the transurethral resection of bladder tumour (TURBT) and intravesical chemotherapy should be performed in the management of bladder cancer. The application of chemotherapy was highly controversial in high risk NMIBC. The courses of BCG maintenance were similar and included 3 years of therapy at full maintenance doses. CONCLUSIONS:The quality of NMIBC guidelines within the past 5 years varied, especially regarding stakeholders, rigour and applicability. Despite many similarities, the recommendations had some inconsistencies in the details.