Dataset Information

Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group.

ABSTRACT:

Background

A number of approaches have been used to grade levels of evidence and the strength of recommendations. The use of many different approaches detracts from one of the main reasons for having explicit approaches: to concisely characterise and communicate this information so that it can easily be understood and thereby help people make well-informed decisions. Our objective was to critically appraise six prominent systems for grading levels of evidence and the strength of recommendations as a basis for agreeing on characteristics of a common, sensible approach to grading levels of evidence and the strength of recommendations.

Methods

Six prominent systems for grading levels of evidence and strength of recommendations were selected and someone familiar with each system prepared a description of each of these. Twelve assessors independently evaluated each system based on twelve criteria to assess the sensibility of the different approaches. Systems used by 51 organisations were compared with these six approaches.

Results

There was poor agreement about the sensibility of the six systems. Only one of the systems was suitable for all four types of questions we considered (effectiveness, harm, diagnosis and prognosis). None of the systems was considered usable for all of the target groups we considered (professionals, patients and policy makers). The raters found low reproducibility of judgements made using all six systems. Systems used by 51 organisations that sponsor clinical practice guidelines included a number of minor variations of the six systems that we critically appraised.

Conclusions

All of the currently used approaches to grading levels of evidence and the strength of recommendations have important shortcomings.

SUBMITTER: Atkins D

PROVIDER: S-EPMC545647 | biostudies-literature | 2004 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group.

Atkins David D Eccles Martin M Flottorp Signe S Guyatt Gordon H GH Henry David D Hill Suzanne S Liberati Alessandro A O'Connell Dianne D Oxman Andrew D AD Phillips Bob B Schünemann Holger H Edejer Tessa Tan-Torres TT Vist Gunn E GE Williams John W JW

BMC health services research 20041222 1

<h4>Background</h4>A number of approaches have been used to grade levels of evidence and the strength of recommendations. The use of many different approaches detracts from one of the main reasons for having explicit approaches: to concisely characterise and communicate this information so that it can easily be understood and thereby help people make well-informed decisions. Our objective was to critically appraise six prominent systems for grading levels of evidence and the strength of recommen ...[more]

PMID: 15615589

Similar Datasets

Project description:BackgroundSystems that are used by different organisations to grade the quality of evidence and the strength of recommendations vary. They have different strengths and weaknesses. The GRADE Working Group has developed an approach that addresses key shortcomings in these systems. The aim of this study was to pilot test and further develop the GRADE approach to grading evidence and recommendations.MethodsA GRADE evidence profile consists of two tables: a quality assessment and a summary of findings. Twelve evidence profiles were used in this pilot study. Each evidence profile was made based on information available in a systematic review. Seventeen people were given instructions and independently graded the level of evidence and strength of recommendation for each of the 12 evidence profiles. For each example judgements were collected, summarised and discussed in the group with the aim of improving the proposed grading system. Kappas were calculated as a measure of chance-corrected agreement for the quality of evidence for each outcome for each of the twelve evidence profiles. The seventeen judges were also asked about the ease of understanding and the sensibility of the approach. All of the judgements were recorded and disagreements discussed.ResultsThere was a varied amount of agreement on the quality of evidence for the outcomes relating to each of the twelve questions (kappa coefficients for agreement beyond chance ranged from 0 to 0.82). However, there was fair agreement about the relative importance of each outcome. There was poor agreement about the balance of benefits and harms and recommendations. Most of the disagreements were easily resolved through discussion. In general we found the GRADE approach to be clear, understandable and sensible. Some modifications were made in the approach and it was agreed that more information was needed in the evidence profiles.ConclusionJudgements about evidence and recommendations are complex. Some subjectivity, especially regarding recommendations, is unavoidable. We believe our system for guiding these complex judgements appropriately balances the need for simplicity with the need for full and transparent consideration of all important issues.

Project description:Health systems guidance (HSG) are systematically developed statements that assist with decisions about options for addressing health systems challenges, including related changes in health systems arrangements. However, the development, appraisal, and reporting of HSG poses unique conceptual and methodological challenges related to the varied types of evidence that are relevant, the complexity of health systems, and the pre-eminence of contextual factors. To address this gap, we are conducting a program of research that aims to create a tool to support the appraisal of HSG and further enhance HSG development and reporting. The focus of this paper was to conduct a knowledge synthesis of the published and grey literatures to determine quality criteria (concepts) relevant for this process.We applied a critical interpretive synthesis (CIS) approach to knowledge synthesis that enabled an iterative, flexible, and dynamic analysis of diverse bodies of literature in order to generate a candidate list of concepts that will constitute the foundational components of the HSG tool. Using our review questions as compasses, we were able to guide the search strategy to look for papers based on their potential relevance to HSG appraisal, development, and reporting. The search strategy included various electronic databases and sources, subject-specific journals, conference abstracts, research reports, book chapters, unpublished data, dissertations, and policy documents. Screening the papers and data extraction was completed independently and in duplicate, and a narrative approach to data synthesis was executed.We identified 43 papers that met eligibility criteria. No existing review was found on this topic, and no HSG appraisal tool was identified. Over one third of the authors implicitly or explicitly identified the need for a high-quality tool aimed to systematically evaluate HSG and contribute to its development/reporting. We identified 30 concepts that may be relevant to the appraisal of HSG and were able to cluster them into three meaningful domains: process principles, content, and context principles.Our study showed the role that the quality criteria play in the development, appraisal, and reporting of HSG and demonstrated the link and resonance within and between the various concepts in the three domains.

Project description:BackgroundSignificant resources are required to write de novo clinical practice guidelines (CPGs). There are many freely-available CPGs internationally, for many health conditions. Developing countries rarely have the resources for de novo CPGs, and there could be efficiencies in using CPGs developed elsewhere. This paper outlines a novel process developed and tested in a resource-constrained country (South Africa) to synthesise findings from multiple international CPGs on allied health (AH) stroke rehabilitation.MethodsMethodologists, policy-makers, content experts and consumers collaborated to describe the pathway of an 'average' stroke patient through the South African public healthcare system and pose questions about best-practice stroke rehabilitation along this pathway. A comprehensive search identified international guidance documents published since January 2010. These were scanned for relevance to the South African AH stroke rehabilitation questions and critically appraised for methodological quality. Recommendations were extracted from guidance documents for each question. Strength of the body of evidence (SoBE) gradings underpinning recommendations were standardised, and composite recommendations were developed using qualitative synthesis. An algorithm was developed to guide assignment of overall SoBE gradings to composite recommendations.ResultsSixteen CPGs were identified, and all were included, as they answered different project questions differently. Methodological quality varied and was unrelated to currency. Seven clusters, outlining 20 composite recommendations were proposed (organise for best practice rehabilitation, operationalise strategies for best practice communication throughout the patient journey, admit to an acute hospital, refer to inpatient rehabilitation, action inpatient rehabilitation, discharge from inpatient rehabilitation and longer-term community-based rehabilitation).ConclusionThe methodological development process, tested by writing a South African AH stroke rehabilitation guideline from existing evidence sources, took 9 months. The process was efficient, collaborative, effective, rewarding and positive. Using the proposed methods, similar synthesis of existing evidence could be conducted in shorter time periods, in other resource-constrained countries, avoiding the need for expensive and time-consuming de novo CPG development.

Project description:ObjectivesContinuous renal replacement therapy (CRRT) is commonly employed in critically ill children and is known to affect antimicrobial pharmacokinetics. There is a lack of readily available, evidence-based antimicrobial dosing recommendations in pediatric CRRT. This study aims to quantify commonly used antimicrobial drugs in pediatric CRRT and identify gaps between contemporary literature-based dosing recommendations and those presented in a frequently used dosing reference.MethodsThe Pediatric Health Information System (PHIS) database was queried from July 1, 2018 through June 30, 2021 to identify admissions in which antimicrobials were billed on the same day as CRRT. Drugs of interest were selected if at least 10% of admission involved administration on at least one CRRT day, with additional clinically important antimicrobials selected by the authors. A comprehensive literature search was performed to identify antimicrobial pharmacokinetic (PK) studies in children for each selected drug. For identified articles, dosing recommendations were extracted and compared to those in a popular tertiary dosing reference (Lexi-Comp Online database). The level of agreement of the dosing recommendations was assessed.Results77 unique antimicrobial agents were identified amongst 812 admissions from 20 different PHIS hospitals. Fifteen antimicrobials were billed on the same day as CRRT in ≥10% of admissions, with 4 additional drugs deemed clinically relevant by the authors. Twenty PK studies were identified for these 19 drugs, and dosing recommendations were included in 8 (42.1%) of them. Seventeen agents (89.5%) had some type of CRRT-specific dosing guidance in Lexi-Comp, with only 1 directly based on a pediatric CRRT study. For the 8 agents with PK data available, Lexi-Comp recommendations matched primary literature dosing guidance in 3 (37.5%). Two (25%) lacked agreement between the Lexi-Comp and primary literature, and the remaining 3 (37.5%) had partial agreement with multiple dosing regimens suggested in the primary literature and at least one of these regimens recommended by Lexi-Comp.ConclusionSignificant gaps exist in the data supporting antimicrobial dosing recommendations for children receiving CRRT. Future studies should focus on antimicrobial dosing in pediatric CRRT, emphasizing provision of robust data from which dosing recommendations can be promptly incorporated into tertiary dosing references.

Dataset Information

Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group.

Background

Methods

Results

Conclusions

Publications

Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets