Dataset Information

Guidelines for guideline developers: a systematic review of grading systems for medical tests.

ABSTRACT:

Background

A variety of systems have been developed to grade evidence and develop recommendations based on the available evidence. However, development of guidelines for medical tests is especially challenging given the typical indirectness of the evidence; direct evidence of the effects of testing on patient important outcomes is usually absent. We compared grading systems for medical tests on how they use evidence in guideline development.

Methods

We used a systematic strategy to look for grading systems specific to medical tests in PubMed, professional guideline websites, via personal correspondence, and handsearching back references of key articles. Using the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument as a starting point, we defined two sets of characteristics to describe these systems: methodological and process ones. Methodological characteristics are features relating to how evidence is gathered, appraised, and used in recommendations. Process characteristics are those relating to the guideline development process. Data were extracted in duplicate and differences resolved through discussion.

Results

Twelve grading systems could be included. All varied in the degree to which methodological and process characteristics were addressed. Having a clinical scenario, identifying the care pathway and/or developing an analytical framework, having explicit criteria for appraising and linking indirect evidence, and having explicit methodologies for translating evidence into recommendations were least frequently addressed. Five systems at most addressed these, to varying degrees of explicitness and completeness. Process wise, features most frequently addressed included involvement of relevant professional groups (8/12), external peer review of completed guidelines (9/12), and recommendations on methods for dissemination (8/12). Characteristics least often addressed were whether the system was piloted (3/12) and funder information (3/12).

Conclusions

Five systems for grading evidence about medical tests in guideline development addressed to differing degrees of explicitness the need for and appraisal of different bodies of evidence, the linking of such evidence, and its translation into recommendations. At present, no one system addressed the full complexity of gathering, assessing and linking different bodies of evidence.

SUBMITTER: Gopalakrishna G

PROVIDER: S-EPMC3716938 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BACKGROUND:Quality indicators (QIs) are used in assessing the quality of healthcare. Evidence-based clinical practice guidelines (CPGs) are relevant sources for generating QIs. In this context, QIs are important tools to assess the implementation of guideline recommendations. However, the methodological approaches to guideline-based QI development vary considerably. In Germany, the guideline classification scheme of the AWMF (German Association of the Scientific Medical Societies) differentiates between S1-, S2k-, S2e-, and S3-CPGs depending on the methodological approach. Thus, S3-CPGs are consensus- and evidence-based CPGs and have the highest methodological standard in Germany. An analysis of the status quo of reported QIs in S3-CPGs found 35 current S3-CPGs, which report 372 different QIs. Currently, there is no gold standard for the development of guideline-based QIs. To our knowledge, no studies have investigated to what extent guideline-based QIs from different CPGs that are related to the same topic are consistent. The objective of this study is to compare guideline-based QIs and their underlying methodological approaches of German S3-CPGs with those of topic-related international CPGs. METHODS:Based on the previous identified German S3-CPGs (n?=?35), which report quality indicators, we will conduct systematic searches in the guidelines databases of G-I-N (Guidelines International Network) and NGC (National Guideline Clearinghouse) to identify international CPGs matching the topics of the S3-CPGs. If necessary, we will search additionally the websites of the particular CPG providers for separate documents with regard to QIs. We will include evidence-based CPGs which report QIs. Reported QIs as well as methods of development and the rationale for QIs will be extracted and compared with those of the S3-CPGs. DISCUSSION:This study will be part of the project "Systematic analysis of the translation of guideline recommendations into quality indicators and development of an evidence- and consensus-based standard," supported by the German Research Association (DFG). The results of this analysis will feed into a subsequent qualitative study, which will consist of structured interviews with developers of international CPGs. Further, the results will be considered in a consensus study on standards of the translation of guideline recommendations into quality indicators in Germany.

Project description:BackgroundElectronic guideline-based decision support systems have been suggested to successfully deliver the knowledge embedded in clinical practice guidelines. A number of studies have already shown positive findings for decision support systems such as drug-dosing systems and computer-generated reminder systems for preventive care services.MethodsA systematic literature search (1990 to December 2008) of the English literature indexed in the Medline database, Embase, the Cochrane Central Register of Controlled Trials, and CRD (DARE, HTA and NHS EED databases) was conducted to identify evaluation studies of electronic multi-step guideline implementation systems in ambulatory care settings. Important inclusion criterions were the multidimensionality of the guideline (the guideline needed to consist of several aspects or steps) and real-time interaction with the system during consultation. Clinical decision support systems such as one-time reminders for preventive care for which positive findings were shown in earlier reviews were excluded. Two comparisons were considered: electronic multidimensional guidelines versus usual care (comparison one) and electronic multidimensional guidelines versus other guideline implementation methods (comparison two).ResultsTwenty-seven publications were selected for analysis in this systematic review. Most designs were cluster randomized controlled trials investigating process outcomes more than patient outcomes. With success defined as at least 50% of the outcome variables being significant, none of the studies were successful in improving patient outcomes. Only seven of seventeen studies that investigated process outcomes showed improvements in process of care variables compared with the usual care group (comparison one). No incremental effect of the electronic implementation over the distribution of paper versions of the guideline was found, neither for the patient outcomes nor for the process outcomes (comparison two).ConclusionsThere is little evidence at the moment for the effectiveness of an increasingly used and commercialized instrument such as electronic multidimensional guidelines. After more than a decade of development of numerous electronic systems, research on the most effective implementation strategy for this kind of guideline-based decision support systems is still lacking. This conclusion implies a considerable risk towards inappropriate investments in ineffective implementation interventions and in suboptimal care.

Project description:The emergency medical services (EMS) cover initiatives and services established to provide essential medical assistance in situations of acute illness. Triage-methods for systematic prioritizing of patients according to how urgent patients need care, including triage of requests of acute medical treatment, are adopted in hospitals as well as in the pre-hospital settings. This systematic review searched to identify available research on the effects of validated triage systems for use in the pre-hospital EMS on health outcomes, patient safety, patient satisfaction, user-friendliness, resource use, goal achievement, and the quality on the information exchange between the different settings of the EMS (for example the quality of documentation). The specific research questions were: 1) are pre-hospital triage systems effective, 2) is one triage system more effective than others, and 3) is it effective to use the same triage system in two or more settings of the EMS-chain? We conducted a systematic literature search in nine databases up to June 2012. We searched for systematic reviews (SRs), randomized controlled trials (RCTs), non-randomized controlled trials (non-RCTs), controlled before and after studies (CBAs) and interrupted time series analyses (ITSs). Two persons independently reviewed titles and abstracts, and the same persons read all possibly relevant full text articles and rated the methodological quality where relevant. The literature search identified 11011 unique references. A total of 120 publications were read in full text. None of the identified articles fulfilled our inclusion criteria, thus our question on the effects of pre-hospital triage systems, if one system is better than other systems, and the question on effects of using the same triage system in two or more settings of the EMS, remain unanswered. We conclude that there is an evidence gap regarding the effects of pre-hospital triage systems and the effects of using the same triage system in two or more settings of the EMS. The finding does not mean that pre-hospital triage systems are ineffective, but that we lack knowledge about potential effects. When introducing a new assessment tool in the EMS, it is timely to conduct well-planned studies aimed to assess the effect.