Dataset Information

Reliability and validity of the AGREE instrument used by physical therapists in assessment of clinical practice guidelines.

ABSTRACT:

Background

The AGREE instrument has been validated for evaluating Clinical Practice Guidelines (CPG) pertaining to medical care. This study evaluated the reliability and validity of physical therapists using the AGREE to assess quality of CPGs relevant to physical therapy practice.

Methods

A total of 69 physical therapists participated and were classified as generalists, specialist or researchers. Pairs of appraisers within each category evaluated independently, a set of 6 CPG selected at random from a pool of 55 CPGs.

Results

Reliability between pairs of appraisers indicated low to high reliability depending on the domain and number of appraisers (0.17-0.81 for single appraiser; 0.30-0.96 when score averaged across a pair of appraisers). The highest reliability was achieved for Rigour of Development, which exceeded ICC> 0.79, if scores from pairs of appraisers were pooled. Adding more than 3 appraisers did not consistently improve reliability. Appraiser type did not determine reliability scores. End-users, including study participants and a separate sample of 102 physical therapy students, found the AGREE useful to guide critical appraisal. The construct validity of the AGREE was supported in that expected differences on Rigour of Development domains were observed between expert panels versus those with no/uncertain expertise (differences of 10-21% p = 0.09-0.001). Factor analysis with varimax rotation, produced a 4-factor solution that was similar, although not in exact agreement with the AGREE Domains. Validity was also supported by the correlation observed (Kendall-tao = 0.69) between Overall Assessment and the Rigour of Development domain.

Conclusion

These findings suggest that the AGREE instrument is reliable and valid when used by physiotherapists to assess the quality of CPG pertaining to physical therapy health services.

SUBMITTER: MacDermid JC

PROVIDER: S-EPMC555572 | biostudies-literature | 2005 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Reliability and validity of the AGREE instrument used by physical therapists in assessment of clinical practice guidelines.

MacDermid Joy C JC Brooks Dina D Solway Sherra S Switzer-McIntyre Sharon S Brosseau Lucie L Graham Ian D ID

BMC health services research 20050302 1

<h4>Background</h4>The AGREE instrument has been validated for evaluating Clinical Practice Guidelines (CPG) pertaining to medical care. This study evaluated the reliability and validity of physical therapists using the AGREE to assess quality of CPGs relevant to physical therapy practice.<h4>Methods</h4>A total of 69 physical therapists participated and were classified as generalists, specialist or researchers. Pairs of appraisers within each category evaluated independently, a set of 6 CPG sel ...[more]

PMID: 15743522

Similar Datasets

Project description:BackgroundClinical practice guidelines (CPGs) are representative methods for promoting the standardization of healthcare and improvement of its quality. Few studies have investigated changes in the quality of CPGs published in a country over time. Our aim was to investigate changes in the quality of CPGs over time in the context of the available infrastructure for CPG development, public interest in healthcare quality, and healthcare providers' responses to this interest.MethodsAll CPGs pertaining to evidence-based medicine (EBM) issued between 2000 and 2014 in Japan (n = 373) were evaluated using the Japanese version of the Appraisal of Guidelines for Research and Evaluation (AGREE) I. Additionally, time trends in quality were analyzed. Using a cut-off point based on the publication year of CPG development literature, the evaluated CPGs were classified into those published until 2008 (pre-2008) and those published since 2009 (post-2008). Subsequently, we compared these groups in terms of 1) first edition CPGs and its second editions, and 2) patients' version of CPGs.ResultsScores on all six domains of AGREE I improved each year. A comparison of the first- and second-edition of CPGs (n = 64) showed that scores on all domains improved significantly after revision. Significant improvement was observed in three domains (#2 stakeholder involvement, #3 rigor of development, and #4 clarity of presentation) in the pre-2008 group and in all domains in the post-2008 group. The comparison between the pre- and post-2008 groups in terms of CPGs for patients showed that the score increased in only one domain (#1 scope and purpose).ConclusionsThe number of published CPGs has been increasing and the quality of CPGs, as assessed using the AGREE I instrument, has been improving. These changes seem to be influenced by improvements in social infrastructure, such as the publication of CPG development procedures, availability of CPG preparation methodology training, and increase in CPG-related skills.

Project description:BackgroundNumerous clinical practice guidelines (CPGs) are published to guide management of osteoporosis. Little is known about their quality or how recommendations have changed over time.ObjectiveTo systematically assess the quality and content of the guidelines on screening for osteoporosis, using the Appraisal of Guidelines for Research and Evaluation (AGREE II) tool, and the Institute of Medicine (IOM) standards for trustworthy guidelines.MethodsWe conducted a systematic search for osteoporosis CPGs published between 2002-2016, using multiple databases and guideline websites. Two reviewers appraised the quality of eligible CPGs using the AGREE II. High quality CPGs were considered if they scored ≥ 60 in four or more domains including the domain for rigor of development. Non-parametric tests were used to test for the change of quality over time. One reviewer assessed the guidelines with IOM standards. We summarized the different evidence grading systems and extracted and compared the recommendations.ResultsA total of 33 CPGs were identified. The mean scores for AGREE II differed by domain (range: 42% to 71%). CPGs scored higher on domains for clarity of presentation, scope and purpose, and rigor of development. CPGs scored lower on domains for stakeholder involvement, editorial independence and applicability. Assessment of CPGs by IOM standards showed that CPGs scored better on standards for systematic review, establishing evidence foundation and rating strength of recommendation, articulation of recommendation, and establishing transparency. While scored lower on standards for updating, external review, and the development group composition. There was no difference in AGREE II and IOM defined guidelines' quality before and after the introduction of the two tools (P values >0.05). The IOM identified four more guidelines as high quality compared to the AGREE II. Examining these additional guidelines indicated that the two tools may give conflicting results especially for the rigor of development domain. Recommendations in certain areas showed substantial differences between guidelines.ConclusionOsteoporosis screening CPGs are of variable quality, and their recommendations often differ. Guideline quality as measured by AGREE II and IOM standards has not improved overtime. Guideline developers should work together to improve the quality and consistency of recommendations to improve the likelihood that their guidelines will be used in practice.

Dataset Information

Reliability and validity of the AGREE instrument used by physical therapists in assessment of clinical practice guidelines.

Background

Methods

Results

Conclusion

Publications

Reliability and validity of the AGREE instrument used by physical therapists in assessment of clinical practice guidelines.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets