Dataset Information

An exploration of how developers use qualitative evidence: content analysis and critical appraisal of guidelines.

ABSTRACT: BACKGROUND:Clinical practice guidelines have become increasingly widely used to guide quality improvement of clinical practice. Qualitative research may be a useful way to improve the quality and implementation of guidelines. The methodology for qualitative evidence used in guidelines development is worthy of further research. METHODS:A comprehensive search was made of WHO, NICE, SIGN, NGC, RNAO, PubMed, Embase, Web of Science, CNKI, Wanfang, CBM, and VIP from January 1, 2011 to February 25, 2020. Guidelines which met IOM criteria and were focused on clinical questions using qualitative research or qualitative evidence, were included. Four authors extracted significant information and entered this onto data extraction forms. The Appraisal of Guidelines for Research and Evaluation (AGREE II) tool was used to evaluate the guidelines' quality. The data were analyzed using SPSS version 17.0 and R version 3.3.2. RESULTS:Sixty four guidelines were identified. The overall quality of the guidelines was high (almost over 60%). Domain 1 (Scope and Purpose) was ranked the highest with a median score of 83% (IQ 78-83). Domain 2 (Stakeholder involvement) and Domain 5 (Applicability) were ranked the lowest with median scores of 67% (IQ 67-78) and 67% (IQ 63-73) respectively. 20% guidelines used qualitative research to identify clinical questions. 86% guidelines used qualitative evidence to support recommendations (mainly based on primary studies, a few on qualitative evidence synthesis). 19% guidelines applied qualitative evidence when considering facilitators and barriers to recommendations' implementation. 52% guideline developers evaluated the quality of the primary qualitative research study using the CASP tool or NICE checklist for qualitative studies. No guidelines evaluated the quality of qualitative evidence synthesis to formulate recommendations. 17% guidelines presented the level of qualitative research using the grade criteria of evidence and recommendation in different forms such as I, III, IV, very low. 28% guidelines described the grades of the recommendations supported by qualitative and quantitative evidence. No guidelines described the grade of recommendations only supported by qualitative evidence. CONCLUSIONS:The majority of the included guidelines were high-quality. Qualitative evidence was mainly used to identify clinical questions, support recommendations, and consider facilitators and barriers to implementation of recommendations'. However, more attention needs to be paid to the methodology. For example, no experts proficient in qualitative research were involved in guideline development groups, no assessment of the quality of qualitative evidence synthesis was included and there was lack of details reported on the level of qualitative evidence or grade of recommendations.

SUBMITTER: Wang YY

PROVIDER: S-EPMC7302150 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An exploration of how developers use qualitative evidence: content analysis and critical appraisal of guidelines.

Wang Yun-Yun YY Liang Dan-Dan DD Lu Cui C Shi Yue-Xian YX Zhang Jing J Cao Yue Y Fang Cheng C Huang Di D Jin Ying-Hui YH

BMC medical research methodology 20200617 1

<h4>Background</h4>Clinical practice guidelines have become increasingly widely used to guide quality improvement of clinical practice. Qualitative research may be a useful way to improve the quality and implementation of guidelines. The methodology for qualitative evidence used in guidelines development is worthy of further research.<h4>Methods</h4>A comprehensive search was made of WHO, NICE, SIGN, NGC, RNAO, PubMed, Embase, Web of Science, CNKI, Wanfang, CBM, and VIP from January 1, 2011 to F ...[more]

PMID: 32552780

Similar Datasets

Project description:Background Tobacco use is a leading preventable cause of morbidity and mortality globally. Clinical practice guidelines for the treatment of tobacco use dependence are of varied scope and quality, making it challenging for users to select and apply recommendations. Objective The study objective is to identify and critically appraise the quality of existing clinical practice guidelines for tobacco cessation. Setting The study occurred between collaborative academic institutions located in Qatar and New Zealand. Methods A systematic literature search was performed for the period 2006-2018 through the following databases: PubMed, EMBASE, CINAHL, ISI Web of Science, Scopus, National Guideline Clearing House, Campbell Library, Health System Evidence, Joanna Briggs Institute Evidence-Based Practice Database, Academic Search Complete, ProQuest, PROSPERO, and Google Scholar. Relevant professional societies' and health agencies' websites were also searched. Two reviewers independently extracted and assessed guidelines' quality using Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument. Main outcome measure Standardized domain scores according to the AGREE II instrument. Results 7741 hits were identified. After removing duplicates and screening, 24 guidelines were included. Highest guideline quality was for National Institute for Health and Care Excellence (NICE) guideline with an overall ranking score of 87.56% and least quality was for Japanese Circulation Society Joint Working Group with an overall score of 29.34%. Domain 4 of AGREE II (clarity of presentation) had the highest average quality score (70.95%), while the lowest average quality scores were for Domain 2 (Rigour of Development) (50.21%) and Domain 5 (Applicability) (45.05%). Conclusion Seven guidelines were judged to be of high quality (overall score of ≥ 70%). Future guidelines for tobacco dependence treatment should use rigorous methods of development and provide applicable recommendations.

Project description:BackgroundThis paper is part of a broader investigation into the ways in which health and social care guideline producers are using qualitative evidence syntheses (QESs) alongside more established methods of guideline development such as systematic reviews and meta-analyses of quantitative data. This study is a content analysis of QESs produced over a 5-year period by a leading provider of guidelines for the National Health Service in the UK (the National Institute for Health and Care Excellence) to explore how closely they match a reporting framework for QES.MethodsGuidelines published or updated between Jan 2015 and Dec 2019 were identified via searches of the National Institute for Health and Care excellence (NICE) website. These guidelines were searched to identify any QES conducted during the development of the guideline. Data relating to the compliance of these syntheses against a reporting framework for QES (ENTREQ) were extracted and compiled, and descriptive statistics used to provide an analysis of the of QES conduct, reporting and use by this major international guideline producer.ResultsQES contributed, in part, to 54 out of a total of 192 guidelines over the five-year period. Although methods for producing and reporting QES have changed substantially over the past decade, this study found that there has been little change in the number or quality of NICE QESs over time. The largest predictor of quality was the centre or team which undertook the synthesis. Analysis indicated that elements of review methods which were similar to those used in quantitative systematic reviews tended to be carried out well and mostly matched the criteria in the reporting framework, but review methods which were more specific to a QES tended to be carried out less well, with fewer examples of criteria in the reporting framework being achieved.ConclusionThe study suggests that use, conduct and reporting of optimal QES methods requires development, as over time the quality of reporting of QES both overall, and by specific centres, has not improved in spite of clearer reporting frameworks and important methodological developments. Further staff training in QES methods may be helpful for reviewers who are more familiar with conventional forms of systematic review if the highest standards of QES are to be achieved. There seems potential for greater use of evidence from qualitative research during guideline development.

Project description:BackgroundConsumers of research (researchers, administrators, educators and clinicians) frequently use standard critical appraisal tools to evaluate the quality of published research reports. However, there is no consensus regarding the most appropriate critical appraisal tool for allied health research. We summarized the content, intent, construction and psychometric properties of published, currently available critical appraisal tools to identify common elements and their relevance to allied health research.MethodsA systematic review was undertaken of 121 published critical appraisal tools sourced from 108 papers located on electronic databases and the Internet. The tools were classified according to the study design for which they were intended. Their items were then classified into one of 12 criteria based on their intent. Commonly occurring items were identified. The empirical basis for construction of the tool, the method by which overall quality of the study was established, the psychometric properties of the critical appraisal tools and whether guidelines were provided for their use were also recorded.ResultsEighty-seven percent of critical appraisal tools were specific to a research design, with most tools having been developed for experimental studies. There was considerable variability in items contained in the critical appraisal tools. Twelve percent of available tools were developed using specified empirical research. Forty-nine percent of the critical appraisal tools summarized the quality appraisal into a numeric summary score. Few critical appraisal tools had documented evidence of validity of their items, or reliability of use. Guidelines regarding administration of the tools were provided in 43% of cases.ConclusionsThere was considerable variability in intent, components, construction and psychometric properties of published critical appraisal tools for research reports. There is no "gold standard' critical appraisal tool for any study design, nor is there any widely accepted generic tool that can be applied equally well across study types. No tool was specific to allied health research requirements. Thus interpretation of critical appraisal of research reports currently needs to be considered in light of the properties and intent of the critical appraisal tool chosen for the task.

Project description:Background: Many organisations in Australia undertake systematic reviews to inform development of evidence-based guidelines or would like to do so. However, the substantial resources required to produce systematic reviews limit the feasibility of evidence-based approaches to guideline development. We are working with Australian guideline developers to design, build and test systems that make creating evidence-based guidelines easier and more efficient. Methods: To understand the evidence needs of guideline developers and to inform the development of potential tools and services, we conducted 16 semi-structured interviews with Australian guideline developers. Developers were involved in different types of guidelines, represented both new and established guideline groups, and had access to widely different levels of resources. Results: All guideline developers recognised the importance of having access to timely evidence to support their processes, but were frequently overwhelmed by the scale of this task. Groups developing new guidelines often underestimated the time, expertise and work involved in completing searching and screening. Many were grappling with the challenge of updating and were keen to explore alternatives to the blanket updating of the full guideline. Horizon-scanning and evidence signalling were seen as providing more pragmatic approaches to updating, although some were wary of challenges posed by receiving evidence on a too-frequent basis. Respondents were aware that new technologies, such as machine learning, offered potentially large time and resource savings. Conclusions: As well as the constant challenge of managing financial constraints, Australian guideline developers seeking to develop clinical guidelines face several critical challenges. These include acquiring appropriate methodological expertise, investing in information technology, coping with the proliferation of research output, feasible publication and dissemination options, and keeping guidance up to date.

Dataset Information

An exploration of how developers use qualitative evidence: content analysis and critical appraisal of guidelines.

Publications

An exploration of how developers use qualitative evidence: content analysis and critical appraisal of guidelines.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets