Dataset Information

Decision-Making about Healthcare Related Tests and Diagnostic Strategies: User Testing of GRADE Evidence Tables.

ABSTRACT:

Objective

To develop guidance on what information to include and how to present it in tables summarizing the evidence from systematic reviews of test accuracy following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.

Methods

To design and refine the evidence tables, we used an iterative process based on the analysis of data from four rounds of discussions, feedback and user testing. During the final round, we conducted one-on-one user testing with target end users. We presented a number of alternative formats of evidence tables to participants and obtained information about users' understanding and preferences.

Results

More than 150 users participated in initial discussions and provided their formal and informal feedback. 20 users completed one-on-one user testing interviews. Almost all participants preferred summarizing the results of systematic reviews of test accuracy in tabular format rather than plain text. Users generally preferred less complex tables but found presenting sensitivity and specificity estimates only as too simplistic. Users found the presentation of test accuracy for several values of prevalence initially confusing but modifying table layout and adding sample clinical scenarios for each prevalence reduced this confusion. Providing information about clinical consequences of testing result was viewed as not feasible for authors of systematic reviews.

Conclusion

We present the current formats for tables presenting test accuracy following the GRADE approach. These tables can be developed using GRADEpro guidelines development tool (www.guidelinedevelopment.org or www.gradepro.org) and are being further developed into electronic interactive tables that will suit the needs of different end users. The formatting of these tables, and how they influence result interpretation and decision-making will be further evaluated in a randomized trial.

SUBMITTER: Mustafa RA

PROVIDER: S-EPMC4608675 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Decision-Making about Healthcare Related Tests and Diagnostic Strategies: User Testing of GRADE Evidence Tables.

Mustafa Reem A RA Wiercioch Wojtek W Santesso Nancy N Cheung Adrienne A Prediger Barbara B Baldeh Tejan T Carrasco-Labra Alonso A Brignardello-Petersen Romina R Neumann Ignacio I Bossuyt Patrick P Garg Amit X AX Lelgemann Monika M Bühler Diedrich D Brozek Jan J Schünemann Holger J HJ

PloS one 20151016 10

<h4>Objective</h4>To develop guidance on what information to include and how to present it in tables summarizing the evidence from systematic reviews of test accuracy following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.<h4>Methods</h4>To design and refine the evidence tables, we used an iterative process based on the analysis of data from four rounds of discussions, feedback and user testing. During the final round, we conducted one-on-one user testin ...[more]

PMID: 26474310

Similar Datasets

Project description:BackgroundAccurate diagnosis is a fundamental aspect of appropriate healthcare. However, clinicians need guidance when implementing diagnostic tests given the number of tests available and resource constraints in healthcare. Practitioners of health often feel compelled to implement recommendations in guidelines, including recommendations about the use of diagnostic tests. However, the understanding about diagnostic tests by guideline panels and the methodology for developing recommendations is far from completely explored. Therefore, we evaluated the factors that guideline developers and users need to consider for the development of implementable recommendations about diagnostic tests.MethodsUsing a critical analysis of the process, we present the results of a case study using the Grading of Recommendations Applicability, Development and Evaluation (GRADE) approach to develop a clinical practice guideline for the diagnosis of Cow Milk Allergy with the World Allergy Organization.ResultsTo ensure that guideline panels can develop informed recommendations about diagnostic tests, it appears that more emphasis needs to be placed on group processes, including question formulation, defining patient-important outcomes for diagnostic tests, and summarizing evidence. Explicit consideration of concepts of diagnosis from evidence-based medicine, such as pre-test probability and treatment threshold, is required to facilitate the work of a guideline panel and to formulate implementable recommendations.DiscussionThis case study provides useful guidance for guideline developers and clinicians about what they ought to demand from clinical practice guidelines to facilitate implementation and strengthen confidence in recommendations about diagnostic tests. Applying a structured framework like the GRADE approach with its requirement for transparency in the description of the evidence and factors that influence recommendations facilitates laying out the process and decision factors that are required for the development, interpretation, and implementation of recommendations about diagnostic tests.

Project description:BackgroundResearch shows that people with stable angina need decision support when considering elective treatments. Initial treatment is with medicines but patients may gain further benefit with invasive percutaneous coronary intervention (PCI). Choosing between these treatments can be challenging for patients because both confer similar benefits but have different risks. Patient decision aids (PtDAs) are evidence-based interventions that support shared decision-making (SDM) when making healthcare decisions. This study aimed to develop and user-test a digital patient decision aid (CONNECT) to facilitate SDM for people with stable angina considering invasive treatment with elective PCI.MethodsA multi-phase study was conducted to develop and test CONNECT (COroNary aNgioplasty dECision Tool) using approaches recommended by the International Patient Decision Aid Standards Collaboration: (i) Steering Group assembled, (ii) review of clinical guidance, (iii) co-design workshops with patients and cardiology health professionals, (iv) first prototype developed and 'alpha' tested (semi-structured cognitive interviews and 12-item acceptability questionnaire) with patients, cardiologists and cardiac nurses, recruited from two hospitals in Northern England, and (v) final PtDA refined following iterative user-feedback. Quantitative data were analysed descriptively and qualitative data from the interviews analysed using deductive content analysis.ResultsCONNECT was developed and user-tested with 34 patients and 29 cardiology health professionals. Findings showed that CONNECT was generally acceptable, usable, comprehensible, and desirable. Participants suggested that CONNECT had the potential to improve care quality by personalising consultations and facilitating SDM and informed consent. Patient safety may be improved as CONNECT includes questions about symptom burden which can identify asymptomatic patients unlikely to benefit from PCI, as well as those who may need to be fast tracked because of worsening symptoms.ConclusionsCONNECT is the first digital PtDA for people with stable angina considering elective PCI, developed in the UK using recommended processes and fulfilling international quality criteria. CONNECT shows promise as an approach to facilitate SDM and should be evaluated in a clinical trial. Further work is required to standardise the provision of probabilistic risk information for people considering elective PCI and to understand how CONNECT can be accessible to underserved communities.

Project description:Design and objectivesWe performed a systematic review of studies evaluating healthcare provider (HCP) trainings in shared decision-making (SDM) to analyse their evaluation strategies.Setting and participantsHCP trainings in SDM from all healthcare settings.MethodsWe searched scientific databases (Medline, PsycInfo, CINAHL), performed reference and citation tracking, contacted experts in the field and scanned the Canadian inventory of SDM training programmes for healthcare professionals. We included articles reporting data of summative evaluations of HCP trainings in SDM. Two reviewers screened records, assessed full-text articles, performed data extraction and assessed study quality with the integrated quality criteria for review of multiple study designs (ICROMS) tool. Analysis of evaluation strategies included data source use, use of unpublished or published measures and coverage of Kirkpatrick's evaluation levels. An evaluation framework based on Kirkpatrick's evaluation levels and the Quadruple Aim framework was used to categorise identified evaluation outcomes.ResultsOut of 7234 records, we included 41 articles reporting on 30 studies: cluster-randomised (n=8) and randomised (n=9) controlled trials, controlled (n=1) and non-controlled (n=7) before-after studies, mixed-methods (n=1), qualitative (n=1) and post-test (n=3) studies. Most studies were conducted in the USA (n=9), Germany (n=8) or Canada (n=7) and evaluated physician trainings (n=25). Eleven articles met ICROMS quality criteria. Almost all studies (n=27) employed HCP-reported outcomes for training evaluation and most (n=19) additionally used patient-reported (n=12), observer-rated (n=10), standardised patient-reported (n=2) outcomes or training process and healthcare data (n=10). Most studies employed a mix of unpublished and published measures (n=17) and covered two (n=12) or three (n=10) Kirkpatrick's levels. Identified evaluation outcomes covered all categories of the proposed framework.ConclusionsStrategies to evaluate HCP trainings in SDM varied largely. The proposed evaluation framework maybe useful to structure future evaluation studies, but international agreement on a core set of outcomes is needed to improve evidence.Prospero registration numberCRD42016041623.

Dataset Information

Decision-Making about Healthcare Related Tests and Diagnostic Strategies: User Testing of GRADE Evidence Tables.

Objective

Methods

Results

Conclusion

Publications

Decision-Making about Healthcare Related Tests and Diagnostic Strategies: User Testing of GRADE Evidence Tables.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets