Dataset Information

Strategies to assess the validity of recommendations: a study protocol.

ABSTRACT: BACKGROUND: Clinical practice guidelines (CPGs) become quickly outdated and require a periodic reassessment of evidence research to maintain their validity. However, there is little research about this topic. Our project will provide evidence for some of the most pressing questions in this field: 1) what is the average time for recommendations to become out of date?; 2) what is the comparative performance of two restricted search strategies to evaluate the need to update recommendations?; and 3) what is the feasibility of a more regular monitoring and updating strategy compared to usual practice?. In this protocol we will focus on questions one and two. METHODS: The CPG Development Programme of the Spanish Ministry of Health developed 14 CPGs between 2008 and 2009. We will stratify guidelines by topic and by publication year, and include one CPG by strata.We will develop a strategy to assess the validity of CPG recommendations, which includes a baseline survey of clinical experts, an update of the original exhaustive literature searches, the identification of key references (reference that trigger a potential recommendation update), and the assessment of the potential changes in each recommendation.We will run two alternative search strategies to efficiently identify important new evidence: 1) PLUS search based in McMaster Premium LiteratUre Service (PLUS) database; and 2) a Restrictive Search (ReSe) based on the least number of MeSH terms and free text words needed to locate all the references of each original recommendation.We will perform a survival analysis of recommendations using the Kaplan-Meier method and we will use the log-rank test to analyse differences between survival curves according to the topic, the purpose, the strength of recommendations and the turnover. We will retrieve key references from the exhaustive search and evaluate their presence in the PLUS and ReSe search results. DISCUSSION: Our project, using a highly structured and transparent methodology, will provide guidance of when recommendations are likely to be at risk of being out of date. We will also assess two novel restrictive search strategies which could reduce the workload without compromising rigour when CPGs developers check for the need of updating.

SUBMITTER: Martinez Garcia L

PROVIDER: S-EPMC3765147 | biostudies-other | 2013

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Strategies to assess the validity of recommendations: a study protocol.

Martínez García Laura L Sanabria Andrea Juliana AJ Araya Ignacio I Lawson Jennifer J Haynes R Brian RB Rigau David D Solà Ivan I Díaz Del Campo Petra P Estrada Maria Dolors MD Etxeandia-Ikobaltzeta Itziar I García Álvarez Elvira E Gracia Javier J Kotzeva Anna A Louro-González Arturo A Salcedo-Fernandez Flavia F Trujillo-Martín Maria Mar MM Alonso-Coello Pablo P

Implementation science : IS 20130822

<h4>Background</h4>Clinical practice guidelines (CPGs) become quickly outdated and require a periodic reassessment of evidence research to maintain their validity. However, there is little research about this topic. Our project will provide evidence for some of the most pressing questions in this field: 1) what is the average time for recommendations to become out of date?; 2) what is the comparative performance of two restricted search strategies to evaluate the need to update recommendations?; ...[more]

PMID: 23967896

Similar Datasets

Project description:BackgroundMany junior doctors must prepare to manage acutely ill patients in the emergency department. The setting is often stressful, and urgent treatment decisions are needed. Overlooking symptoms and making wrong choices may lead to substantial patient morbidity or death, and it is essential to ensure that junior doctors are competent. Virtual reality (VR) software can provide standardized and unbiased assessment, but solid validity evidence is necessary before implementation.ObjectiveThis study aimed to gather validity evidence for using 360-degree VR videos with integrated multiple-choice questions (MCQs) to assess emergency medicine skills.MethodsFive full-scale emergency medicine scenarios were recorded with a 360-degree video camera, and MCQs were integrated into the scenarios to be played in a head-mounted display. We invited 3 groups of medical students with different experience levels to participate: first- to third-year medical students (novice group), last-year medical students without emergency medicine training (intermediate group), and last-year medical students with completed emergency medicine training (experienced group). Each participant's total test score was calculated based on the number of correct MCQ answers (maximum score of 28), and the groups' mean scores were compared. The participants rated their experienced presence in emergency scenarios using the Igroup Presence Questionnaire (IPQ) and their cognitive workload with the National Aeronautics and Space Administration Task Load Index (NASA-TLX).ResultsWe included 61 medical students from December 2020 to December 2021. The experienced group had significantly higher mean scores than the intermediate group (23 vs 20; P=.04), and the intermediate group had significantly higher scores than the novice group (20 vs 14; P<.001). The contrasting groups' standard-setting method established a pass-or-fail score of 19 points (68% of the maximum possible score of 28). Interscenario reliability was high, with a Cronbach α of 0.82. The participants experienced the VR scenarios with a high degree of presence with an IPQ score of 5.83 (on a scale from 1-7), and the task was shown to be mentally demanding with a NASA-TLX score of 13.30 (on a scale from 1-21).ConclusionsThis study provides validity evidence to support using 360-degree VR scenarios to assess emergency medicine skills. The students evaluated the VR experience as mentally demanding with a high degree of presence, suggesting that VR is a promising new technology for emergency medicine skills assessment.

Project description:The uptake of Clinical Practice Guideline (CPG) recommendations that improve outcomes in heart failure (HF) remains suboptimal. We will conduct a systematic review to identify implementation strategies that improve physician adherence to class I recommendations, those with clear evidence that benefits outweigh the risks. We will use American, Canadian and European HF guidelines as our reference.We will conduct a literature search in the databases of MEDLINE, EMBASE, HEALTHSTAR, CINAHL, Cochrane Library, Campbell Collaboration, Joanna Briggs Institute Evidence Based Practice, Centre for Reviews and Dissemination and Evidence Based Practice Centres. We will include prospective studies evaluating implementation interventions aimed at improving uptake of class I CPG recommendations in HF. We will extract data in duplicate. We will classify interventions according to their level of application (ie, provider, organisation, systems level) and common underlying characteristics (eg, education, decision-support, financial incentives) using the Cochrane Effective Practice and Organisation of Care Taxonomy. We will assess the impact of the intervention on adherence to the CPGs. Outcomes will include proportion of eligible patients who were: prescribed a CPG-recommended pharmacological treatment; referred for device consideration; provided self-care education at discharge; and provided left ventricular function assessment. We will include clinical outcomes such as hospitalisations, readmissions and mortality, if data is available. We will identify the common elements of successful and failing interventions, and examine the context in which they were applied, using the Process Redesign contextual framework. We will synthesise the results narratively and, if appropriate, will pool results for meta-analysis.In this review, we will assess the impact of implementation strategies and contextual factors on physician adherence to HF CPGs. We will explore why some interventions may succeed in one setting and fail in another. We will disseminate our findings through briefing reports, publications and presentations.CRD42015017155.

Project description:BackgroundHealth care practice based on research evidence requires that evidence is synthesised, and that recommendations based on this evidence are implemented. It also requires an intermediate step: translating synthesised evidence into practice recommendations. There is considerable literature on evidence synthesis and implementation, but little on how guideline development groups (GDGs) produce recommendations. This is a complex process, with many influences on communication and decision-making, e.g., the quality of evidence, methods of presentation, practical/resource constraints, individual values, professional and scientific interests, social and psychological processes. To make this process more transparent and potentially effective, we need to understand these influences. Psychological theories of decision-making and social influence provide a framework for this understanding.ObjectivesThis study aims to investigate the processes by which GDGs formulate recommendations, drawing on psychological theories of decision-making and social influence. The findings will potentially inform the further evolution of GDG methods, such as choice of members and procedures for presenting evidence, conducting discussion and formulating recommendations.MethodsLongitudinal observation of the meetings of three National Institute of Health and Clinical Excellence (NICE) GDGs, one from each of acute, mental health and public health, will be tape recorded and transcribed. Interviews with a sample of GDG members at the beginning, middle, and end of the GDG's work will be recorded and transcribed. Site documents including relevant e-mail interchanges, GDG meeting minutes, and stakeholders' responses to the drafts of the recommendations will be collected. Data will be selected for analysis if they refer to either evidence or recommendations; the focus is on "hot spots", e.g., dilemmas, conflicts, and uncertainty. Data will be analysed thematically and by content analysis, drawing on psychological theories of decision-making and social influence.

Project description:BackgroundIdentifying feasible and effective implementation strategies that are contextually appropriate is a challenge for researchers and implementers, exacerbated by the lack of conceptual clarity surrounding terms and definitions for implementation strategies, as well as a literature that provides imperfect guidance regarding how one might select strategies for a given healthcare quality improvement effort. In this study, we will engage an Expert Panel comprising implementation scientists and mental health clinical managers to: establish consensus on a common nomenclature for implementation strategy terms, definitions and categories; and develop recommendations to enhance the match between implementation strategies selected to facilitate the use of evidence-based programs and the context of certain service settings, in this case the U.S. Department of Veterans Affairs (VA) mental health services.Methods/designThis study will use purposive sampling to recruit an Expert Panel comprising implementation science experts and VA mental health clinical managers. A novel, four-stage sequential mixed methods design will be employed. During Stage 1, the Expert Panel will participate in a modified Delphi process in which a published taxonomy of implementation strategies will be used to establish consensus on terms and definitions for implementation strategies. In Stage 2, the panelists will complete a concept mapping task, which will yield conceptually distinct categories of implementation strategies as well as ratings of the feasibility and effectiveness of each strategy. Utilizing the common nomenclature developed in Stages 1 and 2, panelists will complete an innovative menu-based choice task in Stage 3 that involves matching implementation strategies to hypothetical implementation scenarios with varying contexts. This allows for quantitative characterizations of the relative necessity of each implementation strategy for a given scenario. In Stage 4, a live web-based facilitated expert recommendation process will be employed to establish expert recommendations about which implementations strategies are essential for each phase of implementation in each scenario.DiscussionUsing a novel method of selecting implementation strategies for use within specific contexts, this study contributes to our understanding of implementation science and practice by sharpening conceptual distinctions among a comprehensive collection of implementation strategies.

Dataset Information

Strategies to assess the validity of recommendations: a study protocol.

Publications

Strategies to assess the validity of recommendations: a study protocol.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets