Dataset Information

Developing a questionnaire to evaluate an automated audit & feedback intervention: a Rand-modified Delphi method.

ABSTRACT:

Background

Audit and feedback (A&F) is a widely used implementation strategy to evaluate and improve medical practice. The optimal design of an A&F system is uncertain and structured process evaluations are currently lacking. This study aimed to develop and validate a questionnaire to evaluate the use of automated A&F systems.

Methods

Based on the Clinical Performance Feedback Intervention Theory (CP-FIT) and the REFLECT-52 (REassessing audit & Feedback interventions: a tooL for Evaluating Compliance with suggested besT practices) evaluation tool a questionnaire was designed for the purpose of evaluating automated A&F systems. A Rand-modified Delphi method was used to develop the process evaluation and obtain validation. Fourteen experts from different domains in primary care consented to participate and individually scored the questions on a 9-point Likert scale. Afterwards, the questions were discussed in a consensus meeting. After approval, the final questionnaire was compiled.

Results

A 34-question questionnaire composed of 57 items was developed and presented to the expert panel. The consensus meeting resulted in a selection of 31 questions, subdivided into 43 items. A final list of 30 questions consisting of 42 items was obtained.

Conclusion

A questionnaire consisting of 30 questions was drawn up for the assessment and improvement of automated A&F systems, based on CP-FIT and REFLECT-52 theory and approved by experts. Next steps will be piloting and implementation of the questionnaire.

SUBMITTER: Van den Wyngaert I

PROVIDER: S-EPMC10998400 | biostudies-literature | 2024 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Developing a questionnaire to evaluate an automated audit & feedback intervention: a Rand-modified Delphi method.

Van den Wyngaert Ine I Van Pottelbergh Gijs G Coteur Kristien K Vaes Bert B Van den Bulck Steve S

BMC health services research 20240405 1

<h4>Background</h4>Audit and feedback (A&F) is a widely used implementation strategy to evaluate and improve medical practice. The optimal design of an A&F system is uncertain and structured process evaluations are currently lacking. This study aimed to develop and validate a questionnaire to evaluate the use of automated A&F systems.<h4>Methods</h4>Based on the Clinical Performance Feedback Intervention Theory (CP-FIT) and the REFLECT-52 (REassessing audit & Feedback interventions: a tooL for E ...[more]

PMID: 38581009

Similar Datasets

Project description:BackgroundChronic kidney disease (CKD) is a common chronic condition and a rising public health issue with increased morbidity and mortality, even at an early stage. Primary care has a pivotal role in the early detection and in the integrated management of CKD which should be of high quality. The quality of care for CKD can be assessed using quality indicators (QIs) and if these QIs are extractable from the electronic medical record (EMR) of the general physician, the number of patients whose quality of care can be evaluated, could increase vastly. Therefore the aim of this study is to develop QIs which are evidence based, EMR extractable and which can be used as a framework to automate quality assessment.MethodsWe used a Rand-modified Delphi method to develop QIs for CKD in primary care. A questionnaire was designed by extracting recommendations from international guidelines based on the SMART principle and the EMR extractability. A multidisciplinary expert panel, including patients, individually scored the recommendations for measuring high quality care on a 9-point Likert scale. The results were analyzed based on the median Likert score, prioritization and agreement. Subsequently, the recommendations were discussed in a consensus meeting for their in- or exclusion. After a final appraisal by the panel members this resulted in a core set of recommendations, which were then transformed into QIs.ResultsA questionnaire composed of 99 recommendations was extracted from 10 international guidelines. The consensus meeting resulted in a core set of 36 recommendations that were translated into 36 QIs. This final set consists of QIs concerning definition & classification, screening, diagnosis, management consisting of follow up, treatment & vaccination, medication & patient safety and referral to a specialist. It were mostly the patients participating in the panel who stressed the importance of the QIs concerning medication & patient safety and a timely referral to a specialist.ConclusionThis study provides a set of 36 EMR extractable QIs for measuring the quality of primary care for CKD. These QIs can be used as a framework to automate quality assessment for CKD in primary care.

Project description:BackgroundAudit and Feedback (A&F) is one of the most common strategies used to improve quality in healthcare. However, there is still lack of awareness regarding the enabling factors and barriers that could influence its effectiveness. The aim of this study was to develop a questionnaire to measure the knowledge, attitudes and behaviors of general practitioners (GPs) regarding A&F. The study was performed in the context of the EASY-NET program (project code NET-2016-02364191).MethodsThe survey was developed according to two steps. Firstly, a scoping review was performed in order to map the literature on the existing similar instruments with the aim of identifying the sub-domains and possible items to include in a preliminary version of the questionnaire. In the second phase, the questionnaire was reviewed by a multidisciplinary group of experts and administrated to a convenience sample in a pilot survey.ResultsTen papers were included in the scoping review. The survey target and development methodology were heterogenous among the studies. The knowledge, attitudes and behaviors domains were assessed in six, nine and seven studies, respectively. In the first step, 126 pertinent items were extracted and categorized as follows: 8 investigated knowledge, 93 investigated attitudes, and 25 investigated behaviors. Then, 2 sub-domains were identified for knowledge, 14 for attitudes and 7 for behavior. Based on these results, a first version of the survey was developed via consensus among two authors and then revised by the multidisciplinary group of experts in the field of A&F. The final version of the survey included 36 items: 8 in the knowledge domain, 19 in the attitudes domain and 9 in the behaviors domain. The results of the pilot study among 15 GPs suggested a good acceptability and item relevance and accuracy, with positive answers totaling 100% and 93.3% in the proposed questions.ConclusionsThe methodology used has shown to be a good strategy for the development of the survey. The survey will be administrated before and after the implementation of an A&F intervention to assess both baseline characteristics and changes after the intervention.

Project description:BackgroundPrehypertension is a preclinical state of hypertension which leads to an increased likelihood of coronary heart disease, myocardial infarction, cerebrovascular disease as well as target organ damage. Addressing pre-hypertension through early lifestyle interventions is crucial to mitigating these detrimental effects and improving long-term health outcomes. So, the main objective of this study is to develop a lifestyle intervention program (LSIP) for the management of prehypertension using consensus building approach.MethodsIt was a three round online modified Delphi study with 70 members panellists. All panellists had an experience of prehypertension either as patients (n = 30) or professionals (n = 40). Round 1 included initial recommendations developed from a previous systematic review and metanalysis, which were rated by panellists for their importance on a 5-point Likert scale. Panellists could also suggest additional items in the Round 1. Round 2 and 3 included all items from the Round 1 with new items suggested by the panellists. Data was analysed descriptively using SPSS version 29. All items receiving at least 70% of all respondents combined rating of 'Important' and 'Very Important' in Round 3 were included in the final set of recommendations.ResultsFifty-one panellists (80.9%) (patients = 25, professionals = 26) completed Round 3. Twenty-six recommendation items were included in the Round 1. Twenty new items were added in Round 2 with 46 total items in Round 2 and 3. Thirty-five of these items reached consensus in Round 3. The final set of recommendation comprised of 15 educational. 10 dietary, and 10 exercise recommendations.ConclusionThis modified Delphi study developed a comprehensive LSIP for the prevention of prehypertension, incorporating a holistic approach with educational, dietary, and exercise components aimed at the general population. Previously established standards of care (SOC) for managing prehypertension varied significantly and often provided fragmented guidance particularly on physical activity and education. This preventive model offers a novel and scalable approach for early intervention in prehypertension, potentially reducing reliance on medications and improving long-term health outcomes.

Project description:BackgroundOlder adults are at high risk of developing delirium in the emergency department (ED); however, it is under-recognized in routine clinical care. Lack of detection and treatment is associated with poor outcomes, such as mortality. Performance measures (PMs) are needed to identify variations in quality care to help guide improvement strategies. The purpose of this study is to gain consensus on a set of quality statements and PMs that can be used to evaluate delirium care quality for older ED patients.MethodsA 3-round modified e-Delphi study was conducted with ED clinical experts. In each round, participants rated quality statements according to the concepts of importance and actionability, then their associated PMs according to the concept of necessity (1-9 Likert scales), with the ability to comment on each. Consensus and stability were evaluated using a priori criteria using descriptive statistics. Qualitative data was examined to identify themes within and across quality statements and PMs, which went through a participant validation exercise in the final round.ResultsTwenty-two experts participated, 95.5% were from west or central Canada. From 10 quality statements and 24 PMs, consensus was achieved for six quality statements and 22 PMs. Qualitative data supported justification for including three quality statements and one PM that achieved consensus slightly below a priori criteria. Three overarching themes emerged from the qualitative data related to quality statement actionability. Nine quality statements, nine structure PMs, and 14 process PMs are included in the final set, addressing four areas of delirium care: screening, diagnosis, risk reduction and management.ConclusionResults provide a set of quality statements and PMs that are important, actionable, and necessary to a diverse group of clinical experts. To our knowledge, this is the first known study to develop a de novo set of guideline-based quality statements and PMs to evaluate the quality of delirium care older adults receive in the ED setting.

Project description:ObjectiveCommunity paramedicine programme are often designed to address repeated and non-urgent use of paramedic services by providing patients with alternatives to the traditional 'treat and transport' ambulance model of care. We sought to investigate the level of consensus that could be found by a panel of experts regarding appropriate health, social and environmental domains that should be assessed in community paramedicine home visit programme.DesignWe applied the RAND/UCLA Appropriateness Method in a modified Delphi method to investigate the level of consensus on assessment domains for use in community paramedicine home visit programme.Setting and participantsWe included a multi-national panel of 17 experts on community paramedicine and in-home assessment from multiple settings (paramedicine, primary care, mental health, home and community care, geriatric care).MeasuresA list of potential assessment categories was established after a targeted literature review and confirmed by panel members. Over multiple rounds, panel members scored the appropriateness of 48 assessment domains on a Likert scale from 0 (not appropriate) to 5 (very appropriate). Scores were then reviewed at an in-person meeting and a finalised list of assessment domains was generated.ResultsAfter the preliminary round of scoring, all 48 assessment domains had scores that demonstrated consensus. Nine assessment domains (18.8%) demonstrated a wider range of rated appropriateness. No domains were found to be not appropriate. Achieving consensus about the appropriateness of assessment domains on the first round of scoring negated the need for subsequent rounds of scoring. The in-person meeting resulted in re-grouping assessment domains and adding an additional domain about urinary continence.ConclusionAn international panel of experts with knowledge about in-home assessment by community paramedics demonstrated a high level of agreement on appropriate patient assessment domains for community paramedicine home visit programme. Community paramedicine home visit programme are likely to have similar patient populations. A standardised assessment instrument may be viable in multiple settings.

Dataset Information

Developing a questionnaire to evaluate an automated audit & feedback intervention: a Rand-modified Delphi method.

Background

Methods

Results

Conclusion

Publications

Developing a questionnaire to evaluate an automated audit & feedback intervention: a Rand-modified Delphi method.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets