Dataset Information

Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool.

ABSTRACT:

Objectives

Only rudimentary tools exist to support health system organizations to evaluate their public and patient engagement (PPE) activities. This study responds to this gap by developing a generic evaluation tool for use in a wide range of organizations.

Methods

The evaluation tool was developed through an iterative, collaborative process informed by a review of published and grey literature and with the input of Canadian PPE researchers and practitioners. Over a 3-year period, structured e-mail, telephone and face-to-face exchanges, including a modified Delphi process, were used to produce an evaluation tool that includes core principles of high-quality engagement, expected outcomes for each principle and three unique evaluation questionnaires that were tested and revised with input from 65 end users.

Results

The tool is structured around four core principles of 'quality engagement': (i) integrity of design and process; (ii) influence and impact; (iii) participatory culture; and (iv) collaboration and common purpose. Three unique questionnaires were developed to assess each of these four evaluation domains from the following perspectives: (i) those who participate in PPE activities; (ii) those who plan, execute or sponsor PPE activities within organizations; and (iii) those who provide the leadership and capacity for PPE within their organizations.

Conclusions

This is the first known collaboration of researchers and practitioners in the co-design of a comprehensive PPE evaluation tool aimed at three distinct respondent groups and for use in a wide range of health system organization settings.

SUBMITTER: Abelson J

PROVIDER: S-EPMC5152717 | biostudies-literature | 2016 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool.

Abelson Julia J Li Kathy K Wilson Geoff G Shields Kristin K Schneider Colleen C Boesveld Sarah S

Health expectations : an international journal of public participation in health care and health policy 20150625 4

<h4>Objectives</h4>Only rudimentary tools exist to support health system organizations to evaluate their public and patient engagement (PPE) activities. This study responds to this gap by developing a generic evaluation tool for use in a wide range of organizations.<h4>Methods</h4>The evaluation tool was developed through an iterative, collaborative process informed by a review of published and grey literature and with the input of Canadian PPE researchers and practitioners. Over a 3-year period ...[more]

PMID: 26113295

Similar Datasets

Project description:BackgroundDespite growing evidence that deprescribing can improve clinical outcomes, quality of life and reduce the likelihood of adverse drug events, the practice is not widespread, particularly in hospital settings. Clinical risk assessment tools, like the Drug Burden Index (DBI), can help prioritise patients for medication review and prioritise medications to deprescribe, but are not integrated within routine care. The aim of this study was to conduct formative usability testing of a computerised decision support (CDS) tool, based on DBI, to identify modifications required to the tool prior to trialling in practice.MethodsOur CDS tool comprised a DBI MPage in the electronic medical record (clinical workspace) that facilitated review of a patient's DBI and medication list, access to deprescribing resources, and the ability to deprescribe. Two rounds of scenario-based formative usability testing with think-aloud protocol were used. Seventeen end-users participated in the testing, including junior and senior doctors, and pharmacists.ResultsParticipants expressed positive views about the DBI CDS tool but testing revealed a number of clear areas for improvement. These primarily related to terminology used (i.e. what is a DBI and how is it calculated?), and consistency of functionality and display. A key finding was that users wanted the CDS tool to look and function in a similar way to other decision support tools in the electronic medical record. Modifications were made to the CDS tool in response to user feedback.ConclusionUsability testing proved extremely useful for identifying components of our CDS tool that were confusing, difficult to locate or to understand. We recommend usability testing be adopted prior to implementation of any digital health intervention. We hope our revised CDS tool equips clinicians with the knowledge and confidence to consider discontinuation of inappropriate medications in routine care of hospitalised patients. In the next phase of our project, we plan to pilot test the tool in practice to evaluate its uptake and effectiveness in supporting deprescribing in routine hospital care.

Project description:BackgroundHealth system guidance (HSG) provides recommendations aimed to address health system challenges. However, there is a paucity of methods to direct, appraise, and report HSG. Earlier research identified 30 candidate criteria (concepts) that can be used to evaluate the quality of HSG and guide development and reporting requirements. The objective of this paper was to describe two studies aimed at evaluating the importance of these 30 criteria, design a draft HSG appraisal tool, and test its usability.MethodsThis study involved a two-step survey process. In step 1, respondents rated the 30 concepts for appropriateness to, relevance to, and priority for health system decisions and HSG. This led to a draft tool. In step 2, respondents reviewed HSG documents, appraised them using the tool, and answered a series of questions. Descriptive analyses were computed.ResultsFifty participants were invited in step 1, and we had a response rate of 82 %. The mean response rates for each concept within each survey question were universally favorable. There was also an overall agreement about the need for a high-quality tool to systematically direct the development, appraisal, and reporting of HSG. Qualitative feedback and a consensus process by the team led to refinements to some of the concepts and the creation of a beta (draft) version of the HSG tool. In step 2, 35 participants were invited and we had a response rate of 74 %. Exploratory analyses showed that the quality of the HSGs reviewed varied as a function of the HSG item and the specific document assessed. A favorable consensus was reached with participants agreeing that the HSG items were easy to understand and easy to apply. Moreover, the overall agreement was high for the usability of the tool to systematically direct the development (85 %), appraisal (92 %), and reporting (81 %) of HSG. From this process, version 1.0 of the HSG appraisal tool was generated complete with 32 items (and their descriptions) and 4 domains.ConclusionsThe final tool, named the Appraisal of Guidelines for Research and Evaluation for Health Systems (AGREE-HS) (version 1), defines expectations of HSG and facilitates informed decisions among policymakers on health system delivery, financial, and governance arrangements.

Dataset Information

Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool.

Objectives

Methods

Results

Conclusions

Publications

Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets