Patient and public engagement in research and health system decision making: A systematic review of evaluation tools.
Ontology highlight
ABSTRACT:
SUBMITTER: Boivin A
PROVIDER: S-EPMC6250878 | biostudies-other | 2018 Dec
REPOSITORIES: biostudies-other
Publications
Patient and public engagement in research and health system decision making: A systematic review of evaluation tools.
Boivin Antoine A L'Espérance Audrey A Gauvin François-Pierre FP Dumez Vincent V Macaulay Ann C AC Lehoux Pascale P Abelson Julia J
Health expectations : an international journal of public participation in health care and health policy 20180730 6
<h4>Background</h4>Patient and public engagement is growing, but evaluative efforts remain limited. Reviews looking at evaluation tools for patient engagement in individual decision making do exist, but no similar articles in research and health systems have been published.<h4>Objective</h4>Systematically review and appraise evaluation tools for patient and public engagement in research and health system decision making.<h4>Methods</h4>We searched literature published between January 1980 and Fe ...[more]
Similar Datasets
Project description:BackgroundPatient, public, consumer, and community (P2C2) engagement in organization-, community-, and systemlevel healthcare decision-making is increasing globally, but its formal evaluation remains challenging. To define a taxonomy of possible P2C2 engagement metrics and compare existing evaluation tools against this taxonomy, we conducted a systematic review.MethodsA broad search strategy was developed for English language publications available from January 1962 through April 2015 in PubMed, Embase, Sociological Abstracts, PsycINFO, EconLit, and the gray literature. A publication was excluded if: (1) the setting was not healthcare delivery (ie, we excluded non-health sectors, such as urban planning; research settings; and public health settings not involving clinical care delivery); (2) the P2C2 engagement was episodic; or (3) the concept of evaluation or possible evaluation metrics were absent. To be included as an evaluation tool, publications had to contain an evaluative instrument that could be employed with minimal modification by a healthcare organization.ResultsA total of 199 out of 3953 publications met exclusion and inclusion criteria. These were qualitatively analyzed using inductive content analysis to create a comprehensive taxonomy of 116 possible metrics for evaluating P2C2 engagement. 44 outcome metrics were grouped into three domains (internal, external, and aggregate outcomes) that included six subdomains: impact on engagement participants, impact on services provided by the healthcare organization, impact on the organization itself, influence on the broader public, influence on population health, and engagement cost-effectiveness. The 72 process metrics formed four domains (direct process metrics; surrogate process metrics; aggregate process metrics; and preconditions for engagement) that comprised sixteen subdomains. We identified 23 potential tools for evaluating P2C2 engagement. The identified tools were published between 1973-2015 and varied in their coverage of the taxonomy, methodology used (qualitative, quantitative, or mixed), and intended evaluators (organizational leaders, P2C2 participants, external evaluators, or some combination). Parts of the metric taxonomy were absent from all tools.ConclusionBy comprehensively mapping potential outcome and process metrics as well as existing P2C2 engagement tools, this review supports high-quality P2C2 engagement globally by informing the selection of existing evaluation tools and identifying gaps where new tools are needed.
| S-EPMC6186472 | biostudies-literature
Project description:BackgroundPublic health professionals are increasingly expected to engage in evidence-informed decision making to inform practice and policy decisions. Evidence-informed decision making involves the use of research evidence along with expertise, existing public health resources, knowledge about community health issues, the local context and community, and the political climate. The National Collaborating Centre for Methods and Tools has identified a seven step process for evidence-informed decision making. Tools have been developed to support public health professionals as they work through each of these steps. This paper provides an overview of tools used in three Canadian public health departments involved in a study to develop capacity for evidence-informed decision making.MethodsAs part of a knowledge translation and exchange intervention, a Knowledge Broker worked with public health professionals to identify and apply tools for use with each of the steps of evidence-informed decision making. The Knowledge Broker maintained a reflective journal and interviews were conducted with a purposive sample of decision makers and public health professionals. This paper presents qualitative analysis of the perceived usefulness and usability of the tools.ResultsTools were used in the health departments to assist in: question identification and clarification; searching for the best available research evidence; assessing the research evidence for quality through critical appraisal; deciphering the 'actionable message(s)' from the research evidence; tailoring messages to the local context to ensure their relevance and suitability; deciding whether and planning how to implement research evidence in the local context; and evaluating the effectiveness of implementation efforts. Decision makers provided descriptions of how the tools were used within the health departments and made suggestions for improvement. Overall, the tools were perceived as valuable for advancing and sustaining evidence-informed decision making.ConclusionTools are available to support the process of evidence-informed decision making among public health professionals. The usability and usefulness of these tools for advancing and sustaining evidence-informed decision making are discussed, including recommendations for the tools' application in other public health settings beyond this study. Knowledge and awareness of these tools may assist other health professionals in their efforts to implement evidence-informed practice.
| S-EPMC4223550 | biostudies-literature
Project description:BackgroundThe use of research evidence to underpin public health policy is strongly promoted. However, its implementation has not been straightforward. The objectives of this systematic review were to synthesise empirical evidence on the use of research evidence by public health decision makers in settings with universal health care systems.MethodsTo locate eligible studies, 13 bibliographic databases were screened, organisational websites were scanned, key informants were contacted and bibliographies of included studies were scrutinised. Two reviewers independently assessed studies for inclusion, extracted data and assessed methodological quality. Data were synthesised as a narrative review.Findings18 studies were included: 15 qualitative studies, and three surveys. Their methodological quality was mixed. They were set in a range of country and decision making settings. Study participants included 1063 public health decision makers, 72 researchers, and 174 with overlapping roles. Decision making processes varied widely between settings, and were viewed differently by key players. A range of research evidence was accessed. However, there was no reliable evidence on the extent of its use. Its impact was often indirect, competing with other influences. Barriers to the use of research evidence included: decision makers' perceptions of research evidence; the gulf between researchers and decision makers; the culture of decision making; competing influences on decision making; and practical constraints. Suggested (but largely untested) ways of overcoming these barriers included: research targeted at the needs of decision makers; research clearly highlighting key messages; and capacity building. There was little evidence on the role of research evidence in decision making to reduce inequalities.ConclusionsTo more effectively implement research informed public health policy, action is required by decision makers and researchers to address the barriers identified in this systematic review. There is an urgent need for evidence to support the use of research evidence to inform public health decision making to reduce inequalities.
| S-EPMC3144216 | biostudies-literature
Project description:BackgroundWith healthcare striving to shift to a more person-centered delivery model, patient and family involvement must have a bigger role in shaping this. While many initiatives involving patients and family members focus on self-care, a broader understanding of patient participation is necessary. Ensuring a viable and sustainable critical number of qualified patients and family members to support this shift will be of utmost importance. The purpose of this study was to understand how health systems are intentionally investing in the training and skill development of patients and family members.MethodsPatient co-investigators and researchers conducted a scoping review of the existing literature on methods adopted by healthcare systems to build the skills and capacity of patients to participate in healthcare decision-making using a recognized methodological framework. Six electronic databases were searched to identify studies. Two independent reviewers screened titles and abstracts and full-text papers for inclusion. The research team independently extracted data. Any disagreements were resolved by achieving consensus through discussion. Quantitative and qualitative content synthesis, as well as a quality assessment, was conducted.ResultsAfter eliminating duplicates, the search resulted in 9428 abstracts. Four hundred fifty-eight articles were reviewed and 15 articles were included. Four themes emerged: forums (33%), patient instructors (20%), workshops (33%), and co-design (13%). Four of the identified studies measured the impact and overall effectiveness of the respective programs. Examples of how patient and family members were supported (invested in) included advocacy training to support future involvement in engagement activities, a training program to conduct patient-led research, involvement in an immersive experience-based co-design initiative, and involvement in training pharmacy students. Overall, these studies found positive outcomes when patients and family members were recipients of these opportunities.ConclusionsThe results of this scoping review demonstrate that an evidence base around programs to advance patient engagement is largely absent. An opportunity exists for further research to identify strategies and measures to support patient engagement in healthcare decision-making.
| S-EPMC6474065 | biostudies-literature
Project description:BackgroundAs citizens, patients and family members are participating in numerous and expanding roles in health system organizations, attention has turned to evaluating these efforts. The context-specific nature of engagement requires evaluation tools to be carefully designed for optimal use. We sought to address this need by assessing the appropriateness and feasibility of a generic tool across a range of health system organizations, engagement activities and patient groups.MethodsWe used a mixed-methods implementation research design to study the implementation of an engagement evaluation tool in seven health system organizations in Ontario, Canada focusing on two key implementation outcome variables: appropriateness and feasibility. Data were collected through respondent feedback questions (binary and open-ended) at the end of the tool's three questionnaires as well as interviews and debriefing discussions with engagement professionals and patient partners from collaborating organizations.ResultsThe three questionnaires comprising the evaluation tool were collectively administered 29 times to 405 respondents yielding a 52% response rate (90% and 53% of respondents respectively assessed the survey's appropriateness and feasibility [quantitatively or qualitatively]). The questionnaires' basic properties were rated highly by all respondents. Concrete suggestions were provided for improving the appropriateness and feasibility of the questionnaires (or components within) for different engagement activity and organization types, and for enhancing the timing of implementation.Discussion and conclusionsOur study findings offer guidance for health system organizations and evaluators to support the optimal use of engagement evaluation tools across a variety of health system settings, engagement activities and respondent groups.
| S-EPMC6803403 | biostudies-literature
Project description:Background:High quality public engagement and involvement (PEI) in data-intensive health research is seen as one way of ensuring that social legitimacy, i.e. a social license, is conferred through public acceptance of the need for research use of their data. This is a complex research area, and portfolios of involvement have been suggested, but not yet evaluated, to support the role of public contributors. The study aim was to evaluate if and how membership of a data-intensive research public forum can act as a mechanism for enhancing members' personal development. Our objective was to understand the circumstances and mechanisms that help to explain how, why and for whom involvement with a public forum enhanced those members' personal development. Method:Qualitative data were collected from 15 current and previous members, via semi-structured interviews, notes from meetings, and consultations with and feedback from members. Data were critically compared, contrasted and reviewed until no new themes could be discerned and then condensed into context-mechanism-outcome (CMO) configurations. Realist evaluation was used to generate a theoretical and empirical appreciation of the contextual circumstances and mechanisms which help to explain the extent to which involvement with a public forum would enhance members' personal development and, if so, how, why, and for whom. Results:Three CMO configurations were identified. All of them showed that using the portfolio facilitated growth in forum members' personal development, but only where the members valued using the portfolio. This was particularly so for female members. Members valued the portfolio in one or more of three ways: as a tool to record and evidence activities, to facilitate reflective practice or as a guiding framework. Conclusions:Data analysis and consideration of the three CMO configurations suggests a refined middle range theory that 'The use of a portfolio as a framework for learning in a public forum will facilitate members' personal development if they value its use as a framework for learning'. Further work is needed to confirm these findings both elsewhere in data-intensive health research and in other complex research areas using public forums for PEI. Plain English summary:Public engagement and involvement in health research is now well established and makes a valuable contribution to the research process. However, little is known about its impact on participants. This article investigates how involvement in a data-intensive health research public forum impacts on public forum members, rather than the research process. Personal involvement portfolios were used to support their involvement work and help evaluate if and how involvement in research activities enhanced members' personal development. Taking a realist evaluation approach, 'Context-Mechanism-Outcome' configurations were used to explore how membership of a public forum might enhance public forum members' personal development. The Context-Mechanism-Outcome configuration refers to an exploration of what influences the extent to which an intervention is successful or unsuccessful in producing positive outcomes and tries to identify the reasons why it is successful for some and unsuccessful for others. However, evidence from this realist evaluation recommends that engagement and involvement should always be underpinned by procedures which ensure that public contributors receive ongoing and tailored guidance and support throughout the process.
| S-EPMC7325137 | biostudies-literature
Project description:ObjectivesOnly rudimentary tools exist to support health system organizations to evaluate their public and patient engagement (PPE) activities. This study responds to this gap by developing a generic evaluation tool for use in a wide range of organizations.MethodsThe evaluation tool was developed through an iterative, collaborative process informed by a review of published and grey literature and with the input of Canadian PPE researchers and practitioners. Over a 3-year period, structured e-mail, telephone and face-to-face exchanges, including a modified Delphi process, were used to produce an evaluation tool that includes core principles of high-quality engagement, expected outcomes for each principle and three unique evaluation questionnaires that were tested and revised with input from 65 end users.ResultsThe tool is structured around four core principles of 'quality engagement': (i) integrity of design and process; (ii) influence and impact; (iii) participatory culture; and (iv) collaboration and common purpose. Three unique questionnaires were developed to assess each of these four evaluation domains from the following perspectives: (i) those who participate in PPE activities; (ii) those who plan, execute or sponsor PPE activities within organizations; and (iii) those who provide the leadership and capacity for PPE within their organizations.ConclusionsThis is the first known collaboration of researchers and practitioners in the co-design of a comprehensive PPE evaluation tool aimed at three distinct respondent groups and for use in a wide range of health system organization settings.
| S-EPMC5152717 | biostudies-literature
Project description:BACKGROUND: A compelling ethical rationale supports patient engagement in healthcare research. It is also assumed that patient engagement will lead to research findings that are more pertinent to patients' concerns and dilemmas. However; it is unclear how to best conduct this process. In this systematic review we aimed to answer 4 key questions: what are the best ways to identify patient representatives? How to engage them in designing and conducting research? What are the observed benefits of patient engagement? What are the harms and barriers of patient engagement? METHODS: We searched MEDLINE, EMBASE, PsycInfo, Cochrane, EBSCO, CINAHL, SCOPUS, Web of Science, Business Search Premier, Academic Search Premier and Google Scholar. Included studies were published in English, of any size or design that described engaging patients or their surrogates in research design. We conducted an environmental scan of the grey literature and consulted with experts and patients. Data were analyzed using a non-quantitative, meta-narrative approach. RESULTS: We included 142 studies that described a spectrum of engagement. In general, engagement was feasible in most settings and most commonly done in the beginning of research (agenda setting and protocol development) and less commonly during the execution and translation of research. We found no comparative analytic studies to recommend a particular method. Patient engagement increased study enrollment rates and aided researchers in securing funding, designing study protocols and choosing relevant outcomes. The most commonly cited challenges were related to logistics (extra time and funding needed for engagement) and to an overarching worry of a tokenistic engagement. CONCLUSIONS: Patient engagement in healthcare research is likely feasible in many settings. However, this engagement comes at a cost and can become tokenistic. Research dedicated to identifying the best methods to achieve engagement is lacking and clearly needed.
| S-EPMC3938901 | biostudies-other
Project description:INTRODUCTION:To advance person- and family-centred healthcare, government initiatives have supported the engagement of patients and family caregivers in decision-making in healthcare systems. There is, however, no consensus on how to define success for such initiatives. This scoping review aims to identify the key elements for defining the quality of patient and family caregiver engagement in decision-making across the engagement domains (individual, community/organisation, system) of British Columbia's healthcare system. We will use those elements to develop a conceptual evaluation framework. METHODS AND ANALYSIS:This scoping review follows Arskey and O'Malley's methodology. (1) The research question was identified through team discussions. (2) Articles for data source will be identified using a librarian-informed search strategy for seven bibliographic databases as well as grey literature sources. (3) Selected articles will be relevant to the evaluation of patient and family caregiver engagement in healthcare systems. (4) Two researchers will independently extract data into predefined and emerging categories. (5) The researchers will reconcile and organise the identified elements. The research team's collective perspective will then refine the elements, and select, interpret and summarise the results. (6) Persons from key stakeholder groups will be consulted to refine the emergent conceptual framework. ETHICS AND DISSEMINATION:We will seek ethics approval for the stakeholder consultation. This study follows an integrated knowledge translation approach. The results will inform evaluation of the Patients as Partners Initiative of the British Columbia Ministry of Health, and will be disseminated as a scientific article, a research brief, and presentations at conferences and stakeholder meetings.
| S-EPMC6858224 | biostudies-literature
Project description:IntroductionAs the world's population ages, there has been increasing attention to developing health policies to support older adults. Engaging older adults in policy-making is one way to ensure that policy decisions align with their needs and priorities. However, ageist stereotypes often underestimate older adults' ability to participate in such initiatives. This scoping review aims to describe the characteristics and impacts of public engagement initiatives designed to help inform health policy-making for older adults.MethodsA systematic search of peer-reviewed and grey literature (English only) describing public engagement initiatives in health policy-making for older adults was conducted using six electronic databases, Google and the Participedia website. No geographical, methodological or time restrictions were applied to the search. Eligibility criteria were purposefully broad to capture a wide array of relevant engagement initiatives. The outcomes of interest included participants, engagement methods and reported impacts.ResultsThis review included 38 papers. The majority of public engagement initiatives were funded or initiated by governments or government agencies as a formal activity to address policy issues, compared to initiatives without a clear link to a specific policy-making process (e.g., research projects). While most initiatives engaged older adults as target participants, there was limited reporting on efforts to achieve participant diversity. Consultation-type engagement activities were most prevalent, compared to deliberative and collaborative approaches. Impacts of public engagement were frequently reported without formal evaluations. Notably, a few articles reported negative impacts of such initiatives.ConclusionThis review describes how public engagement practices have been conducted to help inform health policy-making for older adults and the documented impacts. The findings can assist policymakers, government staff, researchers and seniors' advocates in supporting the design and execution of public engagement initiatives in this policy sector.Patient or public contributionOlder adult partners from the McMaster University Collaborative for Health and Aging provided strategic advice throughout the key phases of this review, including developing a review protocol, data charting and synthesis and interpreting and presenting the review findings. This collaborative partnership was an essential aspect of this review, enhancing its relevance and meaningfulness for older adults.
| S-EPMC11347750 | biostudies-literature