Dataset Information

Realist evaluation of the impact of the research translation process on health system sustainability: a study protocol.

ABSTRACT:

Introduction

Sustainability at a system level relates to the capacity of the system to be able to service the ongoing health needs of the population. It is a multifaceted concept encompassing both the affordability and efficiency of a system and the system's ability to adapt and change.To address issues that currently threaten health system sustainability, healthcare leaders, policy makers, clinicians and researchers are searching for solutions to ensure the delivery of safe, value-based care into the future. The timely translation of research evidence into sustainable interventions that can be adopted into the health system is one way of bolstering the sustainability of the system as a whole. We present a protocol for the realist evaluation of a research translation funding programme to understand how the research translation process contributes to health system sustainability and value-based healthcare.

Methods and analysis

Underpinned by the realist evaluation framework, we will: (1) Develop the Initial Program Theory (IPT) of the research translation process; (2) Test the program theory through case study analysis; and (3) Refine and consolidate the theory through stakeholder consultation. The evaluation uses a case example of a research translation programme, chosen for its representation of a microcosm of the broader health system and the heterogeneity of service improvement activities taking place within it. Across the three phases, analysis of data from documents about the research translation program and interviews and focus groups with stakeholders and program users will draw on the context (C), mechanism (M), outcome (O) formula that is core to realist evaluation. In addition, system dynamic methods will capture the feedback loops and complex relationships among the IPT and context-mechanism-outcome configurations. This approach to evaluation of a research translation funding programme may be adapted to similar programmes operating in other settings.

Ethics and dissemination

Curtin University Human Research Ethics Committee, Western Australia, approved this study (approval number: HRE2020-0464). Results will be published in scientific journals, and communicated to respondents and relevant partners.

SUBMITTER: Mosedale A

PROVIDER: S-EPMC9196166 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe informed consent process aims to provide potential participants with information about health research that enables them to make an informed decision as to whether they choose to participate, or not. However, it remains unclear as to whether the process is effective for those who are under-served in health research. It is a pivotal issue within health research that the diversity of people who participate is broadened. The National Institute for Health Research (NIHR) pledges to support equality, diversity and inclusion, actively creating opportunities for all citizens whom are eligible, to take part in health research.MethodsIn order to understand how the informed consent process for under-served populations in health research works, under what circumstances and in what respects, a realist review approach will be undertaken. Searches will be carried out using electronic databases (EMBASE, MEDLINE, Web of Science and PsycINFO), along with selected websites and grey literature. Development of initial rough programme theory(ies) will lead to a more refined programme theory that will provide an explanation of context, mechanism and outcomes. Stakeholder involvement by NIHR (Public) Research Champions, health professionals and clinical academics will provide expert opinion about concepts and programme theory.DiscussionFindings of this realist review will highlight how the informed consent process in health research affects the experience and decision-making process of potential participants from under-served populations. They will be written up in accordance with RAMESES guidelines and disseminated to patients and the public, health researchers, health professionals and policymakers through peer-reviewed publication, presentations and discussions. The review will contribute to our understanding of the mechanisms that cause both positive and negative outcomes in the informed consent process for those whom are often under-represented in health research to inform policy, study design and delivery.

Project description:Achievement of improved maternal and child health (MCH) outcomes continues to be an issue of international priority, particularly for sub-Saharan African countries such as Nigeria. Evidence suggests that the use of Community Health Workers (CHWs) can be effective in broadening access to, and coverage of, health services and improving MCH outcomes in such countries.In this paper, we report the methodology for a 5-year study which aims to evaluate the context, processes, outcomes and longer-term sustainability of a Nigerian CHW scheme. Evaluation of complex interventions requires a comprehensive understanding of intervention context, mechanisms and outcomes. The multidisciplinary and mixed-method realist approach will facilitate such evaluation. A favourable policy environment within which the study is conducted will ensure the successful uptake of results into policy and practice. A realist evaluation provides an overall methodological framework for this multidisciplinary and mixed methods research, which will be undertaken in Anambra state. The study will draw upon health economics, social sciences and statistics. The study comprises three steps: (1) initial theory development; (2) theory validation and (3) theory refinement and development of lessons learned. Specific methods for data collection will include in-depth interviews and focus group discussions with purposefully identified key stakeholders (managers, service providers and service users), document reviews, analyses of quantitative data from the CHW programme and health information system, and a small-scale survey. The impact of the programme on key output and outcome indicators will be assessed through an interrupted time-series analysis (ITS) of monthly quantitative data from health information system and programme reports. Ethics approvals for this study were obtained from the University of Leeds and the University of Nigeria.This study will provide a timely and important contribution to health systems strengthening specifically within Anambra state in southeast Nigeria but also more widely across Nigeria. This paper should be of interest to researchers who are interested in adapting and applying robust methodologies for assessing complex health system interventions. The paper will also be useful to policymakers and practitioners who are interested in commissioning and engaging in such complex evaluations to inform policies and practices.

Project description:INTRODUCTION:Responsiveness to service users' views is a widely recognised objective of health systems. A key component of responsive health systems is effective interaction between users and service providers. Despite a growing literature on patient feedback from high-income settings, less is known about effectiveness of such systems in low-income and middle-income countries. METHODOLOGY AND ANALYSIS:This paper disseminates the protocol for an 18-month 'RESPOND' project that aims to evaluate the system of collecting and responding to user feedback in Bangladesh. This mixed-method study uses a realist evaluation approach to examine user feedback systems at two Upazila health complexes in Comilla District of Bangladesh, and comprises three steps: (1) initial theory development; (2) theory validation; and (3) theory refinement and development of lessons learnt. The project also uses (1) process evaluation to understand causal mechanisms and contexts of implementation; (2) statistical analysis of patient feedback to clarify the nature of issues reported; (3) social science methods to illuminate feedback processes and user and provider experiences; and (4) health policy and systems research to clarify issues related to integration of feedback systems with quality assurance and human resource management. During data analysis, qualitative and quantitative findings will be integrated throughout to help achieve study objectives. Analysis of qualitative and quantitative data will be done using a convergent mixed-methods model, involving continuous triangulation of multiple data sets to facilitate greater understanding of the context of user feedback systems including the links with relevant policies, practices and programmes. ETHICS AND DISSEMINATION:Ethics approvals were obtained from the University of Leeds and the Bangladesh Medical Research Council. All data collected for this study will be anonymised, and identifying characteristics of respondents will not appear in a final manuscript or reports. The study findings will be presented at scientific conferences and published in peer-reviewed journals.

Project description:BackgroundKnowledge translation (KT) is an important means of improving the health service quality. Most research on the effectiveness of KT strategies has focused on individual strategies, i.e., those directly targeting the modification of allied health professionals' knowledge, attitudes, and behaviors, for example. In general, these strategies are moderately effective in changing practices (maximum 10% change). Effecting change in organizational contexts (e.g., change readiness, general and specific organizational capacity, organizational routines) is part of a promising new avenue to service quality improvement through the implementation of evidence-based practices. The objective of this study will be to identify why, how, and under what conditions organizational KT strategies have been shown to be effective or ineffective in changing the (a) knowledge, (b) attitudes, and (c) clinical behaviors of allied health professionals in traumatology settings.MethodsThis is a realist review protocol involving four iterative steps: (1) Initial theory formulation, (2) search for Evidence search, (3) knowledge extraction and synthesis, and (4) recommendations. We will search electronic databases such as PubMed, Embase, CINHAL, Cochrane Library, and Conference Proceedings Citation Index - Science. The studies included will be those relating to the use of organizational KT strategies in trauma settings, regardless of study designs, published between January 1990 and October 2020, and presenting objective measures that demonstrate change in allied health professionals' knowledge, attitudes, and clinical behaviors. Two independent reviewers will select, screen, and extract the data related to all relevant sources in order to refine or refute the context-mechanism-outcome (CMO) configurations developed in the initial theory and identify new CMO configurations.DiscussionUsing a systematic and rigorous method, this review will help guide decision-makers and researchers in choosing the best organizational strategies to optimize the implementation of evidence-based practices.Systematic review registrationPROSPERO CRD42020216105.

Project description:Background:High quality public engagement and involvement (PEI) in data-intensive health research is seen as one way of ensuring that social legitimacy, i.e. a social license, is conferred through public acceptance of the need for research use of their data. This is a complex research area, and portfolios of involvement have been suggested, but not yet evaluated, to support the role of public contributors. The study aim was to evaluate if and how membership of a data-intensive research public forum can act as a mechanism for enhancing members' personal development. Our objective was to understand the circumstances and mechanisms that help to explain how, why and for whom involvement with a public forum enhanced those members' personal development. Method:Qualitative data were collected from 15 current and previous members, via semi-structured interviews, notes from meetings, and consultations with and feedback from members. Data were critically compared, contrasted and reviewed until no new themes could be discerned and then condensed into context-mechanism-outcome (CMO) configurations. Realist evaluation was used to generate a theoretical and empirical appreciation of the contextual circumstances and mechanisms which help to explain the extent to which involvement with a public forum would enhance members' personal development and, if so, how, why, and for whom. Results:Three CMO configurations were identified. All of them showed that using the portfolio facilitated growth in forum members' personal development, but only where the members valued using the portfolio. This was particularly so for female members. Members valued the portfolio in one or more of three ways: as a tool to record and evidence activities, to facilitate reflective practice or as a guiding framework. Conclusions:Data analysis and consideration of the three CMO configurations suggests a refined middle range theory that 'The use of a portfolio as a framework for learning in a public forum will facilitate members' personal development if they value its use as a framework for learning'. Further work is needed to confirm these findings both elsewhere in data-intensive health research and in other complex research areas using public forums for PEI. Plain English summary:Public engagement and involvement in health research is now well established and makes a valuable contribution to the research process. However, little is known about its impact on participants. This article investigates how involvement in a data-intensive health research public forum impacts on public forum members, rather than the research process. Personal involvement portfolios were used to support their involvement work and help evaluate if and how involvement in research activities enhanced members' personal development. Taking a realist evaluation approach, 'Context-Mechanism-Outcome' configurations were used to explore how membership of a public forum might enhance public forum members' personal development. The Context-Mechanism-Outcome configuration refers to an exploration of what influences the extent to which an intervention is successful or unsuccessful in producing positive outcomes and tries to identify the reasons why it is successful for some and unsuccessful for others. However, evidence from this realist evaluation recommends that engagement and involvement should always be underpinned by procedures which ensure that public contributors receive ongoing and tailored guidance and support throughout the process.

Project description:IntroductionThere has been a lot of attention on the role of human resource management interventions to improve delivery of health services in low- and middle-income countries. However, studies on this subject are few due to limited research on implementation of programmes and methodological difficulties in conducting experimental studies on human resource interventions. The authors present the protocol of an evaluation of a district-level capacity-building intervention to identify the determinants of performance of health workers in managerial positions and to understand how changes (if any) are brought about.Methods and analysisThe aim of this study is to understand how capacity building works. The authors will use realist evaluation to evaluate an intervention in Karnataka, India. The intervention is a capacity-building programme that seeks to improve management capacities of health managers at district and subdistrict levels through periodic classroom-based teaching and mentoring support at the workplace. The authors conducted interviews and reviewed literature on capacity building in health to draw out the programme theory of the intervention. Based on this, the authors formulated hypothetical pathways connecting the expected outcomes of the intervention (planning and supervision) to the inputs (contact classes and mentoring). The authors prepared a questionnaire to assess elements of the programme theory-organisational culture, self-efficacy and supervision. The authors shall conduct a survey among health managers as well as collect qualitative data through interviews with participants and non-participants selected purposively based on their planning and supervision performance. The authors will construct explanations in the form of context-mechanism-outcome configurations from the results. This will be iterative and the authors will use a realist evaluation framework to refine the explanatory theories that are based on the findings to explain and validate an improved theory on 'what works for whom and under what conditions'.DiscussionThe scope for applying realist evaluation to study human resource management interventions in health are discussed.

Project description:BackgroundAn intervention's success depends on how participants interact with it in local settings. Process evaluation examines these interactions, indicating why an intervention was or was not effective, and how it (and similar interventions) can be improved for better contextual fit. This is particularly important for innovative trials like Supporting Policy In health with Research: an Intervention Trial (SPIRIT), where causal mechanisms are poorly understood. SPIRIT was testing a multi-component intervention designed to increase the capacity of health policymakers to use research.MethodsOur mixed-methods process evaluation sought to explain variation in observed process effects across the six agencies that participated in SPIRIT. Data collection included observations of intervention workshops (n = 59), purposively sampled interviews (n = 76) and participant feedback forms (n = 553). Using a realist approach, data was coded for context-mechanism-process effect configurations (retroductive analysis) by two authors.ResultsIntervention workshops were very well received. There was greater variation of views regarding other aspects of SPIRIT such as data collection, communication and the intervention's overall value. We identified nine inter-related mechanisms that were crucial for engaging participants in these policy settings: (1) Accepting the premise (agreeing with the study's assumptions); (2) Self-determination (participative choice); (3) The Value Proposition (seeing potential gain); (4) 'Getting good stuff' (identifying useful ideas, resources or connections); (5) Self-efficacy (believing 'we can do this!'); (6) Respect (feeling that SPIRIT understands and values one's work); (7) Confidence (believing in the study's integrity and validity); (8) Persuasive leadership (authentic and compelling advocacy from leaders); and (9) Strategic insider facilitation (local translation and mediation). These findings were used to develop tentative explanatory propositions and to revise the programme theory.ConclusionThis paper describes how SPIRIT functioned in six policy agencies, including why strategies that worked well in one site were less effective in others. Findings indicate a complex interaction between participants' perception of the intervention, shifting contextual factors, and the form that the intervention took in each site. Our propositions provide transferable lessons about contextualised areas of strength and weakness that may be useful in the development and implementation of similar studies.