Dataset Information

The development of ORACLe: a measure of an organisation's capacity to engage in evidence-informed health policy.

ABSTRACT:

Background

Evidence-informed policymaking is more likely if organisations have cultures that promote research use and invest in resources that facilitate staff engagement with research. Measures of organisations' research use culture and capacity are needed to assess current capacity, identify opportunities for improvement, and examine the impact of capacity-building interventions. The aim of the current study was to develop a comprehensive system to measure and score organisations' capacity to engage with and use research in policymaking, which we entitled ORACLe (Organisational Research Access, Culture, and Leadership).

Method

We used a multifaceted approach to develop ORACLe. Firstly, we reviewed the available literature to identify key domains of organisational tools and systems that may facilitate research use by staff. We interviewed senior health policymakers to verify the relevance and applicability of these domains. This information was used to generate an interview schedule that focused on seven key domains of organisational capacity. The interview was pilot-tested within four Australian policy agencies. A discrete choice experiment (DCE) was then undertaken using an expert sample to establish the relative importance of these domains. This data was used to produce a scoring system for ORACLe.

Results

The ORACLe interview was developed, comprised of 23 questions addressing seven domains of organisational capacity and tools that support research use, including (1) documented processes for policymaking; (2) leadership training; (3) staff training; (4) research resources (e.g. database access); and systems to (5) generate new research, (6) undertake evaluations, and (7) strengthen relationships with researchers. From the DCE data, a conditional logit model was estimated to calculate total scores that took into account the relative importance of the seven domains. The model indicated that our expert sample placed the greatest importance on domains (2), (3) and (4).

Conclusion

We utilised qualitative and quantitative methods to develop a system to assess and score organisations' capacity to engage with and apply research to policy. Our measure assesses a broad range of capacity domains and identifies the relative importance of these capacities. ORACLe data can be used by organisations keen to increase their use of evidence to identify areas for further development.

SUBMITTER: Makkar SR

PROVIDER: S-EPMC4712550 | biostudies-literature | 2016 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The development of ORACLe: a measure of an organisation's capacity to engage in evidence-informed health policy.

Makkar Steve R SR Turner Tari T Williamson Anna A Louviere Jordan J Redman Sally S Haynes Abby A Green Sally S Brennan Sue S

Health research policy and systems 20160114

<h4>Background</h4>Evidence-informed policymaking is more likely if organisations have cultures that promote research use and invest in resources that facilitate staff engagement with research. Measures of organisations' research use culture and capacity are needed to assess current capacity, identify opportunities for improvement, and examine the impact of capacity-building interventions. The aim of the current study was to develop a comprehensive system to measure and score organisations' capa ...[more]

PMID: 26769570

Similar Datasets

Project description:BACKGROUND:Ensuring health policies are informed by evidence still remains a challenge despite efforts devoted to this aim. Several tools and approaches aimed at fostering evidence-informed policy-making (EIPM) have been developed, yet there is a lack of availability of indicators specifically devoted to assess and support EIPM. The present study aims to overcome this by building a set of measurable indicators for EIPM intended to infer if and to what extent health-related policies are, or are expected to be, evidence-informed for the purposes of policy planning as well as formative and summative evaluations. METHODS:The indicators for EIPM were developed and validated at international level by means of a two-round internet-based Delphi study conducted within the European project 'REsearch into POlicy to enhance Physical Activity' (REPOPA). A total of 82 researchers and policy-makers from the six European countries (Denmark, Finland, Italy, the Netherlands, Romania, the United Kingdom) involved in the project and international organisations were asked to evaluate the relevance and feasibility of an initial set of 23 indicators developed by REPOPA researchers on the basis of literature and knowledge gathered from the previous phases of the project, and to propose new indicators. RESULTS:The first Delphi round led to the validation of 14 initial indicators and to the development of 8 additional indicators based on panellists' suggestions; the second round led to the validation of a further 11 indicators, including 6 proposed by panellists, and to the rejection of 6 indicators. A total of 25 indicators were validated, covering EIPM issues related to human resources, documentation, participation and monitoring, and stressing different levels of knowledge exchange and involvement of researchers and other stakeholders in policy development and evaluation. CONCLUSION:The study overcame the lack of availability of indicators to assess if and to what extent policies are realised in an evidence-informed manner thanks to the active contribution of researchers and policy-makers. These indicators are intended to become a shared resource usable by policy-makers, researchers and other stakeholders, with a crucial impact on fostering the development of policies informed by evidence.

Project description:There is increasing interest globally in the use of more rigorous processes to ensure that maternal, newborn, and child health (MNCH) care recommendations are informed by the best available research evidence use. The purpose of this study was to engage Nigerian MNCH policy-makers and other stakeholders to consider issues around research to policy and practice interface and to assess their existing knowledge and capacity on the use of research evidence for policy-making and practice.The study design is a cross-sectional evaluation of MNCH stakeholders' knowledge as it pertains different dimensions of research to practice. This was undertaken during a national MNCH stakeholders' engagement event convened under the auspices of the West African Health Organization (WAHO) and the Federal Ministry of Health (FMoH) in Abuja, Nigeria. A questionnaire was administered to participants, which was designed to assess participants' knowledge, capacity and organizational process of generation, synthesis and utilization of research evidence in policy-making regarding MNCH.A total of 40 participants signed the informed consent form and completed the questionnaire. The mean ratings (MNRs) of participants' knowledge of electronic databases and capacity to identify and obtain relevant research evidence from electronic databases ranged from 3.62-3.68 on the scale of 5. The MNRs of participants' level of understanding of a policy brief, a policy dialogue and the role of researchers in policy-making ranged from 3.50-3.86. The MNRs of participants' level of understanding of evidence in policy-making context, types and sources of evidence, capacity to identify, select, adapt, and transform relevant evidence into policy ranged from 3.63-4.08. The MNRs of the participants' organization's capacity to cover their geographical areas of operation were generally low ranging from 3.32-3.38 in terms of manpower, logistics, facilities, and external support. The lowest MNR of 2.66 was recorded in funding.The outcomes of this study suggest that a stakeholders' engagement event can serve as an important platform to assess policy-makers' knowledge and capacity for evidence-informed policy-making and for the promotion of evidence use in the policy process.

Project description:BackgroundCurrent incentive programmes are not sufficient to motivate researchers and policy-makers to use research evidence in policy-making. We conducted a mixed-methods design to identify context-based policy options for strengthening motivations among health researchers and policy-makers to support evidence-informed health policy-making (EIHP) in Iran.MethodsThis study was conducted in 2019 in two phases. In the first phase, we conducted a scoping review to extract interventions implemented or proposed to strengthen motivations to support EIHP. Additionally, we employed a comparative case study design for reviewing the performance evaluation (PE) processes in Iran and other selected countries to determine the current individual and organizational incentives to encourage EIHP. In the second phase, we developed two policy briefs and then convened two policy dialogues, with 12 and 8 key informants, respectively, where the briefs were discussed. Data were analysed using manifest content analysis in order to propose contextualized policy options.ResultsThe policy options identified to motivate health researchers and policy-makers to support EIHP in Iran were: revising the criteria of academic PE; designing appropriate incentive programmes for nonacademic researchers; developing an indicator for the evaluation of research impact on policy-making or health outcomes; revising the current policies of scientific journals; revising existing funding mechanisms; presenting the knowledge translation plan when submitting a research proposal, as a mandatory condition; encouraging and supporting mechanisms for increasing interactions between policy-makers and researchers; and revising some administrative processes (e.g. managers and staff PEs; selection, appointment, and changing managers and reward mechanisms).ConclusionsThe current individual or organizational incentives are mainly focused on publications, rather than encouraging researchers and policy-makers to support EIHP. Relying more on incentives that consider the other impacts of research (e.g. impacts on health system and policy, or health outcomes) is recommended. These incentives may encourage individuals and organizations to be more involved in conducting research evidence, resulting in promoting EIHP.Trial registrationNA.

Project description:BackgroundThe institutionalization of evidence-informed health policy-making (EIHP) is complex and complicated. It is complex because it has many players and is complicated because its institutionalization will require many changes that will be challenging to make. Like many other issues, strengthening EIHP needs a road map, which should consider challenges and address them through effective, harmonized and contextualized strategies. This study aims to develop a road map for enhancing EIHP in Iran based on steps of planning.MethodsThis study consisted of three phases: (1) identifying barriers to EIHP, (2) recognizing interventions and (3) measuring the use of evidence in Iran's health policy-making. A set of activities was established for conducting these, including foresight, systematic review and policy dialogue, to identify the current and potential barriers for the first phase. For the second phase, an evidence synthesis was performed through a scoping review, by searching the websites of benchmark institutions which had good examples of EIHP practices in order to extract and identify interventions, and through eight policy dialogues and two broad opinion polls to contextualize the list of interventions. Simultaneously, two qualitative-quantitative studies were conducted to design and use a tool for assessing EIHP in the third phase.ResultsWe identified 97 barriers to EIHP and categorized them into three groups, including 35 barriers on the "generation of evidence" (push side), 41 on the "use of evidence" (pull side) and 21 on the "interaction between these two" (exchange side). The list of 41 interventions identified through evidence synthesis and eight policy dialogues was reduced to 32 interventions after two expert opinion polling rounds. These interventions were classified into four main strategies for strengthening (1) the education and training system (6 interventions), (2) the incentives programmes (7 interventions), (3) the structure of policy support organizations (4 interventions) and (4) the enabling processes to support EIHP (15 interventions).ConclusionThe policy options developed in the study provide a comprehensive framework to chart a path for strengthening the country's EIHP considering both global practices and the context of Iran. It is recommended that operational plans be prepared for road map interventions, and the necessary resources provided for their implementation. The implementation of the road map will require attention to the principles of good governance, with a focus on transparency and accountability. Video abstract.

Project description:BackgroundPrevious surveys of organizations that support the development of evidence-informed health policies have focused on organizations that produce clinical practice guidelines (CPGs) or undertake health technology assessments (HTAs). Only rarely have surveys focused at least in part on units that directly support the use of research evidence in developing health policy on an international, national, and state or provincial level (i.e., government support units, or GSUs) that are in some way successful or innovative or that support the use of research evidence in low- and middle-income countries (LMICs).MethodsWe drew on many people and organizations around the world, including our project reference group, to generate a list of organizations to survey. We modified a questionnaire that had been developed originally by the Appraisal of Guidelines, Research and Evaluation in Europe (AGREE) collaboration and adapted one version of the questionnaire for organizations producing CPGs and HTAs, and another for GSUs. We sent the questionnaire by email to 176 organizations and followed up periodically with non-responders by email and telephone.ResultsWe received completed questionnaires from 152 (86%) organizations. More than one-half of the organizations (and particularly HTA agencies) reported that examples from other countries were helpful in establishing their organization. A higher proportion of GSUs than CPG- or HTA-producing organizations involved target users in the selection of topics or the services undertaken. Most organizations have few (five or fewer) full-time equivalent (FTE) staff. More than four-fifths of organizations reported providing panels with or using systematic reviews. GSUs tended to use a wide variety of explicit valuation processes for the research evidence, but none with the frequency that organizations producing CPGs, HTAs, or both prioritized evidence by its quality. Between one-half and two-thirds of organizations do not collect data systematically about uptake, and roughly the same proportions do not systematically evaluate their usefulness or impact in other ways.ConclusionThe findings from our survey, the most broadly based of its kind, both extend or clarify the applicability of the messages arising from previous surveys and related documentary analyses, such as how the 'principles of evidence-based medicine dominate current guideline programs' and the importance of collaborating with other organizations. The survey also provides a description of the history, structure, processes, outputs, and perceived strengths and weaknesses of existing organizations from which those establishing or leading similar organizations can draw.

Project description:BackgroundPrevious efforts to produce case descriptions have typically not focused on the organizations that produce research evidence and support its use. External evaluations of such organizations have typically not been analyzed as a group to identify the lessons that have emerged across multiple evaluations. Case descriptions offer the potential for capturing the views and experiences of many individuals who are familiar with an organization, including staff, advocates, and critics.MethodsWe purposively sampled a subgroup of organizations from among those that participated in the second (interview) phase of the study and (once) from among other organizations with which we were familiar. We developed and pilot-tested a case description data collection protocol, and conducted site visits that included both interviews and documentary analyses. Themes were identified from among responses to semi-structured questions using a constant comparative method of analysis. We produced both a brief (one to two pages) written description and a video documentary for each case.ResultsWe conducted 51 interviews as part of the eight site visits. Two organizational strengths were repeatedly cited by individuals participating in the site visits: use of an evidence-based approach (which was identified as being very time-consuming) and existence of a strong relationship between researchers and policymakers (which can be challenged by conflicts of interest). Two organizational weaknesses - a lack of resources and the presence of conflicts of interest - were repeatedly cited by individuals participating in the site visits. Participants offered two main suggestions for the World Health Organization (and other international organizations and networks): 1) mobilize one or more of government support, financial resources, and the participation of both policymakers and researchers; and 2) create knowledge-related global public goods.ConclusionThe findings from our case descriptions, the first of their kind, intersect in interesting ways with the messages arising from two systematic reviews of the factors that increase the prospects for research use in policymaking. Strong relationships between researchers and policymakers bodes well given such interactions appear to increase the prospects for research use. The time-consuming nature of an evidence-based approach, on the other hand, suggests the need for more efficient production processes that are 'quick and clean enough.' Our case descriptions and accompanying video documentaries provide a rich description of organizations supporting the use of research evidence, which can be drawn upon by those establishing or leading similar organizations, particularly in low- and middle-income countries.

Project description:BackgroundWhile calls for institutionalization of evidence-informed policy-making (EIP) have become stronger in recent years, there is a paucity of methods that governments and organizational knowledge brokers can use to sustain and integrate EIP as part of mainstream health policy-making. The objective of this paper was to conduct a knowledge synthesis of the published and grey literatures to develop a theoretical framework with the key features of EIP institutionalization.MethodsWe applied a critical interpretive synthesis (CIS) that allowed for a systematic, yet iterative and dynamic analysis of heterogeneous bodies of literature to develop an explanatory framework for EIP institutionalization. We used a "compass" question to create a detailed search strategy and conducted electronic searches to identify papers based on their potential relevance to EIP institutionalization. Papers were screened and extracted independently and in duplicate. A constant comparative method was applied to develop a framework on EIP institutionalization. The CIS was triangulated with the findings of stakeholder dialogues that involved civil servants, policy-makers and researchers.ResultsWe identified 3001 references, of which 88 papers met our eligibility criteria. This CIS resulted in a definition of EIP institutionalization as the "process and outcome of (re-)creating, maintaining and reinforcing norms, regulations, and standard practices that, based on collective meaning and values, actions as well as endowment of resources, allow evidence to become-over time-a legitimate and taken-for-granted part of health policy-making". The resulting theoretical framework comprised six key domains of EIP institutionalization that capture both structure and agency: (1) governance; (2) standards and routinized processes; (3) partnership, collective action and support; (4) leadership and commitment; (5) resources; and (6) culture. Furthermore, EIP institutionalization is being achieved through five overlapping stages: (i) precipitating events; (ii) de-institutionalization; (iii) semi-institutionalization (comprising theorization and diffusion); (iv) (re)-institutionalization; and (v) renewed de-institutionalization processes.ConclusionsThis CIS advances the theoretical and conceptual discussions on EIP institutionalization, and provides new insights into an evidence-informed framework for initiating, strengthening and/or assessing efforts to institutionalize EIP.

Dataset Information

The development of ORACLe: a measure of an organisation's capacity to engage in evidence-informed health policy.

Background

Method

Results

Conclusion

Publications

The development of ORACLe: a measure of an organisation's capacity to engage in evidence-informed health policy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets