Dataset Information

Evaluating and Using Observational Evidence: The Contrasting Views of Policy Makers and Epidemiologists.

ABSTRACT:

Background

Currently, little is known about the types of evidence used by policy makers. This study aimed to investigate how policy makers in the health domain use and evaluate evidence and how this differs from academic epidemiologists. By having a better understanding of how policy makers select, evaluate, and use evidence, academics can tailor the way in which that evidence is produced, potentially leading to more effective knowledge translation.

Methods

An exploratory mixed-methods study design was used. Quantitative measures were collected via an anonymous online survey (n?=?28), with sampling from three health-related government and non-government organizations. Semi-structured interviews with policy makers (n?=?20) and epidemiologists (n?=?6) were conducted to gather qualitative data.

Results

Policy makers indicated systematic reviews were the preferred research resource (19%), followed closely by qualitative research (16%). Neither policy makers nor epidemiologists used grading instruments to evaluate evidence. In the web survey, policy makers reported that consistency and strength of evidence (93%), the quality of data (93%), bias in the evidence (79%), and recency of evidence (79%) were the most important factors taken into consideration when evaluating the available evidence. The same results were found in the qualitative interviews. Epidemiologists focused on the methodology used in the study. The most cited barriers to using robust evidence, according to policy makers, were political considerations (60%), time limitations (55%), funding (50%), and research not being applicable to current policies (50%).

Conclusion

The policy maker's investigation did not report a systematic approach to evaluating evidence. Although there was some overlap between what policy makers and epidemiologists identified as high-quality evidence, there was also some important differences. This suggests that the best scientific evidence may not routinely be used in the development of policy. In essence, the policy-making process relied on other jurisdictions' policies and the opinions of internal staff members as primary evidence sources to inform policy decisions. Findings of this study suggest that efforts should be directed toward making scientific information more systematically available to policy makers.

SUBMITTER: O'Donoughue Jenkins L

PROVIDER: S-EPMC5138237 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluating and Using Observational Evidence: The Contrasting Views of Policy Makers and Epidemiologists.

O'Donoughue Jenkins Lily L Kelly Paul M PM Cherbuin Nicolas N Anstey Kaarin J KJ

Frontiers in public health 20161206

<h4>Background</h4>Currently, little is known about the types of evidence used by policy makers. This study aimed to investigate how policy makers in the health domain use and evaluate evidence and how this differs from academic epidemiologists. By having a better understanding of how policy makers select, evaluate, and use evidence, academics can tailor the way in which that evidence is produced, potentially leading to more effective knowledge translation.<h4>Methods</h4>An exploratory mixed-me ...[more]

PMID: 27999772

Similar Datasets

Project description:BackgroundOpportunities exist to disseminate evidence-based cancer control strategies to state-level policy makers in both the legislative and executive branches. We explored factors that influence the likelihood that state-level policy makers will find a policy brief understandable, credible, and useful.MethodsA systematic approach was used to develop four types of policy briefs on the topic of mammography screening to reduce breast cancer mortality: data-focused brief with state-level data, data-focused brief with local-level data, story-focused brief with state-level data, and story-focused brief with local-level data. Participants were recruited from three groups of state-level policy makers-legislative staff, legislators, and executive branch administrators- in six states that were randomly chosen after stratifying all 50 states by population size and dominant political party in state legislature. Participants from each of the three policy groups were randomly assigned to receive one of the four types of policy briefs and completed a questionnaire that included a series of Likert scale items. Primary outcomes-whether the brief was understandable, credible, likely to be used, and likely to be shared-were measured by a 5-point Likert scale according to the degree of agreement (1 = strongly disagree, 5 = strongly agree). Data were analyzed with analysis of variance and with classification trees. All statistical tests were two-sided.ResultsData on response to the policy briefs (n = 291) were collected from February through December 2009 (overall response rate = 35%). All three policy groups found the briefs to be understandable and credible, with mean ratings that ranged from 4.3 to 4.5. The likelihood of using the brief (the dependent variable) differed statistically significantly by study condition for staffers (P = .041) and for legislators (P = .018). Staffers found the story-focused brief containing state-level data most useful, whereas legislators found the data-focused brief containing state-level data most useful. Exploratory classification trees showed distinctive patterns for brief usefulness across the three policy groups.ConclusionOur results suggest that taking a "one-size-fits-all" approach when delivering information to policy makers may be less effective than communicating information based on the type of policy maker.

Project description:IntroductionMost studies have reported barriers to guideline usage mainly from doctors' perspective; few have reported the perspective of other stakeholders. This study aimed to determine the views and barriers to adherence of a national clinical practice guideline (CPG) on management of hypertension from the perspectives of policymakers, doctors and allied healthcare professionals.MethodsThis study used a qualitative approach with purposive sampling. Seven in depth interviews and six focus group discussions were conducted with 35 healthcare professionals (policy makers, doctors, pharmacists and nurses) at a teaching hospital in Kuala Lumpur, Malaysia, between February and June 2013. All interviews were audio-recorded, transcribed verbatim and checked. Thematic approach was used to analyse the data.ResultsTwo main themes and three sub-themes emerged from this study. The main themes were (1) variation in the use of CPG and (2) barriers to adherence to CPG. The three sub-themes for barriers were issues inherent to the CPG, systems and policy that is not supportive of CPG use, and attitudes and behaviour of stakeholders. The main users of the CPG were the primary care doctors. Pharmacists only partially use the guidelines, while nurses and policy makers were not using the CPG at all. Participants had suggested few strategies to improve usage and adherence to CPG. First, update the CPG regularly and keep its content simple with specific sections for allied health workers. Second, use technology to facilitate CPG accessibility and provide protected time for implementation of CPG recommendations. Third, incorporate local CPG in professional training, link CPG adherence to key performance indicators and provide incentives for its use.ConclusionsBarriers to the use of CPG hypertension management span across all stakeholders. The development and implementation of CPG focused mainly on doctors with lack of involvement of other healthcare stakeholders. Guidelines should be made simple, current, reliable, accessible, inclusive of all stakeholders and with good policy support.

Project description:BackgroundKnowledge Translation (KT) and data visualization play a vital role in the dissemination of data and information to improve healthcare systems. A better understanding of KT and its utility requires examining its processes, and how these interact with available data tools and platforms and various users. In this context, the aim of this paper is to understand how relevant users interact with data visualization tools, in particular Global Burden of Disease (GBD) visualizations, while also examining KT processes related to data visualization.MethodsA qualitative case-study consisting of semi-structured interviews with eight policy officers. Interviewees were selected by the Institute for Health Metrics and Evaluation (IHME) from three countries: Canada, Kenya and New Zealand. Data were analyzed through framework coding, using qualitative analysis software.ResultsPolicy officers' responses indicated that data can prompt action by engaging users, and effective delivery and translation of data was enhanced by data visualization tools. GBD was considered valuable for use in policy (e.g., planning and prioritizing health policy; facilitating accountability; and tracking and monitoring progress and trends over time and between countries). Using GBD and data visualization tools, participants quickly and easily accessed key information and turned it into actionable messages; engaging visuals captured decision-makers' attention while providing information in a digestible, time-saving manner. However, participants emphasized an overall disconnect between research and public health. Functionality and technical issues, e.g., absence of tool guides and tool complexity, as well as lacking buy-in and awareness of certain tools from those less familiar with GBD, were named as major barriers. In order to address this "know-do" gap, user-friendly knowledge translation tools were stated as crucial, as was the importance of collaboration and leveraging different insights from data generators and users to inform health policy.ConclusionsPolicy officers aware of KT are willing to utilize data visualization tools as they value them as an engaging and important mechanism to foster the use of GBD data in policy-making. To further facilitate policy officers' efforts to effectively use GBD data in policy and practice, further research is required into how users perceive and interact with data visualization and other KT tools.

Project description:BACKGROUND:Providing healthcare professionals with health surveillance data aims to support professional and organisational behaviour change. The UK Five Year Antimicrobial Resistance (AMR) Strategy 2013 to 2018 identified better access to and use of surveillance data as a key component. Our aim was to determine the extent to which data on antimicrobial use and resistance met the perceived needs of health care professionals and policy-makers at national, regional and local levels, and how provision could be improved. METHODS:We conducted 41 semi-structured interviews with national policy makers in the four Devolved Administrations and 71 interviews with health care professionals in six locations across the United Kingdom selected to achieve maximum variation in terms of population and health system characteristics. Transcripts were analysed thematically using a mix of a priori reasoning guided by the main topics in the interview guide together with themes emerging inductively from the data. Views were considered at three levels - primary care, secondary care and national - and in terms of availability of data, current uses, benefits, gaps and potential improvements. RESULTS:Respondents described a range of uses for prescribing and resistance data. The principal gaps identified were prescribing in private practice, internet prescribing and secondary care (where some hospitals did not have electronic prescribing systems). Some respondents under-estimated the range of data available. There was a perception that the responsibility for collecting and analysing data often rests with a few individuals who may lack sufficient time and appropriate skills. CONCLUSIONS:There is a need to raise awareness of data availability and the potential value of these data, and to ensure that data systems are more accessible. Any skills gap at local level in how to process and use data needs to be addressed. This requires an identification of the best methods to improve support and education relating to AMR data systems.

Project description:BackgroundA growing body of evidence shows that mobile health (mHealth) interventions may improve treatment and care for the rapidly rising number of patients with noncommunicable diseases (NCDs) in sub-Saharan Africa (SSA). A recent realist review developed a framework highlighting the influence of context factors, including predisposing characteristics, needs, and enabling resources (PNE), for the long-term success of mHealth interventions. The views of policy makers will ultimately determine implementation and scale-up of mHealth interventions in SSA. However, their views about necessary conditions for sustainability and scale-up remain unexplored.ObjectiveThis study aimed to understand the views of policy makers in Ghana with regard to the most important factors for successful implementation, sustainability, and scale-up of mHealth NCD interventions.MethodsMembers of the technical working group responsible for Ghana's national NCD policy were interviewed about their knowledge of and attitude toward mHealth and about the most important factors contributing to long-term intervention success. Using qualitative methods and applying a qualitative content analysis approach, answers were categorized according to the PNE framework.ResultsA total of 19 policy makers were contacted and 13 were interviewed. Interviewees had long-standing work experience of an average of 26 years and were actively involved in health policy making in Ghana. They were well-informed about the potential of mHealth, and they strongly supported mHealth expansion in the country. Guided by the PNE framework's categories, the policy makers ascertained which critical factors would support the successful implementation of mHealth interventions in Ghana. The policy makers mentioned many factors described in the literature as important for mHealth implementation, sustainability, and scale-up, but they focused more on enabling resources than on predisposing characteristics and need. Furthermore, they mentioned several factors that have been rather unexplored in the literature.ConclusionsThe study shows that the PNE framework is useful to guide policy makers toward a more systematic assessment of context factors that support intervention implementation, sustainability, and scale-up. Furthermore, the framework was refined by adding additional factors. Policy makers may benefit from using the PNE framework at the various stages of mHealth implementation. Researchers may (and should) use the framework when investigating reasons for success (or failure) of interventions.

Project description:BackgroundThe use of health policy and systems research (HPSR) to inform health policy-making is an international challenge. Incorporating HPSR into decision-making primarily involves two groups, namely researchers (knowledge producers) and policy-makers (knowledge users). The purpose of this study was to compare the perceptions of Israeli health systems and policy researchers and health services policy-makers regarding the role of HPSR, factors influencing its uses and potential facilitators and barriers to HPSR, and implementation of knowledge transfer and exchange (KTE) activities.MethodsA cross-sectional survey was administered to researchers and policy-makers in Israel. The survey consisted of seven closed questions. Descriptive analyses were carried out for closed-ended questions and comparative analysis were conducted between groups using the χ2 test.ResultsA total of 37 researchers and 32 policy-makers responded to the survey. While some views were in alignment, others showed differences. More policy-makers than researchers perceived that the use of HPSR in policy was hindered by practical implementation constraints, whereas more researchers felt that its use was hindered by a lack of coordination between knowledge producers and users. A larger percentage of policy-makers, as compared to researchers, reported that facilitators to the KTE process are in place and a larger percentage of researchers perceived barriers within the KTE environment. A larger percentage of policy-makers perceived KTE activities were in place as compared to researchers. Results also showed large differences in the perceptions of the two groups regarding policy formulation and which organisations they perceived as exerting strong influence on policy-making.ConclusionsThis research demonstrated that there are differences in the perceptions of knowledge producers and users about the process of KTE. Future work should focus on minimising the challenges highlighted here and implementing new KTE activities. These activities could include making the researchers aware of the most effective manner in which to package their results, providing training to policy-makers and assuring that policy-makers have technical access to appropriate databases to search for HPSR. These results underscore the need for the groups to communicate and clarify to each other what they can offer and what they require.

Project description:There is increasing interest globally in the use of more rigorous processes to ensure that maternal, newborn, and child health (MNCH) care recommendations are informed by the best available research evidence use. The purpose of this study was to engage Nigerian MNCH policy-makers and other stakeholders to consider issues around research to policy and practice interface and to assess their existing knowledge and capacity on the use of research evidence for policy-making and practice.The study design is a cross-sectional evaluation of MNCH stakeholders' knowledge as it pertains different dimensions of research to practice. This was undertaken during a national MNCH stakeholders' engagement event convened under the auspices of the West African Health Organization (WAHO) and the Federal Ministry of Health (FMoH) in Abuja, Nigeria. A questionnaire was administered to participants, which was designed to assess participants' knowledge, capacity and organizational process of generation, synthesis and utilization of research evidence in policy-making regarding MNCH.A total of 40 participants signed the informed consent form and completed the questionnaire. The mean ratings (MNRs) of participants' knowledge of electronic databases and capacity to identify and obtain relevant research evidence from electronic databases ranged from 3.62-3.68 on the scale of 5. The MNRs of participants' level of understanding of a policy brief, a policy dialogue and the role of researchers in policy-making ranged from 3.50-3.86. The MNRs of participants' level of understanding of evidence in policy-making context, types and sources of evidence, capacity to identify, select, adapt, and transform relevant evidence into policy ranged from 3.63-4.08. The MNRs of the participants' organization's capacity to cover their geographical areas of operation were generally low ranging from 3.32-3.38 in terms of manpower, logistics, facilities, and external support. The lowest MNR of 2.66 was recorded in funding.The outcomes of this study suggest that a stakeholders' engagement event can serve as an important platform to assess policy-makers' knowledge and capacity for evidence-informed policy-making and for the promotion of evidence use in the policy process.

Dataset Information

Evaluating and Using Observational Evidence: The Contrasting Views of Policy Makers and Epidemiologists.

Background

Methods

Results

Conclusion

Publications

Evaluating and Using Observational Evidence: The Contrasting Views of Policy Makers and Epidemiologists.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets