Dataset Information

Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study.

ABSTRACT:

Background

Systematic reviews are infrequently used by health care managers (HCMs) and policy-makers (PMs) in decision-making. HCMs and PMs co-developed and tested novel systematic review of effects formats to increase their use.

Methods

A three-phased approach was used to evaluate the determinants to uptake of systematic reviews of effects and the usability of an innovative and a traditional systematic review of effects format. In phase 1, survey and interviews were conducted with HCMs and PMs in four Canadian provinces to determine perceptions of a traditional systematic review format. In phase 2, systematic review format prototypes were created by HCMs and PMs via Conceptboard©. In phase 3, prototypes underwent usability testing by HCMs and PMs.

Results

Two hundred two participants (80 HCMs, 122 PMs) completed the phase 1 survey. Respondents reported that inadequate format (Mdn?=?4; IQR?=?4; range?=?1-7) and content (Mdn?=?4; IQR?=?3; range?=?1-7) influenced their use of systematic reviews. Most respondents (76%; n?=?136/180) reported they would be more likely to use systematic reviews if the format was modified. Findings from 11 interviews (5 HCMs, 6 PMs) revealed that participants preferred systematic reviews of effects that were easy to access and read and provided more information on intervention effectiveness and less information on review methodology. The mean System Usability Scale (SUS) score was 55.7 (standard deviation [SD] 17.2) for the traditional format; a SUS score?ConclusionsHCMs and PMs co-created prototypes for systematic review of effects formats based on their needs. The prototypes will be compared to a traditional format in a randomized trial.

SUBMITTER: Marquez C

PROVIDER: S-EPMC6014014 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study.

Marquez Christine C Johnson Alekhya Mascarenhas AM Jassemi Sabrina S Park Jamie J Moore Julia E JE Blaine Caroline C Bourdon Gertrude G Chignell Mark M Ellen Moriah E ME Fortin Jacques J Graham Ian D ID Hayes Anne A Hamid Jemila J Hemmelgarn Brenda B Hillmer Michael M Holmes Bev B Holroyd-Leduc Jayna J Hubert Linda L Hutton Brian B Kastner Monika M Lavis John N JN Michell Karen K Moher David D Ouimet Mathieu M Perrier Laure L Proctor Andrea A Noseworthy Thomas T Schuckel Victoria V Stayberg Sharlene S Tonelli Marcello M Tricco Andrea C AC Straus Sharon E SE

Implementation science : IS 20180622 1

<h4>Background</h4>Systematic reviews are infrequently used by health care managers (HCMs) and policy-makers (PMs) in decision-making. HCMs and PMs co-developed and tested novel systematic review of effects formats to increase their use.<h4>Methods</h4>A three-phased approach was used to evaluate the determinants to uptake of systematic reviews of effects and the usability of an innovative and a traditional systematic review of effects format. In phase 1, survey and interviews were conducted wit ...[more]

PMID: 29929538

Similar Datasets

Project description:BackgroundRapid reviews are increasingly used by policy agencies to access relevant research in short timeframes. Despite the growing number of programmes, little is known about how rapid reviews are used by health policy agencies. This study examined whether and how rapid reviews commissioned using a knowledge brokering programme were used by Australian policy-makers.MethodsThis study used interview data to examine the use of 139 rapid reviews by health policy agencies that were commissioned between 2006 and 2015. Transcripts were coded to identify how rapid reviews were used, the type of policy processes in which they were used, what evidence of use was provided and what reasons were given when rapid reviews were not used. Fisher's exact test was used to assess variation between types of agencies.ResultsOverall, 89% of commissioned rapid reviews were used by the commissioning agencies and 338 separate instances of use were identified, namely, on average, three uses per review. Policy-makers used reviews primarily to determine the details of a policy or programme, identify priorities for future action or investment, negotiate interjurisdictional decisions, evaluate alternative solutions for a policy problem, and communicate information to stakeholders. Some variation in use was observed across agencies. Reasons for non-use were related to changes in organisational structures, resources or key personnel in the commissioning agencies, or changes in the broader political environment.ConclusionsThis study found that almost all rapid reviews had been used by the agencies who commissioned them, primarily in policy and programme development, agenda-setting, and to communicate information to stakeholders. Reviews were used mostly in instrumental and conceptual ways and there was little evidence of symbolic use. Variations in use were identified across agencies. The findings suggest that commissioned rapid reviews are an effective means of providing timely relevant research for use in policy processes and that review findings may be applied in a variety of ways.

Project description:BackgroundThe use of research evidence as an input for health decision-making is a need for most health systems. There are a number of approaches for promoting evidence use at different levels of the health system, but knowledge of their effectiveness is still scarce. The objective of this overview was to evaluate the effectiveness of knowledge communication and dissemination interventions, strategies or approaches targeting policy-makers and health managers.MethodsThis overview of systematic reviews used systematic review methods and was conducted according to a predefined and published protocol. A comprehensive electronic search of 13 databases and a manual search in four websites were conducted. Both published and unpublished reviews in English, Spanish or Portuguese were included. A narrative synthesis was undertaken, and effectiveness statements were developed, informed by the evidence identified.ResultsWe included 27 systematic reviews. Three studies included only a communication strategy, while eight only included dissemination strategies, and the remaining 16 included both. None of the selected reviews provided "sufficient evidence" for any of the strategies, while four provided some evidence for three communication and four dissemination strategies. Regarding communication strategies, the use of tailored and targeted messages seemed to successfully lead to changes in the decision-making practices of the target audience. Regarding dissemination strategies, interventions that aimed at improving only the reach of evidence did not have an impact on its use in decisions, while interventions aimed at enhancing users' ability to use and apply evidence had a positive effect on decision-making processes. Multifaceted dissemination strategies also demonstrated the potential for changing knowledge about evidence but not its implementation in decision-making.ConclusionsThere is limited evidence regarding the effectiveness of interventions targeting health managers and policy-makers, as well as the mechanisms required for achieving impact. More studies are needed that are informed by theoretical frameworks or specific tools and using robust methods, standardized outcome measures and clear descriptions of the interventions. We found that passive communication increased access to evidence but had no effect on uptake. Some evidence indicated that the use of targeted messages, knowledge-brokering and user training was effective in promoting evidence use by managers and policy-makers.

Project description:BACKGROUND:It is unclear how to engage a wide range of knowledge users in research. We aimed to map the evidence on engaging knowledge users with an emphasis on policy-makers, health system managers, and policy analysts in the knowledge synthesis process through a scoping review. METHODS:We used the Joanna Briggs Institute guidance for scoping reviews. Nine electronic databases (e.g., MEDLINE), two grey literature sources (e.g., OpenSIGLE), and reference lists of relevant systematic reviews were searched from 1996 to August 2016. We included any type of study describing strategies, barriers and facilitators, or assessing the impact of engaging policy-makers, health system managers, and policy analysts in the knowledge synthesis process. Screening and data abstraction were conducted by two reviewers independently with a third reviewer resolving discrepancies. Frequency and thematic analyses were conducted. RESULTS:After screening 8395 titles and abstracts followed by 394 full-texts, 84 unique documents and 7 companion reports fulfilled our eligibility criteria. All 84 documents were published in the last 10 years, and half were prepared in North America. The most common type of knowledge synthesis with knowledge user engagement was a systematic review (36%). The knowledge synthesis most commonly addressed an issue at the level of national healthcare system (48%) and focused on health services delivery (17%) in high-income countries (86%). Policy-makers were the most common (64%) knowledge users, followed by healthcare professionals (49%) and government agencies as well as patients and caregivers (34%). Knowledge users were engaged in conceptualization and design (49%), literature search and data collection (52%), data synthesis and interpretation (71%), and knowledge dissemination and application (44%). Knowledge users were most commonly engaged as key informants through meetings and workshops as well as surveys, focus groups, and interviews either in-person or by telephone and emails. Knowledge user content expertise/awareness was a common facilitator (18%), while lack of time or opportunity to participate was a common barrier (12%). CONCLUSIONS:Knowledge users were most commonly engaged during the data synthesis and interpretation phases of the knowledge synthesis conduct. Researchers should document and evaluate knowledge user engagement in knowledge synthesis. REGISTRATION DETAILS:Open Science Framework ( https://osf.io/4dy53/ ).

Project description:BackgroundSystems science methods such as dynamic simulation modelling are well suited to address questions about public health policy as they consider the complexity, context and dynamic nature of system-wide behaviours. Advances in technology have led to increased accessibility and interest in systems methods to address complex health policy issues. However, the involvement of policy decision makers in health-related simulation model development has been lacking. Where end-users have been included, there has been limited examination of their experience of the participatory modelling process and their views about the utility of the findings. This paper reports the experience of end-user decision makers, including senior public health policy makers and health service providers, who participated in three participatory simulation modelling for health policy case studies (alcohol related harm, childhood obesity prevention, diabetes in pregnancy), and their perceptions of the value and efficacy of this method in an applied health sector context.MethodsSemi-structured interviews were conducted with end-user participants from three participatory simulation modelling case studies in Australian real-world policy settings. Interviewees were employees of government agencies with jurisdiction over policy and program decisions and were purposively selected to include perspectives at different stages of model development.ResultsThe 'co-production' aspect of the participatory approach was highly valued. It was reported as an essential component of building understanding of the modelling process, and thus trust in the model and its outputs as a decision-support tool. The unique benefits of simulation modelling included its capacity to explore interactions of risk factors and combined interventions, and the impact of scaling up interventions. Participants also valued simulating new interventions prior to implementation in the real world, and the comprehensive mapping of evidence and its gaps to prioritise future research. The participatory aspect of simulation modelling was time and resource intensive and therefore most suited to high priority complex topics with contested options for intervening.ConclusionThese findings highlight the value of a participatory approach to dynamic simulation modelling to support its utility in applied health policy settings.

Project description:Background Rapid reviews (RRs) are useful products to healthcare policy-makers and other stakeholders, who require timely evidence. Therefore, it is important to assess how well RRs convey useful information in a format that is easy to understand so that decision-makers can make best use of evidence to inform policy and practice. Methods We assessed a diverse sample of 103 RRs against the BRIDGE criteria, originally developed for communicating clearly to support healthcare policy-making. We modified the criteria to increase assessability and to align with RRs. We identified RRs from key database searches and through searching organisations known to produce RRs. We assessed each RR on 26 factors (e.g. organisation of information, lay language use). Results were descriptively analysed. Further, we explored differences between RRs published in journals and those published elsewhere. Results Certain criteria were well covered across the RRs (e.g. all aimed to synthesise research evidence and all provided references of included studies). Further, most RRs provided detail on the problem or issue (96%; n =?99) and described methods to conduct the RR (91%; n =?94), while several addressed political or health systems contexts (61%; n =?63). Many RRs targeted policy-makers and key stakeholders as the intended audience (66%; n =?68), yet only 32% (n =?33) involved their tacit knowledge, while fewer (27%; n =?28) directly involved them reviewing the content of the RR. Only six RRs involved patient partners in the process. Only 23% (n =?24) of RRs were prepared in a format considered to make information easy to absorb (i.e. graded entry) and 25% (n =?26) provided specific key messages. Readability assessment indicated that the text of key RR sections would be hard to understand for an average reader (i.e. would require post-secondary education) and would take 42 (± 36) minutes to read. Conclusions Overall, conformity of the RRs with the modified BRIDGE criteria was modest. By assessing RRs against these criteria, we now understand possible ways in which they could be improved to better meet the information needs of healthcare decision-makers and their potential for innovation as an information-packaging mechanism. The utility and validity of these items should be further explored. Protocol availability The protocol, published on the Open Science Framework, is available at: osf.io/68tj7

Project description:BackgroundThe World Health Organization (WHO) stresses the importance of active case-finding (ACF) for early detection of tuberculosis (TB), especially in the 30 high-burden countries that account for almost 90% of cases globally.ObjectiveTo describe the attitudes of National TB Programme (NTP) managers related to ACF policy development, implementation and scale-up in the 30 high-burden countries, and to review national TB strategic plans.MethodsThis was a mixed-methods study with an embedded design: A cross-sectional survey with NTP managers yielded quantitative and qualitative data. A review of national TB strategic plans complemented the results. All data were analyzed in parallel and merged in the interpretation of the findings.Results23 of the 30 NTP managers (77%) participated in the survey and 22 (73%) national TB strategic plans were reviewed. NTP managers considered managers in districts and regions key stakeholders for both ACF policy development and implementation. Different types of evidence were used to inform ACF policy, while there was a particular demand for local evidence. The NSPs reflected the NTP managers' unanimous agreement on the need for ACF scale-up, but not all included explicit aims and targets related to ACF. The NTP managers recognized that ACF may decrease health systems costs in the long-term, while acknowledging the risk for increased health system costs in the short-term. About 90% of the NTP managers declared that financial and human resources were currently lacking, while they also elaborated on strategies to overcome resource constraints.ConclusionNTP managers stated that ACF should be scaled up but reported resource constraints. Strategies to increase resources exist but may not yet have been fully implemented, e.g. generating local evidence including from operational research for advocacy. Managers in districts and regions were identified as key stakeholders whose involvement could help improve ACF policy development, implementation and scale-up.

Dataset Information

Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study.

Background

Methods

Results

Publications

Enhancing the uptake of systematic reviews of effects: what is the best format for health care managers and policy-makers? A mixed-methods study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure