Dataset Information

The SPARK Tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation.

ABSTRACT:

Background

Groups or institutions funding or conducting systematic reviews in health policy and systems research (HPSR) should prioritise topics according to the needs of policymakers and stakeholders. The aim of this study was to develop and validate a tool to prioritise questions for systematic reviews in HPSR.

Methods

We developed the tool following a four-step approach consisting of (1) the definition of the purpose and scope of tool, (2) item generation and reduction, (3) testing for content and face validity, (4) and pilot testing of the tool. The research team involved international experts in HPSR, systematic review methodology and tool development, led by the Center for Systematic Reviews on Health Policy and Systems Research (SPARK). We followed an inclusive approach in determining the final selection of items to allow customisation to the user's needs.

Results

The purpose of the SPARK tool was to prioritise questions in HPSR in order to address them in systematic reviews. In the item generation and reduction phase, an extensive literature search yielded 40 relevant articles, which were reviewed by the research team to create a preliminary list of 19 candidate items for inclusion in the tool. As part of testing for content and face validity, input from international experts led to the refining, changing, merging and addition of new items, and to organisation of the tool into two modules. Following pilot testing, we finalised the tool, with 22 items organised in two modules - the first module including 13 items to be rated by policymakers and stakeholders, and the second including 9 items to be rated by systematic review teams. Users can customise the tool to their needs, by omitting items that may not be applicable to their settings. We also developed a user manual that provides guidance on how to use the SPARK tool, along with signaling questions.

Conclusion

We have developed and conducted initial validation of the SPARK tool to prioritise questions for systematic reviews in HPSR, along with a user manual. By aligning systematic review production to policy priorities, the tool will help support evidence-informed policymaking and reduce research waste. We invite others to contribute with additional real-life implementation of the tool.

SUBMITTER: Akl EA

PROVIDER: S-EPMC5583759 | biostudies-literature | 2017 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The SPARK Tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation.

Akl Elie A EA Fadlallah Racha R Ghandour Lilian L Kdouh Ola O Langlois Etienne E Lavis John N JN Schünemann Holger H El-Jardali Fadi F

Health research policy and systems 20170904 1

<h4>Background</h4>Groups or institutions funding or conducting systematic reviews in health policy and systems research (HPSR) should prioritise topics according to the needs of policymakers and stakeholders. The aim of this study was to develop and validate a tool to prioritise questions for systematic reviews in HPSR.<h4>Methods</h4>We developed the tool following a four-step approach consisting of (1) the definition of the purpose and scope of tool, (2) item generation and reduction, (3) tes ...[more]

PMID: 28870215

Similar Datasets

Project description:BackgroundConversations about research priorities with members of the public are rarely designed specifically to include people who identify as Lesbian, Gay, Bisexual, Transgender and Queer (LGBTQ+) and are not researchers.MethodsGenerally, to address this gap, and specifically, to inform future research for CLS, we carried out a rapid review of published research priority sets covering LGBTQ+ topics, and an online workshop to prioritise identified themes.ResultsRapid review: results. The rapid review identified 18 LGBTQ+ research priority sets. Some focussed on specific populations such as women or men, younger or older people or people living within families. Five addressed transgender and gender non- conforming populations. All of the research priority sets originated from English-speaking, high and middle-income countries (UK, US, Canada, and Australia), and date from 2016 onwards. Prioritization approaches were wide-ranging from personal commentary to expert workshops and surveys. Participants involved in setting priorities mostly included research academics, health practitioners and advocacy organisations, two studies involved LGBTQ+ public in their process. Research priorities identified in this review were then grouped into themes which were prioritised during the workshop. Workshop: results. For the workshop, participants were recruited using local (Cambridge, UK) LGBTQ+ networks and a national advert to a public involvement in research matching website to take part in an online discussion workshop. Those that took part were offered payment for their time in preparing for the workshop and taking part. Participants personal priorities and experiences contributed to a consensus development process and a final ranked list of seven research themes and participants' experiences of healthcare, mental health advocacy, care homes, caring responsibilities, schools and family units added additional context.ConclusionsFrom the workshop the three research themes prioritised were: healthcare services delivery, prevention, and particular challenges / intersectionality of multiple challenges for people identifying as LGBTQ+. Research themes interconnected in many ways and this was demonstrated by the comments from workshop participants. This paper offers insights into why these priorities were important from participants' perspectives and detail about how to run an inclusive and respectful public involvement research exercise. On a practical level these themes will directly inform future research direction for CLS.

Project description:BackgroundSystematic identification of evidence in health policy can be time-consuming and challenging. This study examines three questions pertaining to systematic reviews on obesity prevention policy, in order to identify the most efficient search methods: (1) What percentage of the primary studies selected for inclusion in the reviews originated in scholarly as opposed to gray literature? (2) How much of the primary scholarly literature in this topic area is indexed in PubMed/MEDLINE? (3) Which databases index the greatest number of primary studies not indexed in PubMed, and are these databases searched consistently across systematic reviews?MethodsWe identified systematic reviews on obesity prevention policy and explored their search methods and citations. We determined the percentage of scholarly vs. gray literature cited, the most frequently cited journals, and whether each primary study was indexed in PubMed. We searched 21 databases for all primary study articles not indexed in PubMed to determine which database(s) indexed the highest number of these relevant articles.ResultsIn total, 21 systematic reviews were identified. Ten of the 21 systematic reviews reported searching gray literature, and 12 reviews ultimately included gray literature in their analyses. Scholarly articles accounted for 577 of the 649 total primary study papers. Of these, 495 (76%) were indexed in PubMed. Google Scholar retrieved the highest number of the remaining 82 non-PubMed scholarly articles, followed by Scopus and EconLit. The Journal of the American Dietetic Association was the most-cited journal.ConclusionsResearchers can maximize search efficiency by searching a small yet targeted selection of both scholarly and gray literature resources. A highly sensitive search of PubMed and those databases that index the greatest number of relevant articles not indexed in PubMed, namely multidisciplinary and economics databases, could save considerable time and effort. When combined with a gray literature search and additional search methods, including cited reference searching and consulting with experts, this approach could help maintain broad retrieval of relevant studies while improving search efficiency. Findings also have implications for designing specialized databases for public health research.

Project description:BackgroundSystematic reviews are increasingly used to inform health policy-making. The conflicts of interest (COI) of the authors of systematic reviews may bias their results and influence their conclusions. This may in turn lead to misguided public policies and systems level decisions. In order to mitigate the adverse impact of COI, scientific journals require authors to disclose their COIs. The objective of this study was to assess the frequency and different types of COI that authors of systematic reviews on health policy and systems research (HSPR) report.MethodsWe conducted a cross sectional survey. We searched the Health Systems Evidence (HSE) database of McMaster Health Forum for systematic reviews published in 2015. We extracted information regarding the characteristics of the systematic reviews and the associated COI disclosures. We conducted descriptive analyses.ResultsEighty percent of systematic reviews included authors' COI disclosures. Of the 160 systematic reviews that included COI disclosures, 15% had at least one author reporting at least one type of COI. The two most frequently reported types of COI were individual financial COI and individual scholarly COI (11% and 4% respectively). Institutional COIs were less commonly reported than individual COIs (3% and 15% respectively) and non-financial COIs were less commonly reported than financial COIs (6% and 14% respectively). Only one systematic review reported the COI disclosure by editors, and none reported disclosure by peer reviewers. All COI disclosures were in the form of a narrative statement in the main document and none in an online document.ConclusionA fifth of systematic reviews in HPSR do not include a COI disclosure statement, highlighting the need for journals to strengthen and/or better implement their COI disclosure policies. While only 15% of identified disclosure statements report any COI, it is not clear whether this indicates a low frequency of COI versus an underreporting of COI, or both.

Project description:Integrating knowledge from across the natural and social sciences is necessary to effectively address societal tradeoffs between human use of biological diversity and its preservation. Collaborative processes can change the ways decision makers think about scientific evidence, enhance levels of mutual trust and credibility, and advance the conservation policy discourse. Canada has responsibility for a large fraction of some major ecosystems, such as boreal forests, Arctic tundra, wetlands, and temperate and Arctic oceans. Stressors to biological diversity within these ecosystems arise from activities of the country's resource-based economy, as well as external drivers of environmental change. Effective management is complicated by incongruence between ecological and political boundaries and conflicting perspectives on social and economic goals. Many knowledge gaps about stressors and their management might be reduced through targeted, timely research. We identify 40 questions that, if addressed or answered, would advance research that has a high probability of supporting development of effective policies and management strategies for species, ecosystems, and ecological processes in Canada. A total of 396 candidate questions drawn from natural and social science disciplines were contributed by individuals with diverse organizational affiliations. These were collaboratively winnowed to 40 by our team of collaborators. The questions emphasize understanding ecosystems, the effects and mitigation of climate change, coordinating governance and management efforts across multiple jurisdictions, and examining relations between conservation policy and the social and economic well-being of Aboriginal peoples. The questions we identified provide potential links between evidence from the conservation sciences and formulation of policies for conservation and resource management. Our collaborative process of communication and engagement between scientists and decision makers for generating and prioritizing research questions at a national level could be a model for similar efforts beyond Canada.

Project description:BACKGROUND:There is increasing interest in using systems thinking to tackle 'wicked' policy problems in preventive health, but this can be challenging for policy-makers because the literature is amorphous and often highly theoretical. Little is known about how best to support health policy-makers to gain skills in understanding and applying systems thinking for policy action. METHODS:In-depth interviews were conducted with 18 policy-makers who are participating in an Australian research collaboration that uses a systems approach. Our aim was to explore factors that support policy-makers to use systems approaches, and to identify any impacts of systems thinking on policy thinking or action, including the pathways through which these impacts occurred. RESULTS:All 18 policy-makers agreed that systems thinking has merit but some questioned its practical policy utility. A small minority were confused about what systems thinking is or which approaches were being used in the collaboration. The majority were engaged with systems thinking and this group identified concrete impacts on their work. They reported using systems-focused research, ideas, tools and resources in policy work that were contributing to the development of practical methodologies for policy design, scaling up, implementation and evaluation; and to new prevention narratives. Importantly, systems thinking was helping some policy-makers to reconceptualise health problems and contexts, goals, potential policy solutions and methods. In short, they were changing how they think about preventive health. CONCLUSION:These results show that researchers and policy-makers can put systems thinking into action as part of a research collaboration, and that this can result in discernible impacts on policy processes. In this case, action-oriented collaboration and capacity development over a 5-year period facilitated mutual learning and practical application. This indicates that policy-makers can get substantial applied value from systems thinking when they are involved in extended co-production processes that target policy impact and are supported by responsive capacity strategies.

Project description:BackgroundIt is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems.ObjectiveThis study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital's computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems.MethodsThe Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability.ResultsThe participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals' knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity.ConclusionsThis initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.

Dataset Information

The SPARK Tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation.

Background

Methods

Results

Conclusion

Publications

The SPARK Tool to prioritise questions for systematic reviews in health policy and systems research: development and initial validation.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure