Dataset Information

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines.

ABSTRACT:

Background

The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate 'lumping' of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions.

Methods

The first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions.

Results

The proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective.

Conclusions

Despite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making.

SUBMITTER: Smith EA

PROVIDER: S-EPMC7860217 | biostudies-literature | 2021 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines.

Smith Ellesha A EA Cooper Nicola J NJ Sutton Alex J AJ Abrams Keith R KR Hubbard Stephanie J SJ

BMC public health 20210203 1

<h4>Background</h4>The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate 'lumping' of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to ...[more]

PMID: 33535975

Similar Datasets

Project description:BackgroundLiterature related to the effectiveness of knowledge translation (KT) strategies used in public health is lacking. The capacity to seek, analyze, and synthesize evidence-based information in public health is linked to greater success in making policy choices that have the best potential to yield positive outcomes for populations. The purpose of this systematic review is to identify the effectiveness of KT strategies used to promote evidence-informed decision making (EIDM) among public health decision makers.MethodsA search strategy was developed to identify primary studies published between 2000-2010. Studies were obtained from multiple electronic databases (CINAHL, Medline, EMBASE, and the Cochrane Database of Systematic Reviews). Searches were supplemented by hand searching and checking the reference lists of included articles. Two independent review authors screened studies for relevance, assessed methodological quality of relevant studies, and extracted data from studies using standardized tools.ResultsAfter removal of duplicates, the search identified 64, 391 titles related to KT strategies. Following title and abstract review, 346 publications were deemed potentially relevant, of which 5 met all relevance criteria on full text screen. The included publications were of moderate quality and consisted of five primary studies (four randomized controlled trials and one interrupted time series analysis). Results were synthesized narratively. Simple or single KT strategies were shown in some circumstances to be as effective as complex, multifaceted ones when changing practice including tailored and targeted messaging. Multifaceted KT strategies led to changes in knowledge but not practice. Knowledge translation strategies shown to be less effective were passive and included access to registries of pre-processed research evidence or print materials. While knowledge brokering did not have a significant effect generally, results suggested that it did have a positive effect on those organizations that at baseline perceived their organization to place little value on evidence-informed decision making.ConclusionsNo singular KT strategy was shown to be effective in all contexts. Conclusions about interventions cannot be taken on their own without considering the characteristics of the knowledge that was being transferred, providers, participants and organizations.

Project description:BackgroundEvidence-informed clinical practice guidelines (CPGs) are useful tools to inform transparent healthcare decision-making. Consideration of health economic evidence (HEE) during CPG development in a structured manner remains a challenge globally and locally. This study explored the views, current practice, training needs and challenges faced by CPG developers in the production and use of HEE for CPGs in South Africa.MethodsThis mixed-methods study comprised an online survey and a focus group discussion. The survey was piloted and subsequently sent to CPG role players - evidence reviewers, CPG panellists, academics involved with training in relevant disciplines like health economics and public health, implementers and funders. The focus group participants hold strategic roles in CPG development and health economic activities nationally. The survey evaluated mean values, measures of variability, and percentages for Likert scales, while narrative components were thematically analysed. Focus group data were manually coded, thematically analysed and verified.ResultsThe survey (n = 55 respondents to 245 surveys distributed) and one focus group (n = 5 participants from 10 people invited) occurred between October 2018 and February 2019. We found the most consistent reason why HEE should inform CPG decisions was 'making more efficient use of limited financial resources'. This was explained by numerous context and methodological barriers. Focus groups participants noted that consideration of complex HEE are not achievable without bolstering skills in applying evidence-based medicine principles. Further concerns include lack of clarity of standard methods; inequitable and opaque topic selection across private and public sectors; inadequate skills of CPG panel members to use HEE; and the ability of health economists to communicate results in accessible ways. Overall, in the absence of clarity about process and methods, politics and interests may drive CPG decisions about which interventions to implement.ConclusionsHEE should ideally be considered in CPG decisions in South Africa. However, this will remain hampered until the CPG community agree on methods and processes for using HEE in CPGs. Focused investment by national government to address the challenges identified by the study is imperative for a better return on investment as National Health Insurance moves forward.

Project description:BackgroundPublic health systems internationally are under pressure to meet increasing demand for healthcare in the context of increasing financial resource constraint. There is therefore a need to maximise health outcomes achieved with public healthcare expenditure. This paper aims to establish and synthesize the contemporary evidence base for approaches taken at a system management level to improve efficiency.MethodsRapid Evidence Assessment (REA) methodology was employed. A search strategy was developed and applied (PUBMED, MEDLINE) returning 5,377 unique titles. 172 full-text articles were screened to determine relevance with 82 publications included in the final review. Data regarding country, study design, key findings and approaches to efficiency improvement were extracted and a narrative synthesis produced. Publications covering health systems from developed countries were included.ResultsIdentified study designs included policy reviews, qualitative reviews, mixed methods reviews, systematic reviews, literature reviews, retrospective analyses, scoping reviews, narrative papers, regression analyses and opinion papers. While findings revealed no comprehensive frameworks for system-wide efficiency improvement, a range of specific centrally led improvement approaches were identified. Elements associated with success in current approaches included dedicated central functions to drive system-wide efficiency improvement, managing efficiency in tandem with quality and value, and inclusive stakeholder engagement.ConclusionsThe requirement for public health systems to improve efficiency is likely to continue to increase. Reactive cost-cutting measures and short-term initiatives aimed only at reducing expenditure are unlikely to deliver sustainable efficiency improvement. By providing dedicated central system-wide efficiency improvement support, public health system management entities can deliver improved financial, health service and stakeholder outcomes.

Dataset Information

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines.

Background

Methods

Results

Conclusions

Publications

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets