Dataset Information

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines.

ABSTRACT:

Background

The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate 'lumping' of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions.

Methods

The first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions.

Results

The proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective.

Conclusions

Despite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making.

SUBMITTER: Smith EA

PROVIDER: S-EPMC7860217 | biostudies-literature | 2021 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines.

Smith Ellesha A EA Cooper Nicola J NJ Sutton Alex J AJ Abrams Keith R KR Hubbard Stephanie J SJ

BMC public health 20210203 1

<h4>Background</h4>The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate 'lumping' of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to ...[more]

PMID: 33535975

Similar Datasets

Project description:BackgroundLiterature related to the effectiveness of knowledge translation (KT) strategies used in public health is lacking. The capacity to seek, analyze, and synthesize evidence-based information in public health is linked to greater success in making policy choices that have the best potential to yield positive outcomes for populations. The purpose of this systematic review is to identify the effectiveness of KT strategies used to promote evidence-informed decision making (EIDM) among public health decision makers.MethodsA search strategy was developed to identify primary studies published between 2000-2010. Studies were obtained from multiple electronic databases (CINAHL, Medline, EMBASE, and the Cochrane Database of Systematic Reviews). Searches were supplemented by hand searching and checking the reference lists of included articles. Two independent review authors screened studies for relevance, assessed methodological quality of relevant studies, and extracted data from studies using standardized tools.ResultsAfter removal of duplicates, the search identified 64, 391 titles related to KT strategies. Following title and abstract review, 346 publications were deemed potentially relevant, of which 5 met all relevance criteria on full text screen. The included publications were of moderate quality and consisted of five primary studies (four randomized controlled trials and one interrupted time series analysis). Results were synthesized narratively. Simple or single KT strategies were shown in some circumstances to be as effective as complex, multifaceted ones when changing practice including tailored and targeted messaging. Multifaceted KT strategies led to changes in knowledge but not practice. Knowledge translation strategies shown to be less effective were passive and included access to registries of pre-processed research evidence or print materials. While knowledge brokering did not have a significant effect generally, results suggested that it did have a positive effect on those organizations that at baseline perceived their organization to place little value on evidence-informed decision making.ConclusionsNo singular KT strategy was shown to be effective in all contexts. Conclusions about interventions cannot be taken on their own without considering the characteristics of the knowledge that was being transferred, providers, participants and organizations.

Project description:BackgroundEvidence-informed clinical practice guidelines (CPGs) are useful tools to inform transparent healthcare decision-making. Consideration of health economic evidence (HEE) during CPG development in a structured manner remains a challenge globally and locally. This study explored the views, current practice, training needs and challenges faced by CPG developers in the production and use of HEE for CPGs in South Africa.MethodsThis mixed-methods study comprised an online survey and a focus group discussion. The survey was piloted and subsequently sent to CPG role players - evidence reviewers, CPG panellists, academics involved with training in relevant disciplines like health economics and public health, implementers and funders. The focus group participants hold strategic roles in CPG development and health economic activities nationally. The survey evaluated mean values, measures of variability, and percentages for Likert scales, while narrative components were thematically analysed. Focus group data were manually coded, thematically analysed and verified.ResultsThe survey (n = 55 respondents to 245 surveys distributed) and one focus group (n = 5 participants from 10 people invited) occurred between October 2018 and February 2019. We found the most consistent reason why HEE should inform CPG decisions was 'making more efficient use of limited financial resources'. This was explained by numerous context and methodological barriers. Focus groups participants noted that consideration of complex HEE are not achievable without bolstering skills in applying evidence-based medicine principles. Further concerns include lack of clarity of standard methods; inequitable and opaque topic selection across private and public sectors; inadequate skills of CPG panel members to use HEE; and the ability of health economists to communicate results in accessible ways. Overall, in the absence of clarity about process and methods, politics and interests may drive CPG decisions about which interventions to implement.ConclusionsHEE should ideally be considered in CPG decisions in South Africa. However, this will remain hampered until the CPG community agree on methods and processes for using HEE in CPGs. Focused investment by national government to address the challenges identified by the study is imperative for a better return on investment as National Health Insurance moves forward.

Project description:Increasingly, computer simulation models are used for economic and policy evaluation in cancer prevention and control. A model's predictions of key outcomes, such as screening effectiveness, depend on the values of unobservable natural history parameters. Calibration is the process of determining the values of unobservable parameters by constraining model output to replicate observed data. Because there are many approaches for model calibration and little consensus on best practices, we surveyed the literature to catalogue the use and reporting of these methods in cancer simulation models. We conducted a MEDLINE search (1980 through 2006) for articles on cancer-screening models and supplemented search results with articles from our personal reference databases. For each article, two authors independently abstracted pre-determined items using a standard form. Data items included cancer site, model type, methods used for determination of unobservable parameter values and description of any calibration protocol. All authors reached consensus on items of disagreement. Reviews and non-cancer models were excluded. Articles describing analytical models, which estimate parameters with statistical approaches (e.g. maximum likelihood) were catalogued separately. Models that included unobservable parameters were analysed and classified by whether calibration methods were reported and if so, the methods used. The review process yielded 154 articles that met our inclusion criteria and, of these, we concluded that 131 may have used calibration methods to determine model parameters. Although the term 'calibration' was not always used, descriptions of calibration or 'model fitting' were found in 50% (n = 66) of the articles, with an additional 16% (n = 21) providing a reference to methods. Calibration target data were identified in nearly all of these articles. Other methodological details, such as the goodness-of-fit metric, were discussed in 54% (n = 47 of 87) of the articles reporting calibration methods, while few details were provided on the algorithms used to search the parameter space. Our review shows that the use of cancer simulation modelling is increasing, although thorough descriptions of calibration procedures are rare in the published literature for these models. Calibration is a key component of model development and is central to the validity and credibility of subsequent analyses and inferences drawn from model predictions. To aid peer-review and facilitate discussion of modelling methods, we propose a standardized Calibration Reporting Checklist for model documentation.

Dataset Information

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines.

Background

Methods

Results

Conclusions

Publications

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets