Dataset Information

Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews.

ABSTRACT:

Background

Logic models are becoming an increasingly common feature of systematic reviews, as is the use of programme theory more generally in systematic reviewing. Logic models offer a framework to help reviewers to 'think' conceptually at various points during the review, and can be a useful tool in defining study inclusion and exclusion criteria, guiding the search strategy, identifying relevant outcomes, identifying mediating and moderating factors, and communicating review findings.

Methods and findings

In this paper we critique the use of logic models in systematic reviews and protocols drawn from two databases representing reviews of health interventions and international development interventions. Programme theory featured only in a minority of the reviews and protocols included. Despite drawing from different disciplinary traditions, reviews and protocols from both sources shared several limitations in their use of logic models and theories of change, and these were used almost unanimously to solely depict pictorially the way in which the intervention worked. Logic models and theories of change were consequently rarely used to communicate the findings of the review.

Conclusions

Logic models have the potential to be an aid integral throughout the systematic reviewing process. The absence of good practice around their use and development may be one reason for the apparent limited utility of logic models in many existing systematic reviews. These concerns are addressed in the second half of this paper, where we offer a set of principles in the use of logic models and an example of how we constructed a logic model for a review of school-based asthma interventions.

SUBMITTER: Kneale D

PROVIDER: S-EPMC4648510 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews.

Kneale Dylan D Thomas James J Harris Katherine K

PloS one 20151117 11

<h4>Background</h4>Logic models are becoming an increasingly common feature of systematic reviews, as is the use of programme theory more generally in systematic reviewing. Logic models offer a framework to help reviewers to 'think' conceptually at various points during the review, and can be a useful tool in defining study inclusion and exclusion criteria, guiding the search strategy, identifying relevant outcomes, identifying mediating and moderating factors, and communicating review findings. ...[more]

PMID: 26575182

Similar Datasets

Project description:Background: With increasing insecticide resistance in malaria-endemic countries there is an urgent need for safe and effective novel vector control products. To improve the capacity of facilities that test insecticides in sub-Saharan Africa, a programme is supporting seven facilities towards Good Laboratory Practice (GLP) certification, the globally recognized standard for quality management system (QMS) for the conduct of non-clinical and environmental studies. The World Health Organization (WHO) GLP Handbook provides guidance on a stepwise approach to implement a GLP compliant QMS. This study assesses auditor GLP checklists and timings outlined in the WHO GLP Handbook in the real-life context of a Tanzanian insecticide-testing facility, evaluating their implementation in this context. Methods and Principle Findings: We conducted document review and semi-structured interviews with staff at all levels of the test facility to explore factors that influenced progress towards GLP certification. We found that while auditor GLP checklists underemphasised computer systems, they were otherwise broadly applicable. Factors that delayed time to completion of GLP certification included the need for extensive infrastructure improvements, the availability of regional expertise related to GLP, the capacity of national and regional external systems and services to meet GLP compliance requirements, and training development required for Standard Operating Procedure implementation. Conclusion: The standards required for full GLP compliance are rigorous, with an expected completion timeline to implementation of 24 months. This study shows that in low and middle-income countries this timeline may be unrealistic due to challenges related to infrastructure development and lack of regional capacity and expertise. We recommend a comprehensive gap analysis when starting a project, including these areas which are beyond those recommended by the WHO GLP Handbook. These challenges can be successfully overcome and the experience in Tanzania provides key lessons for other facilities seeking GLP certification or the development of similar QMS.

Project description:BackgroundInterventions to improve physical activity behaviour are a core part of public health policy and practice. It is essential that we evaluate these interventions and use the evidence to inform decisions to improve population health. Evaluation of 'real-world' interventions provide an opportunity to generate practice-relevant evidence, however these interventions are difficult to evaluate. Various guidelines have been developed to facilitate evaluation, but evidence about their effectiveness in practice is limited. To explore influences on evaluation practice in an applied context, we conducted a case study of Sport England's 'Get Healthy Get Active' (GHGA) programme. This was a national programme that funded 33 projects that were delivered and evaluated across England. The programme was chosen as it was designed to generate evidence on the role of sport in increasing physical activity and improving health. The study aimed to explore and appraise whether strategies intended to facilitate project evaluation, including funder requirements to use a standardised evaluation framework and specific data collection methods, were effective in generating evidence that enabled the programme to meet its aims.MethodsWe applied a collective case study design involving 35 semi-structured interviews, and documentary analysis of multiple sources of evidence from 23 physical activity projects funded by GHGA. We applied thematic and framework analysis. We developed a logic model and mapped actual outcomes against intended outcomes. A narrative synthesis is provided. We discuss implications for the effective commissioning and evaluation of public health interventions.ResultsWe identified five main themes of influences on evaluation practices that can act as barriers and facilitators to good practice: programme and project design; evaluation design; partnerships; resources; and organisational structures and systems. These influences are context-specific and operate through a complex set of interactions.ConclusionDeveloping a better understanding of how influences on evaluation practice can act as facilitators or barriers is vital to help close current gaps in the evidence-based practice cycle. Critically, organisational structures and systems are needed to facilitate collaborative decision making; integration of projects and evaluation across partners organisations; transfer of knowldege and insights between stakeholders; and more rapid feedback and dissemination.

Project description:INTRODUCTION:Guidelines are systematically developed recommendations to assist practitioner and patient decisions about treatments for clinical conditions. High quality and comprehensive systematic reviews and 'overviews of systematic reviews' (overviews) represent the best available evidence. Many guideline developers, such as the WHO and the Australian National Health and Medical Research Council, recommend the use of these research syntheses to underpin guideline recommendations. We aim to evaluate the impact and use of systematic reviews with and without pairwise meta-analysis or network meta-analyses (NMAs) and overviews in clinical practice guideline (CPG) recommendations. METHODS AND ANALYSIS:CPGs will be retrieved from Turning Research Into Practice and Epistemonikos (2017-2018). The retrieved citations will be sorted randomly and then screened sequentially by two independent reviewers until 50 CPGs have been identified. We will include CPGs that provide at least two explicit recommendations for the management of any clinical condition. We will assess whether reviews or overviews were cited in a recommendation as part of the development process for guidelines. Data extraction will be done independently by two authors and compared. We will assess the risk of bias by examining how each guideline developed clinical recommendations. We will calculate the number and frequency of citations of reviews with or without pairwise meta-analysis, reviews with NMAs and overviews, and whether they were systematically or non-systematically developed. Results will be described, tabulated and categorised based on review type (reviews or overviews). CPGs reporting the use of the Grading of Recommendations, Assessment, Development and Evaluation approach will be compared with those using a different system, and pharmacological versus non-pharmacological CPGs will be compared. ETHICS AND DISSEMINATION:No ethics approval is required. We will present at the Cochrane Colloquium and the Guidelines International Network conference.

Dataset Information

Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews.

Background

Methods and findings

Conclusions

Publications

Developing and Optimising the Use of Logic Models in Systematic Reviews: Exploring Practice and Good Practice in the Use of Programme Theory in Reviews.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets