Dataset Information

Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: A systematic review of measure quality.

ABSTRACT:

Purpose

Understanding the effectiveness of complex, face-to-face health behaviour change interventions requires high-quality measures to assess fidelity of delivery and engagement. This systematic review aimed to (1) identify the types of measures used to monitor fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions and (2) describe the reporting of psychometric and implementation qualities.

Methods

Electronic databases were searched, systematic reviews and reference lists were hand-searched, and 21 experts were contacted to identify articles. Studies that quantitatively measured fidelity of delivery of, and/or engagement with, a complex, face-to-face health behaviour change intervention for adults were included. Data on interventions, measures, and psychometric and implementation qualities were extracted and synthesized using narrative analysis.

Results

Sixty-six studies were included: 24 measured both fidelity of delivery and engagement, 20 measured fidelity of delivery, and 22 measured engagement. Measures of fidelity of delivery included observation (n = 17; 38.6%), self-report (n = 15; 34%), quantitatively rated qualitative interviews (n = 1; 2.3%), or multiple measures (n = 11; 25%). Measures of engagement included self-report (n = 18; 39.1%), intervention records (n = 11; 24%), or multiple measures (n = 17; 37%). Fifty-one studies (77%) reported at least one psychometric or implementation quality; 49 studies (74.2%) reported at least one psychometric quality, and 17 studies (25.8%) reported at least one implementation quality.

Conclusion

Fewer than half of the reviewed studies measured both fidelity of delivery of, and engagement with complex, face-to-face health behaviour change interventions. More studies reported psychometric qualities than implementation qualities. Interpretation of intervention outcomes from fidelity of delivery and engagement measurements may be limited due to a lack of reporting of psychometric and implementation qualities. Statement of contribution What is already known on this subject? Evidence of fidelity and engagement is needed to understand effectiveness of complex interventions Evidence of fidelity and engagement are rarely reported High-quality measures are needed to measure fidelity and engagement What does this study add? Evidence that indicators of quality of measures are reported in some studies Evidence that psychometric qualities are reported more frequently than implementation qualities A recommendation for intervention evaluations to report indicators of quality of fidelity and engagement measures.

SUBMITTER: Walton H

PROVIDER: S-EPMC5655766 | biostudies-literature | 2017 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: A systematic review of measure quality.

Walton Holly H Spector Aimee A Tombor Ildiko I Michie Susan S

British journal of health psychology 20170801 4

<h4>Purpose</h4>Understanding the effectiveness of complex, face-to-face health behaviour change interventions requires high-quality measures to assess fidelity of delivery and engagement. This systematic review aimed to (1) identify the types of measures used to monitor fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions and (2) describe the reporting of psychometric and implementation qualities.<h4>Methods</h4>Electronic databases were sear ...[more]

PMID: 28762607

Similar Datasets

Project description:OBJECTIVES:To understand whether interventions are effective, we need to know whether the interventions are delivered as planned (with fidelity) and engaged with. To measure fidelity and engagement effectively, high-quality measures are needed. We outline a five-step method which can be used to develop quality measures of fidelity and engagement for complex health interventions. We provide examples from a fidelity study conducted within an evaluation of an intervention aimed to increase independence in dementia. METHODS:We propose five steps that can be systematically used to develop fidelity checklists for researchers, providers, and participants to measure fidelity and engagement. These steps include the following: (1) reviewing previous measures, (2) analysing intervention components and developing a framework outlining the content of the intervention, (3) developing fidelity checklists and coding guidelines, (4) obtaining feedback about the content and wording of checklists and guidelines, and (5) piloting and refining checklists and coding guidelines to assess and improve reliability. RESULTS:Three fidelity checklists that can be used reliably were developed to measure fidelity of and engagement with, the Promoting Independence in Dementia (PRIDE) intervention. As these measures were designed to be used by researchers, providers, and participants, we developed two versions of the checklists: one for participants and one for researchers and providers. CONCLUSIONS:The five steps that we propose can be used to develop psychometrically robust and implementable measures of fidelity and engagement for complex health interventions that can be used by different target audiences. By considering quality when developing measures, we can be more confident in the interpretation of intervention outcomes drawn from fidelity and engagement studies. Statement of contribution What is already known on the subject? Fidelity and engagement can be measured using a range of methods, such as observation and self-report. Studies seldom report psychometric and implementation qualities of fidelity measures. What does this study add? A method for developing fidelity and engagement measures for complex health interventions. Guidance on how to consider quality when developing fidelity and engagement measures.

Project description:Objectives: The aim of this study was to (1) demonstrate the development and testing of tools and procedures designed to monitor and assess the integrity of a complex intervention for chronic pain (COping with persistent Pain, Effectiveness Research into Self-management (COPERS) course); and (2) make recommendations based on our experiences.Design: Fidelity assessment of a two-arm randomised controlled trial intervention, assessing the adherence and competence of the facilitators delivering the intervention.Setting: The intervention was delivered in the community in two centres in the UK: one inner city and one a mix of rural and urban locations.Participants: 403 people with chronic musculoskeletal pain were enrolled in the intervention arm and 300 attended the self-management course. Thirty lay and healthcare professionals were trained and 24 delivered the courses (2 per course). We ran 31 courses for up to 16 people per course and all were audio recorded.Interventions: The course was run over three and a half days; facilitators delivered a semistructured manualised course.Outcomes: We designed three measures to evaluate fidelity assessing adherence to the manual, competence and overall impression.Results: We evaluated a random sample of four components from each course (n=122). The evaluation forms were reliable and had good face validity. There were high levels of adherence in the delivery: overall adherence was two (maximum 2, IQR 1.67-2.00), facilitator competence exhibited more variability, and overall competence was 1.5 (maximum 2, IQR 1.25-2.00). Overall impression was three (maximum 4, IQR 2.00-3.00).Conclusions: Monitoring and assessing adherence and competence at the point of intervention delivery can be realised most efficiently by embedding the principles of fidelity measurement within the design stage of complex interventions and the training and assessment of those delivering the intervention. More work is necessary to ensure that more robust systems of fidelity evaluation accompany the growth of complex interventions.Trial registration isrctn no: ISRCTN24426731.

Project description:Behaviour change interventions are likely to be reproducible only if reported clearly. We assessed whether the behaviour change technique taxonomy version 1 (BCTTv1), with and without training in identifying BCTs, improves the clarity and replicability of written reports of observed behaviour change interventions.Three studies assessed effects of using and training in the use of BCTTv1 on the clarity and replicability of intervention descriptions written after observing videos of smoking cessation interventions. Study 1 examined the effects of using and not using BCTTv1. Study 2 examined the effects of using BCTTv1 and training in use of BCTTv1 compared no use and no training. Study 3 employed a within-group design to assess change in descriptions written before and after training. One-hundred and 66 'writers' watched videos of behaviour change interventions and wrote descriptions of the active components delivered. In all studies, the participants' written descriptions were evaluated by (i) 12 'raters' (untrained in BCTTv1) for clarity and replicability and (ii) 12 'coders' (trained in BCTTv1) for reliability of BCT coding. Writers rated the usability and accessibility of using BCTTv1 to write descriptions.Ratings of clarity and replicability did not differ between groups in study 1 (all ps > 0.05), were poorer for trained users in study 2 (all ps < 0.01) and improved following training in study 3 (all ps < 0.05). BCT identification was more reliable from descriptions written by trained BCTTv1 users (p < 0.05; study 2) but not simple use of BCTTv1 (p = 0.93; study 1) or by writers who had written a description without BCTTv1, before training (p = 0.50; study 3). Writers reported that using BCTTv1 was difficult but 'useful', 'good' and 'desirable' and that their descriptions would be clear and replicable (all means above mid-point of the scale).Effects of training to use BCTTv1 on the quality of written reports of observed interventions were mixed, with some suggestion of improved clarity and replicability of reporting in the within- (study 3) but not the between-group studies (studies 1 and 2). Potential benefits of using BCTTv1 may have been limited by the artificial nature and time constraints of the task.

Project description:BackgroundImproving the design and implementation of evidence-based practice depends on successful behaviour change interventions. This requires an appropriate method for characterising interventions and linking them to an analysis of the targeted behaviour. There exists a plethora of frameworks of behaviour change interventions, but it is not clear how well they serve this purpose. This paper evaluates these frameworks, and develops and evaluates a new framework aimed at overcoming their limitations.MethodsA systematic search of electronic databases and consultation with behaviour change experts were used to identify frameworks of behaviour change interventions. These were evaluated according to three criteria: comprehensiveness, coherence, and a clear link to an overarching model of behaviour. A new framework was developed to meet these criteria. The reliability with which it could be applied was examined in two domains of behaviour change: tobacco control and obesity.ResultsNineteen frameworks were identified covering nine intervention functions and seven policy categories that could enable those interventions. None of the frameworks reviewed covered the full range of intervention functions or policies, and only a minority met the criteria of coherence or linkage to a model of behaviour. At the centre of a proposed new framework is a 'behaviour system' involving three essential conditions: capability, opportunity, and motivation (what we term the 'COM-B system'). This forms the hub of a 'behaviour change wheel' (BCW) around which are positioned the nine intervention functions aimed at addressing deficits in one or more of these conditions; around this are placed seven categories of policy that could enable those interventions to occur. The BCW was used reliably to characterise interventions within the English Department of Health's 2010 tobacco control strategy and the National Institute of Health and Clinical Excellence's guidance on reducing obesity.ConclusionsInterventions and policies to change behaviour can be usefully characterised by means of a BCW comprising: a 'behaviour system' at the hub, encircled by intervention functions and then by policy categories. Research is needed to establish how far the BCW can lead to more efficient design of effective interventions.

Project description:BackgroundDecreasing ineffective or harmful healthcare practices (de-implementation) may require different approaches than those used to promote uptake of effective practices (implementation). Few psychological theories differentiate between processes involved in decreasing, versus increasing, behaviour. However, it is unknown whether implementation and de-implementation interventions already use different approaches. We used the behaviour change technique (BCT) taxonomy (version 1) (which includes 93 BCTs organised into 12 groupings) to investigate whether implementation and de-implementation interventions for clinician behaviour change use different BCTs.MethodsIntervention descriptions in 181 articles from three systematic reviews in the Cochrane Library were coded for (a) implementation versus de-implementation and (b) intervention content (BCTs) using the BCT taxonomy (v1). BCT frequencies were calculated and compared using Pearson's chi-squared (χ2), Yates' continuity correction and Fisher's exact test, where appropriate. Identified BCTs were ranked according to frequency and rankings for de-implementation versus implementation interventions were compared and described.ResultsTwenty-nine and 25 BCTs were identified in implementation and de-implementation interventions respectively. Feedback on behaviour was identified more frequently in implementation than de-implementation (Χ2(2, n=178) = 15.693, p = .000057). Three BCTs were identified more frequently in de-implementation than implementation: Behaviour substitution (Χ2(2, n=178) = 14.561, p = .0001; Yates' continuity correction); Monitoring of behaviour by others without feedback (Χ2(2, n=178) = 16.187, p = .000057; Yates' continuity correction); and Restructuring social environment (p = .000273; Fisher's 2-sided exact test).ConclusionsThere were some significant differences between BCTs reported in implementation and de-implementation interventions suggesting that researchers may have implicit theories about different BCTs required for de-implementation and implementation. These findings do not imply that the BCTs identified as targeting implementation or de-implementation are effective, rather simply that they were more frequently used. These findings require replication for a wider range of clinical behaviours. The continued accumulation of additional knowledge and evidence into whether implementation and de-implementation is different will serve to better inform researchers and, subsequently, improve methods for intervention design.

Dataset Information

Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: A systematic review of measure quality.

Purpose

Methods

Results

Conclusion

Publications

Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: A systematic review of measure quality.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure