Dataset Information

Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting.

ABSTRACT: BACKGROUND:Process evaluations are recommended to open the 'black box' of complex interventions evaluated in trials, but there is limited guidance to help researchers design process evaluations. Much current literature on process evaluations of complex interventions focuses on qualitative methods, with less attention paid to quantitative methods. This discrepancy led us to develop our own framework for designing process evaluations of cluster-randomised controlled trials. METHODS:We reviewed recent theoretical and methodological literature and selected published process evaluations; these publications identified a need for structure to help design process evaluations. We drew upon this literature to develop a framework through iterative exchanges, and tested this against published evaluations. RESULTS:The developed framework presents a range of candidate approaches to understanding trial delivery, intervention implementation and the responses of targeted participants. We believe this framework will be useful to others designing process evaluations of complex intervention trials. We also propose key information that process evaluations could report to facilitate their identification and enhance their usefulness. CONCLUSION:There is no single best way to design and carry out a process evaluation. Researchers will be faced with choices about what questions to focus on and which methods to use. The most appropriate design depends on the purpose of the process evaluation; the framework aims to help researchers make explicit their choices of research questions and methods. TRIAL REGISTRATION:Clinicaltrials.gov NCT01425502.

SUBMITTER: Grant A

PROVIDER: S-EPMC3600672 | biostudies-literature | 2013 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting.

Grant Aileen A Treweek Shaun S Dreischulte Tobias T Foy Robbie R Guthrie Bruce B

Trials 20130112

<h4>Background</h4>Process evaluations are recommended to open the 'black box' of complex interventions evaluated in trials, but there is limited guidance to help researchers design process evaluations. Much current literature on process evaluations of complex interventions focuses on qualitative methods, with less attention paid to quantitative methods. This discrepancy led us to develop our own framework for designing process evaluations of cluster-randomised controlled trials.<h4>Methods</h4> ...[more]

PMID: 23311722

Similar Datasets

Project description:BackgroundTreatment non-adherence in randomised trials refers to situations where some participants do not receive their allocated treatment as intended. For cluster randomised trials, where the unit of randomisation is a group of participants, non-adherence may occur at the cluster or individual level. When non-adherence occurs, randomisation no longer guarantees that the relationship between treatment receipt and outcome is unconfounded, and the power to detect the treatment effects in intention-to-treat analysis may be reduced. Thus, recording adherence and estimating the causal treatment effect adequately are of interest for clinical trials.ObjectivesTo assess the extent of reporting of non-adherence issues in published cluster trials and to establish which methods are currently being used for addressing non-adherence, if any, and whether clustering is accounted for in these.MethodsWe systematically reviewed 132 cluster trials published in English in 2011 previously identified through a search in PubMed.ResultsOne-hundred and twenty three cluster trials were included in this systematic review. Non-adherence was reported in 56 cluster trials. Among these, 19 reported a treatment efficacy estimate: per protocol in 15 and as treated in 4. No study discussed the assumptions made by these methods, their plausibility or the sensitivity of the results to deviations from these assumptions.LimitationsThe year of publication of the cluster trials included in this review (2011) could be considered a limitation of this study; however, no new guidelines regarding the reporting and the handling of non-adherence for cluster trials have been published since. In addition, a single reviewer undertook the data extraction. To mitigate this, a second reviewer conducted a validation of the extraction process on 15 randomly selected reports. Agreement was satisfactory (93%).ConclusionDespite the recommendations of the Consolidated Standards of Reporting Trials statement extension to cluster randomised trials, treatment adherence is under-reported. Among the trials providing adherence information, there was substantial variation in how adherence was defined, handled and reported. Researchers should discuss the assumptions required for the results to be interpreted causally and whether these are scientifically plausible in their studies. Sensitivity analyses to study the robustness of the results to departures from these assumptions should be performed.

Project description:ObjectivesTo systematically review and synthesise findings from process evaluations of interventions in trials which measured sedentary behaviour as an outcome in adults to explore: (1) how intervention content, implementation, mechanisms of impact and context influence outcomes and (2) how these interventions are experienced from different perspectives (participants, carers, staff).DesignSystematic review and narrative synthesis underpinned by the Medical Research Council process evaluation framework.Data sourcesDatabases searches were conducted in March 2019 then updated in May 2020 and October 2021 in: CINAHL, SPORTDiscus, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, AMED; EMBASE, PsycINFO, MEDLINE, Web of Science and ProQuest Dissertations & Theses.Eligibility criteriaWe included: Process evaluations of trials including interventions where sedentary behaviour was measured as an outcome in adults aged 16 or over from clinical or non-clinical populations. We excluded studies if interventions were delivered in educational or workplace settings, or if they were laboratory studies focused on immediate effects of breaking sitting.Data extraction and synthesisTwo independent reviewers extracted and coded data into a framework and assessed the quality of studies using the Mixed Methods Appraisal Tool. We synthesised findings using a narrative approach.Results17 process evaluations were included. Five interventions focused on reducing sedentary behaviour or sitting time, 12 aimed to increase physical activity or promote healthier lifestyles. Process evaluations indicated changes in sedentary behaviour outcomes were shaped by numerous factors including: barriers (eg, staffing difficulties and scheduling problems) and facilitators (eg, allowing for flexibility) to intervention delivery; contextual factors (eg, usual lifestyle and religious events) and individual factors (eg, pain, tiredness, illness, age and individual preferences).DiscussionIntervention requires careful consideration of different factors that could influence changes in sedentary behaviour outcomes to ensure that interventions can be tailored to suit different individuals and groups.Prospero registration numberCRD42018087403.

Project description:OBJECTIVE:Systematic reviews can include cluster-randomised controlled trials (C-RCTs), which require different analysis compared with standard individual-randomised controlled trials. However, it is not known whether review authors follow the methodological and reporting guidance when including these trials. The aim of this study was to assess the methodological and reporting practice of Cochrane reviews that included C-RCTs against criteria developed from existing guidance. METHODS:Criteria were developed, based on methodological literature and personal experience supervising review production and quality. Criteria were grouped into four themes: identifying, reporting, assessing risk of bias, and analysing C-RCTs. The Cochrane Database of Systematic Reviews was searched (2nd December 2013), and the 50 most recent reviews that included C-RCTs were retrieved. Each review was then assessed using the criteria. RESULTS:The 50 reviews we identified were published by 26 Cochrane Review Groups between June 2013 and November 2013. For identifying C-RCTs, only 56% identified that C-RCTs were eligible for inclusion in the review in the eligibility criteria. For reporting C-RCTs, only eight (24%) of the 33 reviews reported the method of cluster adjustment for their included C-RCTs. For assessing risk of bias, only one review assessed all five C-RCT-specific risk-of-bias criteria. For analysing C-RCTs, of the 27 reviews that presented unadjusted data, only nine (33%) provided a warning that confidence intervals may be artificially narrow. Of the 34 reviews that reported data from unadjusted C-RCTs, only 13 (38%) excluded the unadjusted results from the meta-analyses. CONCLUSIONS:The methodological and reporting practices in Cochrane reviews incorporating C-RCTs could be greatly improved, particularly with regard to analyses. Criteria developed as part of the current study could be used by review authors or editors to identify errors and improve the quality of published systematic reviews incorporating C-RCTs.

Project description:BackgroundThere are many methodological challenges in the conduct and analysis of cluster randomised controlled trials, but one that has received little attention is that of post-randomisation changes to cluster composition. To illustrate this, we focus on the issue of cluster merging, considering the impact on the design, analysis and interpretation of trial outcomes.MethodsWe explored the effects of merging clusters on study power using standard methods of power calculation. We assessed the potential impacts on study findings of both homogeneous cluster merges (involving clusters randomised to the same arm of a trial) and heterogeneous merges (involving clusters randomised to different arms of a trial) by simulation. To determine the impact on bias and precision of treatment effect estimates, we applied standard methods of analysis to different populations under analysis.ResultsCluster merging produced a systematic reduction in study power. This effect depended on the number of merges and was most pronounced when variability in cluster size was at its greatest. Simulations demonstrate that the impact on analysis was minimal when cluster merges were homogeneous, with impact on study power being balanced by a change in observed intracluster correlation coefficient (ICC). We found a decrease in study power when cluster merges were heterogeneous, and the estimate of treatment effect was attenuated.ConclusionsExamples of cluster merges found in previously published reports of cluster randomised trials were typically homogeneous rather than heterogeneous. Simulations demonstrated that trial findings in such cases would be unbiased. However, simulations also showed that any heterogeneous cluster merges would introduce bias that would be hard to quantify, as well as having negative impacts on the precision of estimates obtained. Further methodological development is warranted to better determine how to analyse such trials appropriately. Interim recommendations include avoidance of cluster merges where possible, discontinuation of clusters following heterogeneous merges, allowance for potential loss of clusters and additional variability in cluster size in the original sample size calculation, and use of appropriate ICC estimates that reflect cluster size.

Project description:BACKGROUND:Randomised controlled trials (RCTs) are used to evaluate social and psychological interventions and inform policy decisions about them. Accurate, complete, and transparent reports of social and psychological intervention RCTs are essential for understanding their design, conduct, results, and the implications of the findings. However, the reporting of RCTs of social and psychological interventions remains suboptimal. The CONSORT Statement has improved the reporting of RCTs in biomedicine. A similar high-quality guideline is needed for the behavioural and social sciences. Our objective was to develop an official extension of the Consolidated Standards of Reporting Trials 2010 Statement (CONSORT 2010) for reporting RCTs of social and psychological interventions: CONSORT-SPI 2018. METHODS:We followed best practices in developing the reporting guideline extension. First, we conducted a systematic review of existing reporting guidelines. We then conducted an online Delphi process including 384 international participants. In March 2014, we held a 3-day consensus meeting of 31 experts to determine the content of a checklist specifically targeting social and psychological intervention RCTs. Experts discussed previous research and methodological issues of particular relevance to social and psychological intervention RCTs. They then voted on proposed modifications or extensions of items from CONSORT 2010. RESULTS:The CONSORT-SPI 2018 checklist extends 9 of the 25 items from CONSORT 2010: background and objectives, trial design, participants, interventions, statistical methods, participant flow, baseline data, outcomes and estimation, and funding. In addition, participants added a new item related to stakeholder involvement, and they modified aspects of the flow diagram related to participant recruitment and retention. CONCLUSIONS:Authors should use CONSORT-SPI 2018 to improve reporting of their social and psychological intervention RCTs. Journals should revise editorial policies and procedures to require use of reporting guidelines by authors and peer reviewers to produce manuscripts that allow readers to appraise study quality, evaluate the applicability of findings to their contexts, and replicate effective interventions.

Dataset Information

Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting.

Publications

Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets