Dataset Information

Designing process evaluations using case study to explore the context of complex interventions evaluated in trials.

ABSTRACT: BACKGROUND:Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail, and whether they can be transferred to other settings and populations. However, historically, context has not been sufficiently explored and reported resulting in the poor uptake of trial results. Therefore, suitable methodologies are needed to guide the investigation of context. Case study is one appropriate methodology, but there is little guidance about what case study design can offer the study of context in trials. We address this gap in the literature by presenting a number of important considerations for process evaluation using a case study design. MAIN TEXT:In this paper, we define context, the relationship between complex interventions and context, and describe case study design methodology. A well-designed process evaluation using case study should consider the following core components: the purpose; definition of the intervention; the trial design, the case, the theories or logic models underpinning the intervention, the sampling approach and the conceptual or theoretical framework. We describe each of these in detail and highlight with examples from recently published process evaluations. CONCLUSIONS:There are a number of approaches to process evaluation design in the literature; however, there is a paucity of research on what case study design can offer process evaluations. We argue that case study is one of the best research designs to underpin process evaluations, to capture the dynamic and complex relationship between intervention and context during implementation. We provide a comprehensive overview of the issues for process evaluation design to consider when using a case study design. TRIAL REGISTRATION:DQIP - ClinicalTrials.gov number, NCT01425502 - OPAL - ISRCTN57746448.

SUBMITTER: Grant A

PROVIDER: S-EPMC7694311 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Designing process evaluations using case study to explore the context of complex interventions evaluated in trials.

Grant Aileen A Bugge Carol C Wells Mary M

Trials 20201127 1

<h4>Background</h4>Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail, and whether they can be transferred to other settings and populations. However, historically, context has not been sufficiently explored and reported resulting in the poor uptake of trial results. Therefore, suitable methodologies are needed to guide the investigation o ...[more]

PMID: 33246496

Similar Datasets

Project description:ObjectivesTo systematically review and synthesise findings from process evaluations of interventions in trials which measured sedentary behaviour as an outcome in adults to explore: (1) how intervention content, implementation, mechanisms of impact and context influence outcomes and (2) how these interventions are experienced from different perspectives (participants, carers, staff).DesignSystematic review and narrative synthesis underpinned by the Medical Research Council process evaluation framework.Data sourcesDatabases searches were conducted in March 2019 then updated in May 2020 and October 2021 in: CINAHL, SPORTDiscus, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, AMED; EMBASE, PsycINFO, MEDLINE, Web of Science and ProQuest Dissertations & Theses.Eligibility criteriaWe included: Process evaluations of trials including interventions where sedentary behaviour was measured as an outcome in adults aged 16 or over from clinical or non-clinical populations. We excluded studies if interventions were delivered in educational or workplace settings, or if they were laboratory studies focused on immediate effects of breaking sitting.Data extraction and synthesisTwo independent reviewers extracted and coded data into a framework and assessed the quality of studies using the Mixed Methods Appraisal Tool. We synthesised findings using a narrative approach.Results17 process evaluations were included. Five interventions focused on reducing sedentary behaviour or sitting time, 12 aimed to increase physical activity or promote healthier lifestyles. Process evaluations indicated changes in sedentary behaviour outcomes were shaped by numerous factors including: barriers (eg, staffing difficulties and scheduling problems) and facilitators (eg, allowing for flexibility) to intervention delivery; contextual factors (eg, usual lifestyle and religious events) and individual factors (eg, pain, tiredness, illness, age and individual preferences).DiscussionIntervention requires careful consideration of different factors that could influence changes in sedentary behaviour outcomes to ensure that interventions can be tailored to suit different individuals and groups.Prospero registration numberCRD42018087403.

Project description:BackgroundNormalization Process Theory (NPT) identifies, characterises and explains key mechanisms that promote and inhibit the implementation, embedding and integration of new health techniques, technologies and other complex interventions. A large body of literature that employs NPT to inform feasibility studies and process evaluations of complex healthcare interventions has now emerged. The aims of this review were to review this literature; to identify and characterise the uses and limits of NPT in research on the implementation and integration of healthcare interventions; and to explore NPT's contribution to understanding the dynamics of these processes.MethodsA qualitative systematic review was conducted. We searched Web of Science, Scopus and Google Scholar for articles with empirical data in peer-reviewed journals that cited either key papers presenting and developing NPT, or the NPT Online Toolkit ( www.normalizationprocess.org ). We included in the review only articles that used NPT as the primary approach to collection, analysis or reporting of data in studies of the implementation of healthcare techniques, technologies or other interventions. A structured data extraction instrument was used, and data were analysed qualitatively.ResultsSearches revealed 3322 citations. We show that after eliminating 2337 duplicates and broken or junk URLs, 985 were screened as titles and abstracts. Of these, 101 were excluded because they did not fit the inclusion criteria for the review. This left 884 articles for full-text screening. Of these, 754 did not fit the inclusion criteria for the review. This left 130 papers presenting results from 108 identifiable studies to be included in the review. NPT appears to provide researchers and practitioners with a conceptual vocabulary for rigorous studies of implementation processes. It identifies, characterises and explains empirically identifiable mechanisms that motivate and shape implementation processes. Taken together, these mean that analyses using NPT can effectively assist in the explanation of the success or failure of specific implementation projects. Ten percent of papers included critiques of some aspect of NPT, with those that did mainly focusing on its terminology. However, two studies critiqued NPT emphasis on agency, and one study critiqued NPT for its normative focus.ConclusionsThis review demonstrates that researchers found NPT useful and applied it across a wide range of interventions. It has been effectively used to aid intervention development and implementation planning as well as evaluating and understanding implementation processes themselves. In particular, NPT appears to have offered a valuable set of conceptual tools to aid understanding of implementation as a dynamic process.

Project description:BackgroundComplex interventions in health care are characterized by multiple interacting components as well as by numerous nonlinear interactions with the social systems within which they are being implemented. The process of developing, evaluating and implementing complex interventions is therefore challenging. Established guidance such as the MRC (Medical Research Council) framework for developing and evaluating complex interventions refers to process evaluations as an integral part of the development of complex evidence-based interventions. Even though the need for process evaluations is recognized, the realization of such approaches is challenging because methodological instruction is sparse, and the phenomenon of interest is complex. A number of theoretical approaches indicating how to conduct process evaluations of complex interventions in health care exist, but a systematic and comprehensive overview of these is missing. Thus, the objective of the systematic scoping review described herein is to provide an overview and analysis of theoretical approaches suitable for the planning and conducting of process evaluations.MethodsThe design and conduct of this review will follow the procedures of a systematic scoping review. The search strategy will be developed following the BeHEMoTh (Behaviour of interest; Health context; Exclusions; Models or Theories) template which has been conceptualized for structured reviews of theory. The systematic search of the MEDLINE (via PubMed), CINAHL (via EBSCO) and PsycInfo (via EBSCO) electronic databases will be complemented by "hand searching" techniques. Study selection, data extraction, and data analysis will be performed by tandems of two researchers independently of each other. Divergent decisions and judgements between the two researchers will be discussed by the whole review team.DiscussionThe findings from this scoping review will provide an overview and comparison of theoretical approaches suitable for process evaluations of complex interventions in health care. The review results will support researchers in choosing the theoretical approach that best fits the respective focus of their process evaluation study.Systematic review registrationThis study has been registered with PROSPERO (International Prospective Register of Systematic Reviews) under registration number CRD42020211732 .

Project description:ObjectiveProcess evaluations (PEs) alongside randomised controlled trials of complex interventions are valuable because they address questions of for whom, how and why interventions had an impact. We synthesised the methods used in PEs of primary care interventions, and their main findings on implementation barriers and facilitators.DesignSystematic review using the UK Medical Research Council guidance for PE as a guide.Data sourcesAcademic databases (MEDLINE, SCOPUS, PsycINFO, Cumulative Index to Nursing and Allied Health Literature, EMBASE and Global Health) were searched from 1998 until June 2018.Eligibility criteriaWe included PE alongside randomised controlled trials of primary care interventions which aimed to improve outcomes for patients with non-communicable diseases.Data extraction and synthesisTwo independent reviewers screened and conducted the data extraction and synthesis, with a third reviewer checking a sample for quality assurance.Results69 studies were included. There was an overall lack of consistency in how PEs were conducted and reported. The main weakness is that only 30 studies were underpinned by a clear intervention theory often facilitated by the use of existing theoretical frameworks. The main strengths were robust sampling strategies, and the triangulation of qualitative and quantitative data to understand an intervention's mechanisms. Findings were synthesised into three key themes: (1) a fundamental mismatch between what the intervention was designed to achieve and local needs; (2) the required roles and responsibilities of key actors were often not clearly understood; and (3) the health system context-factors such as governance, financing structures and workforce-if unanticipated could adversely impact implementation.ConclusionGreater consistency is needed in the reporting and the methods of PEs, in particular greater use of theoretical frameworks to inform intervention theory. More emphasis on formative research in designing interventions is needed to align the intervention with the needs of local stakeholders, and to minimise unanticipated consequences due to context-specific barriers.Prospero registration numberCRD42016035572.

Project description:BACKGROUND:The evaluation of web-based interventions (defined as an intervention that can be downloaded or accessed on the internet through a web browser) in randomized controlled trials (RCTs) has increased over the past two decades. Little is known about how participants' use of the intervention is measured, reported, and analyzed in these studies. OBJECTIVE:This study aimed to review the evaluation of web-based interventions in RCTs, assessing study characteristics and the methods used to record, and adjust for, intervention usage. METHODS:A systematic review of the literature was undertaken to identify all published reports of RCTs that involved a web-based intervention. A random sample of 100 published trials was selected for detailed data extraction. Information on trial characteristics was extracted, including whether web usage data were recorded, and if so, the methods used to gather these data and whether these data were used to inform efficacy analyses. RESULTS:A PubMed search identified 812 trials of web-based interventions published up to the end of 2017 and demonstrated a growing trend over time. Of the 100 studies reviewed, 90 studies collected web usage data, but more than half (49/90, 54%) of these studies did not state the method used for recording web usage. Only four studies attempted to check on the reliability of their web usage data collection methods. A total of 39% (35/90) studies reported patterns or levels of web intervention use, of which 21% (19/90) studies adjusted for intervention use in their outcome analysis, but only two of these used appropriate statistical methods. CONCLUSIONS:Trialists frequently report a measure of web-based intervention usage but do not always report the collection method or provide enough detail on their analysis of web usage. Appropriate statistical methods to account for intervention use are rarely used and are not well reported even in the very few trials in which they are used. The number of trialists who attempt to check on the reliability of their web usage collection methods is extremely low.

Project description:INTRODUCTION:The effectiveness of complex interventions for the management of musculoskeletal disorders has been estimated in many randomised clinical trials (RCTs). These trials inform which interventions are the most effective, however they do not always inform how an intervention achieved its clinical outcomes, nor how and what elements of an intervention were delivered to patients. Such information is useful for translating findings into clinical practice. A few process evaluation studies have been conducted alongside RCTs and a variety of methods have been used. To gain a better understanding of current practices of process evaluation in RCTs in musculoskeletal disorders, this systematic review is designed to answer the following research question: How are process evaluation of complex interventions tested in RCTs in musculoskeletal disorders being conducted? METHODS AND ANALYSIS:We will systematically search seven electronic databases (MEDLINE, SCOPUS, CINAHL, PsycINFO, EMBASE, Web of Science and Cochrane database) from the date of inception to August 2018 for studies on process evaluation of RCTs on non-surgical and non-pharmacological management of musculoskeletal disorders. We will include qualitative and quantitative studies conducted alongside RCTs, reported with the RCTs or separate studies that assessed interventions for musculoskeletal disorders. Two reviewers will screen abstracts and apply prespecified inclusion criteria to identify relevant studies, extract the data and assess the risk of bias within included studies. We will follow recommendations from the 'Cochrane Qualitative and Implementation Methods Group Guidance Series' when assessing methodological strengths and limitations of included studies. We will use a narrative synthesis to describe findings. ETHICS AND DISSEMINATION:Ethical approval is not required as this review will not collect original data. Findings from this systematic review will be presented at a scientific conference and published in a peer reviewed journal. PROSPERO REGISTRATION NUMBER:CRD42018109600.

Project description:BackgroundContext matters for the successful implementation of medical interventions, but its role remains surprisingly understudied. Against the backdrop of antimicrobial resistance, a global health priority, we investigated the introduction of a rapid diagnostic biomarker test (C-reactive protein, or CRP) to guide antibiotic prescriptions in outpatient settings and asked, "Which factors account for cross-country variations in the effectiveness of CRP biomarker test interventions?"MethodsWe conducted a cross-case comparison of CRP point-of-care test trials across Yangon (Myanmar), Chiang Rai (Thailand), and Hanoi (Vietnam). Cross-sectional qualitative data were originally collected as part of each clinical trial to broaden their evidence base and help explain their respective results. We synthesised these data and developed a large qualitative data set comprising 130 interview and focus group participants (healthcare workers and patients) and nearly one million words worth of transcripts and interview notes. Inductive thematic analysis was used to identify contextual factors and compare them across the three case studies. As clinical trial outcomes, we considered patients' and healthcare workers' adherence to the biomarker test results, and patient exclusion to gauge the potential "impact" of CRP point-of-care testing on the population level.ResultsWe identified three principal domains of contextual influences on intervention effectiveness. First, perceived risks from infectious diseases influenced the adherence of the clinical users (nurses, doctors). Second, the health system context related to all three intervention outcomes (via the health policy and antibiotic policy environment, and via health system structures and the ensuing utilisation patterns). Third, the demand-side context influenced the patient adherence to CRP point-of-care tests and exclusion from the intervention through variations in local healthcare-seeking behaviours, popular conceptions of illness and medicine, and the resulting utilisation of the health system.ConclusionsOur study underscored the importance of contextual variation for the interpretation of clinical trial findings. Further research should investigate the range and magnitude of contextual effects on trial outcomes through meta-analyses of large sets of clinical trials. For this to be possible, clinical trials should collect qualitative and quantitative contextual information for instance on their disease, health system, and demand-side environment.Trial registrationClinicalTrials.gov, NCT02758821 registered on 3 May 2016 and NCT01918579 registered on 7 August 2013.

Project description:BackgroundWithin increasingly constrained school timetables, interventions that integrate academic and health education to reduce substance use and violence may hold promise as a category of intervention that can positively affect both academic and health outcomes. There are no current systematic reviews exploring the effectiveness of such interventions or factors that affect their implementation.MethodsA total of 19 bibliographic databases and 32 websites were searched. References were also extracted from the reference lists of included studies, and experts and authors were contacted to identify relevant studies. We included reports with no restrictions on language or date. References were screened on title/abstract and those not thus excluded were screened on full report. Data extraction and appraisal followed the Critical Appraisal Skills Programme, Evidence for Policy and Practice Information and Co-ordinating Centre and Cochrane tools. Extracted process data were qualitatively meta-synthesised for common themes.ResultsSeventy-eight thousand four hundred fifty-one unique references were identified, and 62 reports were included. A total of 16 reports (reporting on 15 studies of 12 interventions) evaluated process. Key facilitators of integrated academic and health curricula were supportive senior management and alignment of the intervention with school ethos; a positive teaching environment, including positive perceptions around the ability to be flexible in the adaptation and delivery of integrated academic and health curricula; positive pre-existing student and teacher attitudes towards intervention content; and parental support of interventions, largely through reinforcement of messaging at home. Important barriers were over-burdened teachers, with little time to learn and implement integrated curricula.ConclusionSeveral useful facilitating and inhibiting factors linked to the implementation of interventions that integrate academic and health education for reduced substance use and/or violence were identified, providing tentative but insightful evidence of context-specific issues that may impact intervention success. However, overall, there is still a considerable gap in our understanding of how to achieve the successful implementation of these interventions.

Dataset Information

Designing process evaluations using case study to explore the context of complex interventions evaluated in trials.

Publications

Designing process evaluations using case study to explore the context of complex interventions evaluated in trials.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets