Dataset Information

The use of Qualitative Comparative Analysis (QCA) to address causality in complex systems: a systematic review of research on public health interventions.

ABSTRACT:

Background

Qualitative Comparative Analysis (QCA) is a method for identifying the configurations of conditions that lead to specific outcomes. Given its potential for providing evidence of causality in complex systems, QCA is increasingly used in evaluative research to examine the uptake or impacts of public health interventions. We map this emerging field, assessing the strengths and weaknesses of QCA approaches identified in published studies, and identify implications for future research and reporting.

Methods

PubMed, Scopus and Web of Science were systematically searched for peer-reviewed studies published in English up to December 2019 that had used QCA methods to identify the conditions associated with the uptake and/or effectiveness of interventions for public health. Data relating to the interventions studied (settings/level of intervention/populations), methods (type of QCA, case level, source of data, other methods used) and reported strengths and weaknesses of QCA were extracted and synthesised narratively.

Results

The search identified 1384 papers, of which 27 (describing 26 studies) met the inclusion criteria. Interventions evaluated ranged across: nutrition/obesity (n = 8); physical activity (n = 4); health inequalities (n = 3); mental health (n = 2); community engagement (n = 3); chronic condition management (n = 3); vaccine adoption or implementation (n = 2); programme implementation (n = 3); breastfeeding (n = 2), and general population health (n = 1). The majority of studies (n = 24) were of interventions solely or predominantly in high income countries. Key strengths reported were that QCA provides a method for addressing causal complexity; and that it provides a systematic approach for understanding the mechanisms at work in implementation across contexts. Weaknesses reported related to data availability limitations, especially on ineffective interventions. The majority of papers demonstrated good knowledge of cases, and justification of case selection, but other criteria of methodological quality were less comprehensively met.

Conclusion

QCA is a promising approach for addressing the role of context in complex interventions, and for identifying causal configurations of conditions that predict implementation and/or outcomes when there is sufficiently detailed understanding of a series of comparable cases. As the use of QCA in evaluative health research increases, there may be a need to develop advice for public health researchers and journals on minimum criteria for quality and reporting.

SUBMITTER: Hanckel B

PROVIDER: S-EPMC8103124 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundChildhood language development is related to long term educational, employment, health and social outcomes. Previous research identifies a complex range of risk and protective factors which result in good and poor language outcomes for children, however children at risk are an underrepresented group in these studies. Our aim is to investigate the combinations of factors (paths) that result in good and poor language outcomes for a group of 5 year old children of mothers experiencing adversity.MethodsThis mixed methods study utilised longitudinal data from a randomised control trial of sustained home visiting (MECSH) to determine the language outcomes in at risk children. Mothers were randomly assigned to a comparison group at entry to the study (prior to child's birth). Their children who were retained at entry to school completed language assessments (n = 41) and were participants in this study. Influence of 13 key factors derived from the extant literature that impact language development were explored. Regression was used to determine the six key factors of influence and these were used in the Qualitative Comparative Analysis (QCA). QCA was employed to examine the necessary and sufficient conditions and paths affecting language development linked to good and poor language outcomes. A post hoc analysis of the risk and protective paths to good and poor language outcomes was also conducted.ResultsThirteen distinct pathways led to good language outcomes and four paths to poor language outcomes in five year old at risk children. A variety of condition combinations resulted in these outcomes, with maternal responsivity, toddler development and number of children in the home being key. High and low maternal education influenced both good and poor language development.ConclusionsThe paths to good and poor language outcomes were different and complex. Most paths to a good language outcome involved protective factors, though not always. In addition, paths to poor language more often involved risk factors. The varied patterns of risk and protective factors point to the need for interventions across the first five years of life in both health and education for families which can respond to these risk and protective patterns.Trial registrationThe original RCT was retrospectively registered in the ANCTR: ACTRN12608000473369 .

Project description:OBJECTIVE:To examine the elements of capacity, a measure of organizational resources supporting program implementation that result in successful completion of public health program objectives in a public health initiative serving 50 communities. DESIGN:We used crisp set Qualitative Comparative Analysis (QCA) to analyze case study and quantitative data collected during the evaluation of the Communities Putting Prevention to Work (CPPW) program. SETTING:CPPW awardee program staff and partners implemented evidence-based public health improvements in counties, cities, and organizations (eg, worksites, schools). PARTICIPANTS:Data came from case studies of 22 CPPW awardee programs that implemented evidence-based, community- and organizational-level public health improvements. INTERVENTION:Program staff implemented a range of evidence-based public health improvements related to tobacco control and obesity prevention. MAIN OUTCOME MEASURE:The outcome measure was completion of approximately 60% of work plan objectives. RESULTS:Analysis of the capacity conditions revealed 2 combinations for completing most work plan objectives: (1) having experience implementing public health improvements in combination with having a history of collaboration with partners; and (2) not having experience implementing public health improvements in combination with having leadership support. CONCLUSION:Awardees have varying levels of capacity. The combinations identified in this analysis provide important insights into how awardees with different combinations of elements of capacity achieved most of their work plan objectives. Even when awardees lack some elements of capacity, they can build it through strategies such as hiring staff and engaging new partners with expertise. In some instances, lacking 1 or more elements of capacity did not prevent an awardee from successfully completing objectives. These findings can help funders and practitioners recognize and assemble different aspects of capacity to achieve more successful programs; awardees can draw on extant organizational strengths to compensate when other aspects of capacity are absent.

Project description:BackgroundComplex healthcare interventions consist of multiple components which may vary in trials conducted in different populations and contexts. Pooling evidence from trials in a systematic review is challenging because it is unclear which components are needed for effectiveness. The potential is recognised for using recipients' views to explore why some complex interventions are effective and others are not. Methods to maximise this potential are poorly developed.MethodsWe used a novel approach to explore how patients' views may explain the disparity in effectiveness of complex interventions. We used qualitative comparative analysis to explore agreement between qualitative syntheses of data on patients' views and evidence from trialed interventions to increase adherence to treatments. We first populated data matrices to reflect whether the content of each trialed intervention could be matched with suggestions arising from patients' views. We then used qualitative comparative analysis software to identify, by a process of elimination, the smallest number of configurations (patterns) of components that corresponded with patients' suggestions and accounted for whether each intervention was effective or ineffective.ResultsWe found suggestions by patients were poorly represented in interventions. Qualitative comparative analysis identified particular combinations of components corresponding with patients' suggestions and with whether an intervention was effective or ineffective. Six patterns were identified for an effective and four for an ineffective intervention. Two types of patterns arose for the effective interventions, one being didactic (providing clear information or instruction) and the other interactive (focusing on personal risk factors).ConclusionsOur analysis highlights how data on patients' views has the potential to identify key components across trials of complex interventions or inform the content of new interventions to be trialed.

Project description:BACKGROUND:Public health evaluation methods have been criticized for being overly reductionist and failing to generate suitable evidence for public health decision-making. A "complex systems approach" has been advocated to account for real world complexity. Qualitative methods may be well suited to understanding change in complex social environments, but guidance on applying a complex systems approach to inform qualitative research remains limited and underdeveloped. This systematic review aims to analyze published examples of process evaluations that utilize qualitative methods that involve a complex systems perspective and proposes a framework for qualitative complex system process evaluations. METHODS AND FINDINGS:We conducted a systematic search to identify complex system process evaluations that involve qualitative methods by searching electronic databases from January 1, 2014-September 30, 2019 (Scopus, MEDLINE, Web of Science), citation searching, and expert consultations. Process evaluations were included if they self-identified as taking a systems- or complexity-oriented approach, integrated qualitative methods, reported empirical findings, and evaluated public health interventions. Two reviewers independently assessed each study to identify concepts associated with the systems thinking and complexity science traditions. Twenty-one unique studies were identified evaluating a wide range of public health interventions in, for example, urban planning, sexual health, violence prevention, substance use, and community transformation. Evaluations were conducted in settings such as schools, workplaces, and neighborhoods in 13 different countries (9 high-income and 4 middle-income). All reported some utilization of complex systems concepts in the analysis of qualitative data. In 14 evaluations, the consideration of complex systems influenced intervention design, evaluation planning, or fieldwork. The identified studies used systems concepts to depict and describe a system at one point in time. Only 4 evaluations explicitly utilized a range of complexity concepts to assess changes within the system resulting from, or co-occurring with, intervention implementation over time. Limitations to our approach are including only English-language papers, reliance on study authors reporting their utilization of complex systems concepts, and subjective judgment from the reviewers relating to which concepts featured in each study. CONCLUSION:This study found no consensus on what bringing a complex systems perspective to public health process evaluations with qualitative methods looks like in practice and that many studies of this nature describe static systems at a single time point. We suggest future studies use a 2-phase framework for qualitative process evaluations that seek to assess changes over time from a complex systems perspective. The first phase involves producing a description of the system and identifying hypotheses about how the system may change in response to the intervention. The second phase involves following the pathway of emergent findings in an adaptive evaluation approach.

Project description:BackgroundGrowing interest in public involvement in health research has led to organisational and policy change. Additionally, an emerging body of policy-oriented scholarship has begun to identify the organisational and network arrangements that shape public involvement activity. Such developments suggest the need to clearly conceptualise and characterise public involvement in health research in terms of governance.MethodsWe drew on an established health research system framework to analyse governance functions related to public involvement, adapting scoping review methods to identify evidence from a corpus of journal papers and policy reports. We drew on the logics of aggregation and top down configuration, using a qualitative interpretive approach to combine and link findings from different studies into framework categories.ResultsWe identified a total of 32 scholarly papers and 13 policy reports (n = 45 included papers) with relevance to governance for public involvement. Included papers were broadly consonant in identifying the need for activity to specify and support public involvement across all four governance functions of stewardship, financing, creating and sustaining resources, and research production and use. However, different visions for public involvement, and the activity required to implement it and achieve impact, were particularly evident with respect to the stewardship function, which seeks to set overall directions for research while addressing the potentially competing demands of a system's many constituents.ConclusionsA governance perspective has considerable value for public involvement in health research systems, supporting efforts to coordinate and institutionalise the burgeoning public involvement enterprise. Furthermore, it highlights challenges for what is, ultimately, a highly political intervention, suggesting that diverse publics must be both involved within health research systems and enrolled as governors of them.