Dataset Information

What to expect when you're evaluating healthcare improvement: a concordat approach to managing collaboration and uncomfortable realities.

ABSTRACT: Evaluation of improvement initiatives in healthcare is essential to establishing whether interventions are effective and to understanding how and why they work in order to enable replication. Although valuable, evaluation is often complicated by tensions and friction between evaluators, implementers and other stakeholders. Drawing on the literature, we suggest that these tensions can arise from a lack of shared understanding of the goals of the evaluation; confusion about roles, relationships and responsibilities; data burdens; issues of data flows and confidentiality; the discomforts of being studied and the impact of disappointing or otherwise unwelcome results. We present a possible approach to managing these tensions involving the co-production and use of a concordat. We describe how we developed a concordat in the context of an evaluation of a complex patient safety improvement programme known as Safer Clinical Systems Phase 2. The concordat development process involved partners (evaluators, designers, funders and others) working together at the outset of the project to agree a set of principles to guide the conduct of the evaluation. We suggest that while the concordat is a useful resource for resolving conflicts that arise during evaluation, the process of producing it is perhaps even more important, helping to make explicit unspoken assumptions, clarify roles and responsibilities, build trust and establish open dialogue and shared understanding. The concordat we developed established some core principles that may be of value for others involved in evaluation to consider. But rather than seeing our document as a ready-made solution, there is a need for recognition of the value of the process of co-producing a locally agreed concordat in enabling partners in the evaluation to work together effectively.

SUBMITTER: Brewster L

PROVIDER: S-EPMC4413682 | biostudies-literature | 2015 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

What to expect when you're evaluating healthcare improvement: a concordat approach to managing collaboration and uncomfortable realities.

Brewster Liz L Aveling Emma-Louise EL Martin Graham G Tarrant Carolyn C Dixon-Woods Mary M

BMJ quality & safety 20150402 5

Evaluation of improvement initiatives in healthcare is essential to establishing whether interventions are effective and to understanding how and why they work in order to enable replication. Although valuable, evaluation is often complicated by tensions and friction between evaluators, implementers and other stakeholders. Drawing on the literature, we suggest that these tensions can arise from a lack of shared understanding of the goals of the evaluation; confusion about roles, relationships an ...[more]

PMID: 25838466

Similar Datasets

Project description:Effective implementation of change in healthcare organisations involves multiple professional and organisational groups and is often impeded by professional and organisational boundaries that present relatively impermeable barriers to sharing knowledge and spreading work practices. Informed by the theory of communities of practice (CoPs), this study explored the effects of intra-organisational and inter-organisational boundaries on the implementation of service improvement within and across primary healthcare settings and on the development of multiprofessional and multi-organisational CoPs during this process.The study was conducted within the Collaboration for Leadership in Applied Health Research and Care (CLAHRC) for Greater Manchester-a collaborative partnership between the University of Manchester and local National Health Service organisations aiming to undertake applied health research and enhance its implementation in clinical practice. It deployed a qualitative embedded case study design, encompassing semistructured interviews, direct observation and documentary analysis, conducted in 2010-2011. The sample included practice doctors, nurses, managers and members of the CLAHRC implementation team.The study showed that in spite of epistemic and status differences, professional boundaries between general practitioners, practice nurses and practice managers co-located in the same practice over a relatively long period of time could be successfully bridged, leading to the formation of multiprofessional CoPs. While knowledge circulated relatively easily within these CoPs, barriers to knowledge sharing emerged at the boundary separating them from other groups existing in the same primary care setting. The strongest boundaries, however, lay between individual general practices, with inter-organisational knowledge sharing and collaboration between them remaining unequally developed across different areas due to historical factors, competition and strong organisational identification. Manipulated emergence of multi-organisational CoPs in the context of primary care may thus be problematic.In cases when manipulated emergence of new CoPs is problematic, boundary issues could be addressed by adopting a developmental perspective on CoPs, which provides an alternative to the analytical and instrumental perspectives previously described in the CoP literature. This perspective implies a pragmatic, situational approach to mapping existing CoPs and their characteristics and potentially modifying them in the process of service improvement through the combination of internal and external facilitation.

Project description:Background: The ABCD National Research Partnership was formed in mid-2010 as a collaboration to harness the expertise, experiences and resources of Aboriginal and Torres Strait Islander community-controlled peak bodies, government and research organisations to improve the quality of Indigenous primary health care. The aim of this study was to apply social network methods to assess collaboration and functioning of the Partnership at two time-points. Methods: A social network analysis (SNA) survey was conducted in early 2013, with a follow-up survey in mid-2014. In the two survey rounds, online surveys were emailed to one senior person of the organisation participating in the Partnership (2013: 14 organisations; 2014: 11 organisations). The surveys collected data on respondent perceptions of the Partnership as well as social network relationship data. Social network methods were used to apply standardised metrics to assess how well the partnership was functioning as a collaborative three years into its operation, and in its fourth year. Results: Most respondents rated the Partnership as successful in progressing toward its goals. Network density and centrality scores show a well-connected partnership spanning different organisational types and states/territories (Northern Territory, Queensland, Western Australia, South Australia, and Far-West New South Wales). High centrality scores reflect high connectivity between key hubs in the network, contributing toward the shared goal of improved Indigenous primary health care. Network diagrams show key structural positions by organisational type, the frequency and intensity of interactions and the strengths and potential vulnerabilities in the partnership network, with comparisons at two time points for the partnership. Conclusions: The study found that the Partnership was effective in securing collaboration across its partners. Partners' contribution of resources reflected their active involvement. There was a high level of agreement on the achievement of the key goals of the Partnership, showing shared sense-making amongst partners. SNA tools assisted with monitoring the network over time to develop strategies supporting connections between partners for sustaining collaborative learning. Study findings identify successful approaches for a research partnership to improve quality of care in Indigenous primary health care and provide encouragement for wider applications for research partnerships and collaborations in Australia and internationally.

Project description:ObjectivesThough multidisciplinary research networks support the practice and effectiveness of continuous quality improvement (CQI) programmes, their characteristics and development are poorly understood. In this study, we examine publication outputs from a research network in Australian Indigenous primary healthcare (PHC) to assess to what extent the research network changed over time.SettingAustralian CQI research network in Indigenous PHC from 2002 to 2019.ParticipantsAuthors from peer-reviewed journal articles and books published by the network.DesignCoauthor networks across four phases of the network (2002-2004; 2005-2009; 2010-2014; 2015-2019) were constructed based on author affiliations and examined using social network analysis methods. Descriptive characteristics included organisation types, Indigenous representation, gender, student authorship and thematic research trends.ResultsWe identified 128 publications written by 308 individual authors from 79 different organisations. Publications increased in number and diversity over each funding phase. During the final phase, publication outputs accelerated for organisations, students, project officers, Indigenous and female authors. Over time there was also a shift in research themes to encompass new clinical areas and social, environmental or behavioural determinants of health. Average degree (8.1), clustering (0.81) and diameter (3) indicated a well-connected network, with a core-periphery structure in each phase (p≤0.03) rather than a single central organisation (degree centralisation=0.55-0.65). Academic organisations dominated the core structure in all funding phases.ConclusionCollaboration in publications increased with network consolidation and expansion. Increased productivity was associated with increased authorship diversity and a decentralised network, suggesting these may be important factors in enhancing research impact and advancing the knowledge and practice of CQI in PHC. Publication diversity and growth occurred mainly in the fourth phase, suggesting long-term relationship building among diverse partners is required to facilitate participatory research in CQI. Despite improvements, further work is needed to address inequities in female authorship and Indigenous authorship.

Project description:IntroductionAs demand for healthcare continues to grow, public health systems are increasingly required to drive efficiency improvement (EI) to address public service funding challenges. Despite this requirement, evidence of EI strategies that have been successful applied at the whole-of-system level is limited. This study reports the development, implementation and evaluation of a novel taxonomy of EI strategies used in public health systems to inform systemwide EI models.Materials and methodsThe public health system in New South Wales, Australia, operates a centralised EI model statewide and was the setting for this study. An audit of EI strategies implemented in the NSW Health system between July 2016 and June 2019 was used to identify all available EI strategies within the study timeframe. A content management approach was applied to audit the strategies, with each strategy coded to an EI focus area. Codes were clustered according to similarity, and category names were assigned to each cluster to form a preliminary taxonomy. Each category was defined and examples were provided. The resulting taxonomy was distributed and evaluated by user feedback survey and pre-post study to assess the impact on EI strategy distribution.ResultsA total of 1127 EI strategies were identified and coded into 263 unique strategies, which were clustered into nine categories to form the taxonomy of EI strategies. Categories included the following: non-clinical contracts and supplies; avoided and preventable activity; clinical service delivery and patient outcomes; finance and operations; recruitment, vacancies and FTE; staffing models; leave management; staff engagement and development; premium staffing; and clinical contracts and supplies. Evaluative findings revealed a perceived reduction in the duplication of EI work, improved access to EI knowledge and improved engagement with EI processes when using the taxonomy. The taxonomy was also associated with wider use of EI strategies.ConclusionsWhole-of-system EI is an increasing requirement. Using a taxonomy to guide systemwide practice appears to be advanta-geous in reducing duplication and guiding practice, with implications for use in health systems internationally.

Dataset Information

What to expect when you're evaluating healthcare improvement: a concordat approach to managing collaboration and uncomfortable realities.

Publications

What to expect when you're evaluating healthcare improvement: a concordat approach to managing collaboration and uncomfortable realities.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets