Dataset Information

Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals.

ABSTRACT: OBJECTIVES:To extend reliability of WHO Behaviourally Anchored Rating Scale (WHOBARS) to measure the quality of WHO Surgical Safety Checklist administration using generalisability theory. In this context, extending reliability refers to establishing generalisability of the tool scores across populations of teams and raters by accounting for the relevant sources of measurement errors. DESIGN:Cross-sectional random effect measurement design assessing surgical teams by the five items on the three Checklist phases, and at three sites by two trained raters simultaneously. SETTING:The data were collected in three tertiary hospitals in Auckland, New Zealand in 2016 and included 60 teams observed in 60 different cases with an equal number of teams (n=20) per site. All elective and acute cases (adults and children) involving surgery under general anaesthesia during normal working hours were eligible. PARTICIPANTS:The study included 243 surgical staff members, 138 (50.12%) women. MAIN OUTCOME MEASURE:Absolute generalisability coefficient that accounts for variance due to items, phases, sites and raters for the WHOBARS measure of the quality of WHO Surgical Safety Checklist administration. RESULTS:The WHOBARS in its present form has demonstrated good generalisability of scores across teams and raters (G absolute=0.83). The largest source of measurement error was the interaction between the surgical team and the rater, accounting for 16.7% (95% CI 16.4 to 16.9) of the total variance in the data. Removing any items from the WHOBARS led to a decrease in the overall reliability of the instrument. CONCLUSIONS:Assessing checklist administration quality is important for promoting improvement in its use, and WHOBARS offers a reliable approach for doing this.

SUBMITTER: Medvedev ON

PROVIDER: S-EPMC6340010 | biostudies-literature | 2019 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals.

Medvedev Oleg N ON Merry Alan F AF Skilton Carmen C Gargiulo Derryn A DA Mitchell Simon J SJ Weller Jennifer M JM

BMJ open 20190109 1

<h4>Objectives</h4>To extend reliability of WHO Behaviourally Anchored Rating Scale (WHOBARS) to measure the quality of WHO Surgical Safety Checklist administration using generalisability theory. In this context, extending reliability refers to establishing generalisability of the tool scores across populations of teams and raters by accounting for the relevant sources of measurement errors.<h4>Design</h4>Cross-sectional random effect measurement design assessing surgical teams by the five items ...[more]

PMID: 30782682

Similar Datasets

Project description:While the WHO Surgical Safety Checklist (the Checklist) can improve patient outcomes, variable administration can erode benefits. We sought to understand and improve how operating room (OR) staff use the Checklist. Our specific aims were to: determine if OR staff can discriminate between good and poor quality of Checklist administration using a validated audit tool (WHOBARS); to determine reliability and accuracy of WHOBARS self-ratings; determine the influence of demographic variables on ratings and explore OR staff attitudes to Checklist administration. DESIGN:Mixed methods study using WHOBARS ratings of surgical cases by OR staff and two independent observers, thematic analysis of staff interviews. PARTICIPANTS:OR staff in three New Zealand hospitals. OUTCOME MEASURES:Reliability of WHOBARS for self-audit; staff attitudes to Checklist administration. RESULTS:Analysis of scores (243 participants, 2 observers, 59 cases) supported tool reliability, with 87% of WHOBARS score variance attributable to differences in Checklist administration between cases. Self-ratings were significantly higher than observer ratings, with some differences between professional groups but error variance from all raters was less than 10%. Key interview themes (33 interviewees) were: Team culture and embedding the Checklist, Information transfer and obstacles, Raising concerns and 'A tick-box exercise'. Interviewees felt the Checklist could promote teamwork and a safety culture, particularly enabling speaking up. Senior staff were of key importance in setting the appropriate tone. CONCLUSIONS:The WHOBARS tool could be useful for self-audit and quality improvement as OR staff can reliably discriminate between good and poor Checklist administration. OR staff self-ratings were lenient compared with external observers suggesting the value of external audit for benchmarking. Small differences between ratings from professional groups underpin the value of including all members of the team in scoring. We identified factors explaining staff perceptions of the Checklist that should inform quality improvement interventions.

Project description:The surgical Safety Checklist (SSC) was introduced in 2008 to improve teamwork and reduce the mortality and morbidity associated with surgery. Although mandated in many health care institutions around the world, challenges in implementation of the SSC continue. To use Normalisation Process Theory (NPT) to help understand how/why implementation of a complex intervention coined Pass The Baton (PTB) could help explain what facets of the Surgical Safety Checklist use led to its' integration in practice, while others were not.A longitudinal multi-method study using survey and interviews was undertaken. Implementation of PTB involved; change champions, audit and feedback, education and prompts. Following implementation, surgical teams were surveyed using the NOrmalization MeAsure Development (NoMAD) and subsequently interviewed to explore the impact of PTB on their use of the checklist at 6 and 12 months respectively. Respondents' self-reported perceptions of implementation of PTB was explained using the four NPT constructs; coherence, cognitive participation, collective action, and reflexive monitoring. Survey data were analysed using descriptive statistics. Interview data were coded inductively and content analysed using a framework derived from NPT.The NoMAD survey response rate was 59/150 (39.3%). Many (45/59, 77.6%) survey respondents saw the value in PTB, while 50/59 (86.2%) would continue to use it; 45/59 (77.6%) believed that PTB could easily be integrated into existing workflows, and 48/59 (82.8%) thought that feedback could improve PTB in the future. A total of 8 interviews were completed with 26 surgical team members. Nurses and physicians held mixed views towards coherence while buy-in and participation relied on individuals' investment in the implementation process and the ability to modify PTB. Participants generally recognised the benefit and value of using PTB in the ongoing implementation the checklist.Workarounds and flexible co-construction in implementation designed to improve team communications in surgery may facilitate their normalisation in practice.

Project description:ObjectiveBold claims have been made for the ability of the WHO surgical checklist to reduce surgical morbidity and mortality and improve patient safety regardless of the setting. Little is known about how far the challenges faced by low-income countries are the same as those in high-income countries or different. We aimed to identify and compare the influences on checklist implementation and compliance in the UK and Africa.DesignEthnographic study involving observations, interviews and collection of documents. Thematic analysis of the data.SettingOperating theatres in one African university hospital and two UK university hospitals.Participants112 h of observations were undertaken. Interviews with 39 theatre and administrative staff were conducted.ResultsMany staff saw value in the checklist in the UK and African hospitals. Some resentment was present in all settings, linked to conflicts between the philosophy behind the checklist and the realities of local cultural, social and economic contexts. Compliance-involving use, completeness and fidelity-was considerably higher, though not perfect, in the UK settings. In these hospitals, compliance was supported by established structures and systems, and was not significantly undermined by major resource constraints; the same was not true of the low-income context. Hierarchical relationships were a major barrier to implementation in all settings, but were more marked in the low-income setting. Introducing a checklist in a professional environment characterised by a lack of accountability and transparency could make the staff feel jeopardised legally, professionally, and personally, and it encouraged them to make misleading records of what had actually been done.ConclusionsSurgical checklist implementation is likely to be optimised, regardless of the setting, when used as a tool in multifaceted cultural and organisational programmes to strengthen patient safety. It cannot be assumed that the introduction of a checklist will automatically lead to improved communication and clinical processes.

Project description:Background and aimsMultiple national guidelines stress the importance for clinicians to possess good therapeutic skills for working with patients with significant relational difficulties (who may receive a diagnosis of personality disorder). Training clinicians in mentalization-based treatment skills (MBT-S) is one approach to address this. The main outcome measure used in MBT-S studies is the Knowledge and Application of MBT Questionnaire (KAMQ). However, an absence of research into the properties and validity of the KAMQ has limited the methodological quality of MBT-S evaluations so far. The aim of this study was therefore to investigate the factor structure, internal consistency, reliability, and validity of the KAMQ.MethodsUsing an existing multiprofessional sample of 217 clinicians from 2014 to 2016, we undertook exploratory factor analysis to determine the factor structure and internal consistency of the KAMQ. Convergent validity of the measure with the Attitudes to Personality Disorder Questionnaire (APDQ) was assessed in a subset of this dataset where both questionnaires had been administered (n = 92). Additionally, by recruiting a new sample of 70 clinicians, we assessed the measure's test-retest reliability.ResultsFactor analysis found three factors underlying 17 of the 20 KAMQ items, relating to therapeutic skills in mentalizing, beliefs about applying MBT in practice, and specific MBT knowledge. The KAMQ was revised following the factor analysis to form the KAMQ-2 with 17 items. Internal consistency (α = .85, 95% confidence interval [CI] = 0.80-0.89) and test-retest reliability (ICC = 0.84, 95% CI = 0.73-0.91) were good. In correlation analyses, the KAMQ-2 showed convergent validity with the main factor from the APDQ (n = 48; r s = 0.39, P < .01).ConclusionThe KAMQ-2 provides a short, reliable self-report instrument which probes clinicians' knowledge about mentalizing skills, and beliefs about using these. There was preliminary evidence for validity. The properties of the KAMQ-2 mean that more robust evaluation and development of MBT-S is now possible.

Project description:BACKGROUND:Many countries are engaged in primary care reforms to support older adults who are living longer in the community. Health Teams Advancing Patient Experience: Strengthening Quality [Health TAPESTRY] is a primary care intervention aimed at supporting older adults that involves trained volunteers, interprofessional teams, technology, and system navigation. This paper examines implementation of Health TAPESTRY in relation to interprofessional teamwork including volunteers. METHODS:This study applied Normalization Process Theory (NPT) and used a descriptive qualitative approach [1] embedded in a mixed-methods, pragmatic randomized controlled trial. It was situated in two primary care practice sites in a large urban setting in Ontario, Canada. Focus groups and interviews were conducted with primary care providers, clinical managers, administrative assistants, volunteers, and a volunteer coordinator. Data was collected at 4?months (June-July 2015) and 12?months (February-March 2016) after intervention start-up. Patients were interviewed at the end of the six-month intervention. Field notes were taken at weekly huddle meetings. RESULTS:Overall, 84 participants were included in 17 focus groups and 13 interviews; 24 field notes were collected. Themes were organized under four NPT constructs of implementation: 1) Coherence- (making sense/understanding of the program's purpose/value) generating comprehensive assessments of older adults; strengthening health promotion, disease prevention, and self-management; enhancing patient-focused care; strengthening interprofessional care delivery; improving coordination of health and community services. 2) Cognitive Participation- (enrolment/buy-in) tackling new ways of working; attaining role clarity. 3) Collective Action- (enactment/operationalizing) changing team processes; reconfiguring resources. 4) Reflective Monitoring- (appraisal) improving teamwork and collaboration; reconfiguring roles and processes. CONCLUSIONS:This study contributes key strategies for effective implementation of interventions involving interprofessional primary care teams. Findings indicate that regular communication among all team members, the development of procedures and/or protocols to support team processes, and ongoing review and feedback are critical to implementation of innovations involving primary care teams. TRIAL REGISTRATION:ClinicalTrials.gov, no. NCT02283723 November 5, 2014. Prospectively registered.

Project description:INTRODUCTION:NetworkZ is a national, insurer-funded multidisciplinary simulation-based team-training programme for all New Zealand surgical teams. NetworkZ is delivered in situ, using full-body commercial simulators integrated with bespoke surgical models. Rolled out nationally over 4?years, the programme builds local capacity through instructor training and provision of simulation resources. We aim to improve surgical patient outcomes by improving teamwork through regular simulation-based multidisciplinary training in all New Zealand hospitals. METHODS AND ANALYSIS:Our primary hypothesis is that surgical patient outcomes will improve following NetworkZ. Our secondary hypotheses are that teamwork processes will improve, and treatment injury claims will decline. In addition, we will explore factors that influence implementation and sustainability of NetworkZ and identify organisational changes following its introduction. The study uses a stepped-wedge cluster design. The intervention will roll out at yearly intervals to four cohorts of five District Health Boards. Allocation to cohort was purposive for year 1, and subsequently randomised. The primary outcome measure is Days Alive and Out of Hospital at 90 days using patient data from an existing national administrative database. Secondary outcomes measures will include analysis of postoperative complications and treatment injury claims, surveys of teamwork and safety culture, in-theatre observations and stakeholder interviews. ETHICS AND DISSEMINATION:We believe this is the first surgical team training intervention to be implemented on a national scale, and a unique opportunity to evaluate a nation-wide team-training intervention for healthcare teams. By using a pre-existing large administrative data set, we have the potential to demonstrate a difference to surgical patient outcomes. This will be of interest to those working in the field of healthcare teamwork, quality improvement and patient safety. New Zealand Health and Disability Ethic Committee approval (#16/NTB/143). TRIAL REGISTRATION NUMBER:Australian and New Zealand Clinical Trials Registry ID ACTRN12617000017325 and the Universal Trial Number is U1111-1189-3992.

Dataset Information

Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals.

Publications

Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure