Dataset Information

Evaluating the Instructional Design and Effect on Knowledge, Teamwork, and Skills of Technology-Enhanced Simulation-Based Training in Obstetrics in Uganda: Stepped-Wedge Cluster Randomized Trial.

ABSTRACT:

Background

Simulation-based training is a common strategy for improving the quality of facility-based maternity services and is often evaluated using Kirkpatrick's theoretical model. The results on the Kirkpatrick levels are closely related to the quality of the instructional design of a training program. The instructional design is generally defined as the "set of prescriptions for teaching methods to improve the quality of instruction with a goal of optimizing learning outcomes."

Objective

The aim of this study is to evaluate the instructional design of a technology-enhanced simulation-based training in obstetrics, the reaction of participants, and the effect on knowledge, teamwork, and skills in a low-income country.

Methods

A stepped-wedge cluster randomized trial was performed in a university hospital in Kampala, Uganda, with an annual delivery volume of over 31,000. In November 2014, a medical simulation center was installed with a full-body birthing simulator (Noelle S550, Gaumard Scientific), an interactive neonate (Simon S102 Newborn CPR Simulator, Gaumard Scientific), and an audio and video recording system. Twelve local obstetricians were trained and certified as medical simulation trainers. From 2014 to 2016, training was provided to 57 residents in groups of 6 to 9 students. Descriptive statistics were calculated for ten instructional design features of the training course measured by the 42-item ID-SIM (Instructional Design of a Simulation Improved by Monitoring). The Wilcoxon signed rank test was conducted to investigate the differences in scores on knowledge, the Clinical Teamwork Scale, and medical technical skills.

Results

The mean scores on the ten instructional design features ranged from 54.9 (95% CI 48.5-61.3) to 84.3 (95% CI 80.9-87.6) out of 100. The highest mean score was given on the feature feedback and the lowest scores on repetitive practice and controlled environment. The overall score for the training day was 92.8 out of 100 (95% CI 89.5-96.1). Knowledge improved significantly, with a test score of 63.4% (95% CI 60.7-66.1) before and 78.9% (95% CI 76.8-81.1) after the training (P<.001). The overall score on the 10-point Clinical Teamwork Scale was 6.0 (95% CI 4.4-7.6) before and 5.9 (95% CI 4.5-7.2) after the training (P=.78). Medical technical skills were scored at 55.5% (95% CI 47.2-63.8) before and 65.6% (95% CI 56.5-74.7) after training (P=.08).

Conclusions

Most instructional design features of a technology-enhanced simulation-based training in obstetrics in a low-income country were scored high, although intervals were large. The overall score for the training day was high, and knowledge did improve after the training program, but no changes in teamwork and (most) medical technical skills were found. The lowest-scored instructional design features may be improved to achieve further learning aims.

Trial registration

ISRCTN Registry ISRCTN98617255; http://www.isrctn.com/ISRCTN98617255.

International registered report identifier (irrid)

RR2-10.1186/s12884-020-03050-3.

SUBMITTER: van Tetering AAC

PROVIDER: S-EPMC8081249 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundAdherence to and completion of tuberculosis (TB) treatment remain problematic in many high-burden countries. 99DOTS is a low-cost digital adherence technology that could increase TB treatment completion.Methods and findingsWe conducted a pragmatic stepped-wedge cluster-randomized trial including all adults treated for drug-susceptible pulmonary TB at 18 health facilities across Uganda over 8 months (1 December 2018-31 July 2019). Facilities were randomized to switch from routine (control period) to 99DOTS-based (intervention period) TB treatment supervision in consecutive months. Patients were allocated to the control or intervention period based on which facility they attended and their treatment start date. Health facility staff and patients were not blinded to the intervention. The primary outcome was TB treatment completion. Due to the pragmatic nature of the trial, the primary analysis was done according to intention-to-treat (ITT) and per protocol (PP) principles. This trial is registered with the Pan African Clinical Trials Registry (PACTR201808609844917). Of 1,913 eligible patients at the 18 health facilities (1,022 and 891 during the control and intervention periods, respectively), 38.0% were women, mean (SD) age was 39.4 (14.4) years, 46.8% were HIV-infected, and most (91.4%) had newly diagnosed TB. In total, 463 (52.0%) patients were enrolled on 99DOTS during the intervention period. In the ITT analysis, the odds of treatment success were similar in the intervention and control periods (adjusted odds ratio [aOR] 1.04, 95% CI 0.68-1.58, p = 0.87). The odds of treatment success did not increase in the intervention period for either men (aOR 1.24, 95% CI 0.73-2.10) or women (aOR 0.67, 95% CI 0.35-1.29), or for either patients with HIV infection (aOR 1.51, 95% CI 0.81-2.85) or without HIV infection (aOR 0.78, 95% CI 0.46-1.32). In the PP analysis, the 99DOTS-based intervention increased the odds of treatment success (aOR 2.89, 95% CI 1.57-5.33, p = 0.001). The odds of completing the intensive phase of treatment and the odds of not being lost to follow-up were similarly improved in PP but not ITT analyses. Study limitations include the likelihood of selection bias in the PP analysis, inability to verify medication dosing in either arm, and incomplete implementation of some components of the intervention.Conclusions99DOTS-based treatment supervision did not improve treatment outcomes in the overall study population. However, similar treatment outcomes were achieved during the control and intervention periods, and those patients enrolled on 99DOTS achieved high treatment completion. 99DOTS-based treatment supervision could be a viable alternative to directly observed therapy for a substantial proportion of patients with TB.Trial registrationPan-African Clinical Trials Registry (PACTR201808609844917).

Project description:BackgroundStepped-wedge cluster randomized trials (SWCRTs) are a type of cluster-randomized trial in which clusters are randomized to cross-over to the active intervention sequentially at regular intervals during the study period. For SWCRTs, sequential imbalances of cluster-level characteristics across the random sequence of clusters may lead to biased estimation. Our study aims to examine the effects of balancing cluster-level characteristics in SWCRTs.MethodsTo quantify the level of cluster-level imbalance, a novel imbalance index was developed based on the Spearman correlation and rank regression of the cluster-level characteristic with the cross-over timepoints. A simulation study was conducted to assess the impact of sequential cluster-level imbalances across different scenarios varying the: number of sites (clusters), sample size, number of cross-over timepoints, site-level intra-cluster correlation coefficient (ICC), and effect sizes. SWCRTs assumed either an immediate "constant" treatment effect, or a gradual "learning" treatment effect which increases over time after crossing over to the active intervention. Key performance metrics included the relative root mean square error (RRMSE) and relative mean bias.ResultsFully-balanced designs almost always had the highest efficiency, as measured by the RRMSE, regardless of the number of sites, ICC, effect size, or sample sizes at each time for SWCRTs with learning effect. A consistent decreasing trend of efficiency was observed by increasing RRMSE as imbalance increased. For example, for a 12-site study with 20 participants per site/timepoint and ICC of 0.10, between the most balanced and least balanced designs, the RRMSE efficiency loss ranged from 52.5% to 191.9%. In addition, the RRMSE was decreased for larger sample sizes, larger number of sites, smaller ICC, and larger effect sizes. The impact of pre-balancing diminished when there was no learning effect.ConclusionThe impact of pre-balancing on preventing efficiency loss was easily observed when there was a learning effect. This suggests benefit of pre-balancing with respect to impacting factors of treatment effects.

Project description:Background Tuberculosis (TB) is among the leading causes of infectious death worldwide. Contact investigation is an evidence-based, World Health Organisation-endorsed intervention for timely TB diagnosis, treatment, and prevention but has not been widely and effectively implemented. Methods We are conducting a stepped-wedge, cluster-randomised, hybrid Type III implementation-effectiveness trial comparing a user-centred to a standard strategy for implementing TB contact investigation in 12 healthcare facilities in Uganda. The user-centred strategy consists of several client-focused components including 1) a TB-education booklet, 2) a contact-identification algorithm, 3) an instructional sputum-collection video, and 4) a community-health-rider service to transport clients, CHWs, and sputum samples, along with several healthcare-worker-focused components, including 1) collaborative improvement meetings, 2) regular audit-and-feedback reports, and 3) a digital group-chat application designed to develop a community of practice. Sites will cross from the standard to the user-centred strategy in six, eight-week transition steps following a randomly determined site-pairing scheme and timeline. The primary implementation outcome is the proportion of symptomatic close contacts completing TB evaluation within 60 days of TB treatment initiation by the index person with TB. The primary clinical effectiveness outcomes are the proportion of contacts diagnosed with and initiating active TB disease treatment and the proportion initiating TB preventative therapy within 60 days. We will assess outcomes from routine source documents using intention-to-treat analyses. We will also conduct nested mixed-methods studies of implementation fidelity and context and perform cost-effectiveness and impact modelling. The Makerere School of Public Health IRB (#554), the Uganda National Council for Science and Technology (#HS1720ES), and the Yale Institutional Review Board (#2000023199) approved the study with a waiver of informed consent for the main trial implementation-effectiveness outcomes. We will submit trial results for publication in a peer-reviewed journal and disseminate findings to local shareholders, including policymakers and representatives of affected communities. Discussion This pragmatic, quasi-experimental implementation trial will inform efforts to find and prevent undiagnosed persons with TB in high-burden setting using contact investigation. It will help assess the suitability of human-centred design and communities of practice for tailoring implementation strategies and sustain evidence-based interventions in low-and-middle-income countries. Trial registration number ClinicalTrials.gov Identifier: NCT05640648.

Project description:BackgroundTuberculosis(TB) is among the leading causes of infectious death worldwide. Contact investigation is an evidence-based, World Health Organisation-endorsed intervention for timely TB diagnosis, treatment, and prevention but has not been widely and effectively implemented.MethodsWe are conducting a stepped-wedge, cluster-randomised, hybrid Type III implementation-effectiveness trial comparing a user-centred to a standard strategy for implementing TB contact investigation in 12 healthcare facilities in Uganda. The user-centred strategy consists of several client-focused components including (1) a TB-education booklet, (2) a contact-identification algorithm, (3) an instructional sputum-collection video, and (4) a community-health-rider service to transport clients, CHWs, and sputum samples, along with several healthcare-worker-focused components, including (1) collaborative improvement meetings, (2) regular audit-and-feedback reports, and (3) a digital group-chat application designed to develop a community of practice. Sites will cross-over from the standard to the user-centred strategy in six, eight-week transition steps following a randomly determined site-pairing scheme and timeline. The primary implementation outcome is the proportion of symptomatic close contacts completing TB evaluation within 60 days of TB treatment initiation by the index person with TB. The primary clinical effectiveness outcomes are the proportion of contacts diagnosed with and initiating active TB disease treatment and the proportion initiating TB preventative therapy within 60 days. We will assess outcomes from routine source documents using intention-to-treat analyses. We will also conduct nested mixed-methods studies of implementation fidelity and context and perform cost-effectiveness and impact modelling. The Makerere School of Public Health IRB(#554), the Uganda National Council for Science and Technology(#HS1720ES), and the Yale Institutional Review Board(#2000023199) approved the study and waived informed consent for the main trial implementation-effectiveness outcomes. We will submit results for publication in peer-reviewed journals and disseminate findings to local policymakers and representatives of affected communities.DiscussionThis pragmatic, quasi-experimental implementation trial will inform efforts to find and prevent undiagnosed persons with TB in high-burden settings using contact investigation. It will also help assess the suitability of human-centred design and communities of practice for tailoring implementation strategies and sustaining evidence-based interventions in low-and-middle-income countries.Trial registrationThe trial was registered(ClinicalTrials.gov Identifier NCT05640648) on 16 November 2022, after the trial launch on 7 March 2022.

Project description:BackgroundOperative vaginal birth is a common procedure used to expedite birth after full cervical dilatation where there is a clinical need to do so (15% of births in the UK in 2016). The acquisition of skills for operative vaginal birth is dependent on the exposure of junior obstetricians to situations in which they can undertake directly supervised learning from senior accouchers. The Royal College of Obstetricians and Gynaecologists has recently introduced the first structured course in operative vaginal birth. To date, there have been no attempts to determine the clinical impact of a structured training package for operative vaginal birth.MethodsThe STROBE study is a quasi-experimental before-after interrupted time-series study of the effect of simulation training in operative vaginal birth for obstetricians on clinical outcomes of women and babies following operative vaginal birth. Similar to a stepped-wedge design, the intervention will be gradually implemented in all participating units but at different time periods. The primary outcome is failed operative vaginal birth with the first intended instrument. Secondary maternal outcomes are; use of second instrument to achieve operative vaginal birth, caesarean section, episiotomy, perineal trauma (1st, 2nd, 3rd, 4th degree tear), cervical tear requiring suturing, general anaesthesia and estimated blood loss. Secondary neonatal outcomes are; Apgar score at one, five, and ten minutes, Umbilical artery pH, shoulder dystocia, admission to Neonatal Intensive Care Unit and death within 28 days of birth. The analysis will be intention-to-treat (per unit) on the primary and secondary outcomes. The STROBE study received approval from the Health Research Authority and is sponsored by North Bristol NHS Trust. Results will be published in an open-access peer-reviewed medical journal within one year of completion of data gathering.DiscussionThe STROBE study will help establish our understanding of the effectiveness of locally-delivered simulation training for operative vaginal birth. Robust evidence supporting the effectiveness of such an approach would add weight to the argument supporting regular, local training for junior obstetricians in operative vaginal birth.Trial registrationISRCTN11760611 05/03/2018 (retrospectively registered).

Project description:BackgroundStepped wedge randomised trial designs involve sequential roll-out of an intervention to participants (individuals or clusters) over a number of time periods. By the end of the study, all participants will have received the intervention, although the order in which participants receive the intervention is determined at random. The design is particularly relevant where it is predicted that the intervention will do more good than harm (making a parallel design, in which certain participants do not receive the intervention unethical) and/or where, for logistical, practical or financial reasons, it is impossible to deliver the intervention simultaneously to all participants. Stepped wedge designs offer a number of opportunities for data analysis, particularly for modelling the effect of time on the effectiveness of an intervention. This paper presents a review of 12 studies (or protocols) that use (or plan to use) a stepped wedge design. One aim of the review is to highlight the potential for the stepped wedge design, given its infrequent use to date.MethodsComprehensive literature review of studies or protocols using a stepped wedge design. Data were extracted from the studies in three categories for subsequent consideration: study information (epidemiology, intervention, number of participants), reasons for using a stepped wedge design and methods of data analysis.ResultsThe 12 studies included in this review describe evaluations of a wide range of interventions, across different diseases in different settings. However the stepped wedge design appears to have found a niche for evaluating interventions in developing countries, specifically those concerned with HIV. There were few consistent motivations for employing a stepped wedge design or methods of data analysis across studies. The methodological descriptions of stepped wedge studies, including methods of randomisation, sample size calculations and methods of analysis, are not always complete.ConclusionWhile the stepped wedge design offers a number of opportunities for use in future evaluations, a more consistent approach to reporting and data analysis is required.