Dataset Information

Evaluating the Instructional Design and Effect on Knowledge, Teamwork, and Skills of Technology-Enhanced Simulation-Based Training in Obstetrics in Uganda: Stepped-Wedge Cluster Randomized Trial.

ABSTRACT:

Background

Simulation-based training is a common strategy for improving the quality of facility-based maternity services and is often evaluated using Kirkpatrick's theoretical model. The results on the Kirkpatrick levels are closely related to the quality of the instructional design of a training program. The instructional design is generally defined as the "set of prescriptions for teaching methods to improve the quality of instruction with a goal of optimizing learning outcomes."

Objective

The aim of this study is to evaluate the instructional design of a technology-enhanced simulation-based training in obstetrics, the reaction of participants, and the effect on knowledge, teamwork, and skills in a low-income country.

Methods

A stepped-wedge cluster randomized trial was performed in a university hospital in Kampala, Uganda, with an annual delivery volume of over 31,000. In November 2014, a medical simulation center was installed with a full-body birthing simulator (Noelle S550, Gaumard Scientific), an interactive neonate (Simon S102 Newborn CPR Simulator, Gaumard Scientific), and an audio and video recording system. Twelve local obstetricians were trained and certified as medical simulation trainers. From 2014 to 2016, training was provided to 57 residents in groups of 6 to 9 students. Descriptive statistics were calculated for ten instructional design features of the training course measured by the 42-item ID-SIM (Instructional Design of a Simulation Improved by Monitoring). The Wilcoxon signed rank test was conducted to investigate the differences in scores on knowledge, the Clinical Teamwork Scale, and medical technical skills.

Results

The mean scores on the ten instructional design features ranged from 54.9 (95% CI 48.5-61.3) to 84.3 (95% CI 80.9-87.6) out of 100. The highest mean score was given on the feature feedback and the lowest scores on repetitive practice and controlled environment. The overall score for the training day was 92.8 out of 100 (95% CI 89.5-96.1). Knowledge improved significantly, with a test score of 63.4% (95% CI 60.7-66.1) before and 78.9% (95% CI 76.8-81.1) after the training (P<.001). The overall score on the 10-point Clinical Teamwork Scale was 6.0 (95% CI 4.4-7.6) before and 5.9 (95% CI 4.5-7.2) after the training (P=.78). Medical technical skills were scored at 55.5% (95% CI 47.2-63.8) before and 65.6% (95% CI 56.5-74.7) after training (P=.08).

Conclusions

Most instructional design features of a technology-enhanced simulation-based training in obstetrics in a low-income country were scored high, although intervals were large. The overall score for the training day was high, and knowledge did improve after the training program, but no changes in teamwork and (most) medical technical skills were found. The lowest-scored instructional design features may be improved to achieve further learning aims.

Trial registration

ISRCTN Registry ISRCTN98617255; http://www.isrctn.com/ISRCTN98617255.

International registered report identifier (irrid)

RR2-10.1186/s12884-020-03050-3.

SUBMITTER: van Tetering AAC

PROVIDER: S-EPMC8081249 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundAdherence to and completion of tuberculosis (TB) treatment remain problematic in many high-burden countries. 99DOTS is a low-cost digital adherence technology that could increase TB treatment completion.Methods and findingsWe conducted a pragmatic stepped-wedge cluster-randomized trial including all adults treated for drug-susceptible pulmonary TB at 18 health facilities across Uganda over 8 months (1 December 2018-31 July 2019). Facilities were randomized to switch from routine (control period) to 99DOTS-based (intervention period) TB treatment supervision in consecutive months. Patients were allocated to the control or intervention period based on which facility they attended and their treatment start date. Health facility staff and patients were not blinded to the intervention. The primary outcome was TB treatment completion. Due to the pragmatic nature of the trial, the primary analysis was done according to intention-to-treat (ITT) and per protocol (PP) principles. This trial is registered with the Pan African Clinical Trials Registry (PACTR201808609844917). Of 1,913 eligible patients at the 18 health facilities (1,022 and 891 during the control and intervention periods, respectively), 38.0% were women, mean (SD) age was 39.4 (14.4) years, 46.8% were HIV-infected, and most (91.4%) had newly diagnosed TB. In total, 463 (52.0%) patients were enrolled on 99DOTS during the intervention period. In the ITT analysis, the odds of treatment success were similar in the intervention and control periods (adjusted odds ratio [aOR] 1.04, 95% CI 0.68-1.58, p = 0.87). The odds of treatment success did not increase in the intervention period for either men (aOR 1.24, 95% CI 0.73-2.10) or women (aOR 0.67, 95% CI 0.35-1.29), or for either patients with HIV infection (aOR 1.51, 95% CI 0.81-2.85) or without HIV infection (aOR 0.78, 95% CI 0.46-1.32). In the PP analysis, the 99DOTS-based intervention increased the odds of treatment success (aOR 2.89, 95% CI 1.57-5.33, p = 0.001). The odds of completing the intensive phase of treatment and the odds of not being lost to follow-up were similarly improved in PP but not ITT analyses. Study limitations include the likelihood of selection bias in the PP analysis, inability to verify medication dosing in either arm, and incomplete implementation of some components of the intervention.Conclusions99DOTS-based treatment supervision did not improve treatment outcomes in the overall study population. However, similar treatment outcomes were achieved during the control and intervention periods, and those patients enrolled on 99DOTS achieved high treatment completion. 99DOTS-based treatment supervision could be a viable alternative to directly observed therapy for a substantial proportion of patients with TB.Trial registrationPan-African Clinical Trials Registry (PACTR201808609844917).

Project description:BackgroundStepped-wedge cluster randomized trials (SWCRTs) are a type of cluster-randomized trial in which clusters are randomized to cross-over to the active intervention sequentially at regular intervals during the study period. For SWCRTs, sequential imbalances of cluster-level characteristics across the random sequence of clusters may lead to biased estimation. Our study aims to examine the effects of balancing cluster-level characteristics in SWCRTs.MethodsTo quantify the level of cluster-level imbalance, a novel imbalance index was developed based on the Spearman correlation and rank regression of the cluster-level characteristic with the cross-over timepoints. A simulation study was conducted to assess the impact of sequential cluster-level imbalances across different scenarios varying the: number of sites (clusters), sample size, number of cross-over timepoints, site-level intra-cluster correlation coefficient (ICC), and effect sizes. SWCRTs assumed either an immediate "constant" treatment effect, or a gradual "learning" treatment effect which increases over time after crossing over to the active intervention. Key performance metrics included the relative root mean square error (RRMSE) and relative mean bias.ResultsFully-balanced designs almost always had the highest efficiency, as measured by the RRMSE, regardless of the number of sites, ICC, effect size, or sample sizes at each time for SWCRTs with learning effect. A consistent decreasing trend of efficiency was observed by increasing RRMSE as imbalance increased. For example, for a 12-site study with 20 participants per site/timepoint and ICC of 0.10, between the most balanced and least balanced designs, the RRMSE efficiency loss ranged from 52.5% to 191.9%. In addition, the RRMSE was decreased for larger sample sizes, larger number of sites, smaller ICC, and larger effect sizes. The impact of pre-balancing diminished when there was no learning effect.ConclusionThe impact of pre-balancing on preventing efficiency loss was easily observed when there was a learning effect. This suggests benefit of pre-balancing with respect to impacting factors of treatment effects.

Project description:BackgroundStepped wedge randomised trial designs involve sequential roll-out of an intervention to participants (individuals or clusters) over a number of time periods. By the end of the study, all participants will have received the intervention, although the order in which participants receive the intervention is determined at random. The design is particularly relevant where it is predicted that the intervention will do more good than harm (making a parallel design, in which certain participants do not receive the intervention unethical) and/or where, for logistical, practical or financial reasons, it is impossible to deliver the intervention simultaneously to all participants. Stepped wedge designs offer a number of opportunities for data analysis, particularly for modelling the effect of time on the effectiveness of an intervention. This paper presents a review of 12 studies (or protocols) that use (or plan to use) a stepped wedge design. One aim of the review is to highlight the potential for the stepped wedge design, given its infrequent use to date.MethodsComprehensive literature review of studies or protocols using a stepped wedge design. Data were extracted from the studies in three categories for subsequent consideration: study information (epidemiology, intervention, number of participants), reasons for using a stepped wedge design and methods of data analysis.ResultsThe 12 studies included in this review describe evaluations of a wide range of interventions, across different diseases in different settings. However the stepped wedge design appears to have found a niche for evaluating interventions in developing countries, specifically those concerned with HIV. There were few consistent motivations for employing a stepped wedge design or methods of data analysis across studies. The methodological descriptions of stepped wedge studies, including methods of randomisation, sample size calculations and methods of analysis, are not always complete.ConclusionWhile the stepped wedge design offers a number of opportunities for use in future evaluations, a more consistent approach to reporting and data analysis is required.

Project description:BACKGROUND:Cardiovascular disease (CVD) is the leading cause of death in the US and incurs high health care costs. While many initiatives promote the implementation of ABCS (aspirin therapy, blood pressure control, cholesterol management, and smoking cessation) measures, most primary care practices (PCPs) lack quality improvement (QI) support and resources to achieve meaningful targets. The Healthy Hearts for Oklahoma (H2O) Study proposes to build a QI infrastructure by (1) constructing a sustainable Oklahoma Primary Healthcare Improvement Collaborative (OPHIC) to support dissemination and implementation (D&I) of QI methods; (2) providing QI support in PCPs to better manage patients at risk for CVD events. Parallel to infrastructure building, H2O aims to conduct a comprehensive evaluation of the QI support D&I in primary care and assess the relationship between QI support uptake and changes in ABCS measures. METHODS:H2O has partnered with public health agencies and communities to build OPHIC and facilitate QI. H2O has 263 small primary care practices across Oklahoma that receive the bundled QI intervention to improve ABCS performance. A stepped-wedge designed is used to evaluate D&I of QI support. Changes in ABCS measures will be estimated as a function of various components of the QI support and capacity and readiness of PCPs to change. Notes from academic detailing and practice facilitation sessions will be analyzed to help interpret findings on ABCS performance. DISCUSSION:H2O program is designed to improve cardiovascular health and outcomes for more than 1.25 million Oklahomans. The infrastructure established as a result of this funding will help reach medically underserved Oklahomans, particularly among rural and tribal populations. Lessons learned from this project will guide future strategies for D&I of evidence-based practices in PCPs. Trained practice facilitators will continue to serve as critical resource to assists small, rural PCPs in adapting to the ever-changing health environment and continue to deliver quality care to their communities.