A randomized comparison of video demonstration versus hands-on training of medical students for vacuum delivery using Objective Structured Assessment of Technical Skills (OSATS).
Ontology highlight
ABSTRACT:
SUBMITTER: Hilal Z
PROVIDER: S-EPMC5369926 | biostudies-literature | 2017 Mar
REPOSITORIES: biostudies-literature
Publications
A randomized comparison of video demonstration versus hands-on training of medical students for vacuum delivery using Objective Structured Assessment of Technical Skills (OSATS).
Hilal Ziad Z Kumpernatz Anne K AK Rezniczek Günther A GA Cetin Cem C Tempfer-Bentz Eva-Katrin EK Tempfer Clemens B CB
Medicine 20170301 11
<h4>Background</h4>To compare medical students' skills for vaginal operative delivery by vacuum extraction (VE) after hands-on training versus video demonstration.<h4>Methods</h4>We randomized medical students to an expert demonstration (group 1) or a hands-on (group 2) training using a standardized VE algorithm on a pelvic training model. Students were tested with a 40-item Objective Structured Assessment of Technical Skills (OSATS) scoring system after training and 4 days later. OSATS scores w ...[more]
Similar Datasets
Project description:Performance assessment in skills training is ideally based on objective, reliable, and clinically relevant indicators of success. The Objective Structured Assessment of Technical Skill (OSATS) is a reliable and valid tool that has been increasingly used in orthopaedic skills training. It uses a global rating approach to structure expert evaluation of technical skills with the experts working from a list of operative competencies that are each rated on a 5-point Likert scale anchored by behavioral descriptors. Given the observational nature of its scoring, the OSATS might not effectively assess the quality of surgical results.(1) Does OSATS scoring in an intraarticular fracture reduction training exercise correlate with the quality of the reduction? (2) Does OSATS scoring in a cadaveric extraarticular fracture fixation exercise correlate with the mechanical integrity of the fixation?Orthopaedic residents at the University of Iowa (six postgraduate year [PGY]-1s) and at the University of Minnesota (seven PGY-1s and eight PGY-2s) undertook a skills training exercise that involved reducing a simulated intraarticular fracture under fluoroscopic guidance. Iowa residents participated three times during 1 month, and Minnesota residents participated twice with 1 month between their two sessions. A fellowship-trained orthopaedic traumatologist rated each performance using a modified OSATS scoring scheme. The quality of the articular reduction obtained was then directly measured. Regression analysis was performed between OSATS scores and two metrics of articular reduction quality: articular surface deviation and estimated contact stress. Another skills training exercise involved fixing a simulated distal radius fracture in a cadaveric specimen. Thirty residents, distributed across four PGY classes (PGY-2 and PGY-3, n = 8 each; PGY-4 and PGY-5, n = 7 each), simultaneously completed the exercise at individual stations. One of three faculty hand surgeons independently scored each performance using a validated OSATS scoring system. The mechanical integrity of each fixation construct was then assessed in a materials testing machine. Regression analysis was performed between OSATS scores and two metrics of fixation integrity: stiffness and failure load.In the intraarticular fracture model, OSATS scores did not correlate with articular reduction quality (maximum surface deviations: R = 0.17, p = 0.25; maximum contact stress: R = 0.22, p = 0.13). Similarly in the cadaveric extraarticular fracture model, OSATS scores did not correlate with the integrity of the mechanical fixation (stiffness: R = 0.10, p = 0.60; failure load: R = 0.30, p = 0.10).OSATS scoring methods do not effectively assess the quality of the surgical result. Efforts must be made to incorporate assessment metrics that reflect the quality of the surgical result.New objective, reliable, and clinically relevant measures of the quality of the surgical result obtained by a trainee are urgently needed. For intraarticular fracture reduction and extraarticular fracture fixation, direct physical measurement of reduction quality and of mechanical integrity of fixation, respectively, meet this need.
| S-EPMC4773332 | biostudies-other
Project description:ObjectiveAssessment of the quality of the final product (QFP) is critical in simulation training, such as the clock face suture (CFS) exercise that is used to assess trainees' needle handling and suturing accuracy. Objective Structured Assessment of Technical Skill (OSATS) scores are the gold standard for the evaluation of trainees. The aim was to investigate variability in the use of OSATS checklists and to evaluate a semi-automatic method of suture analysis vs. OSATS scores.MethodsDetails of 287 CFSs performed by trainees during Fundamentals in Vascular Surgery examinations were collected. All were rated according to a seven item OSATS checklist, including QFP score and an overall score by one or two expert surgeons immediately after completion. Interassessor variability was assessed for the CFS that were assessed by two assessors.In order to assess intra- and interassessor variability, 50 CFS pictures were chosen randomly and submitted to three expert surgeons to rate the QFP twice and to carry out a semi-automatic image analysis of each CFS and the estimated cumulative error (CE; mm) recorded. It was hypothesised that the CE correlates to OSATS checklist items or overall score. Variables were compared for correlation with OSATS results using a linear regression. A Pearson's test was used to confirm the proposed hypothesis.ResultsMean ± standard deviation overall score for the OSATS checklist was 20.61 ± 6.33. Inter- and intra-assessor correlation were statistically significant regarding OSATS checklist items. Both correlations presented a low coefficient of determination, indicating variability. The mean CE was 16.07 ± 4.84 mm, and the correlation between the QFP and CE was statistically significant, proving that CE is an objective metric by which to assess the QFP.ConclusionOSATS score demonstrated intra- and interassessor variability, although there was a significant correlation between scores. CE is an objective metric that is not subject to assessor subjectivity or interassessor variability and is correlated with the gold standard of evaluation.
| S-EPMC9664359 | biostudies-literature
Project description:IntroductionThe aim of this study was to develop a reliable objective structured assessment of technical skills (OSATS) score for linear-stapled, hand-sewn closure of enterotomy intestinal anastomoses (A-OSATS).Materials and methodsThe Delphi methodology was used to create a traditional and weighted A-OSATS score highlighting the more important steps for patient outcomes according to an international expert consensus. Minimally invasive novices, intermediates, and experts were asked to perform a minimally invasive linear-stapled intestinal anastomosis with hand-sewn closure of the enterotomy in a live animal model either laparoscopically or robot-assisted. Video recordings were scored by two blinded raters assessing intrarater and interrater reliability and discriminative abilities between novices (n = 8), intermediates (n = 24), and experts (n = 8).ResultsThe Delphi process included 18 international experts and was successfully completed after 4 rounds. A total of 4 relevant main steps as well as 15 substeps were identified and a definition of each substep was provided. A maximum of 75 points could be reached in the unweighted A-OSATS score and 170 points in the weighted A-OSATS score respectively. A total of 41 anastomoses were evaluated. Excellent intrarater (r = 0.807-0.988, p < 0.001) and interrater (intraclass correlation coefficient = 0.923-0.924, p < 0.001) reliability was demonstrated. Both versions of the A-OSATS correlated well with the general OSATS and discriminated between novices, intermediates, and experts defined by their OSATS global rating scale.ConclusionWith the weighted and unweighted A-OSATS score, we propose a new reliable standard to assess the creation of minimally invasive linear-stapled, hand-sewn anastomoses based on an international expert consensus. Validity evidence in live animal models is provided in this study. Future research should focus on assessing whether the weighted A-OSATS exceeds the predictive capabilities of patient outcomes of the unweighted A-OSATS and provide further validity evidence on using the score on different anastomotic techniques in humans.
| S-EPMC9085690 | biostudies-literature
Project description:Background: Chest tube insertion is a standard intervention for management of various injuries of the thorax. Efficient clinical training of this and similar bed-side procedures is equally demanded and improvable. Here, we propose a nouveau means of assessment and feedback using an Objective Structured Assessment of Technical Skills (OSATS) tool. The modified OSATS for chest drain insertion is evaluated in a pilot trial focusing on chest drain insertion. Methods: Participants in the pilot trial were medical students (3rd-6th year of studies, n=9), junior residents (1st-3rd post-graduate year, n=12), senior residents (4th-6th post-graduate year, n=14), and attending surgeons (n=6) from Heidelberg University. Chest drain insertions on a cadaveric porcine model were rated by experts with the modified OSATS score. Participants' performances were videotaped and subsequently rated by two remote experts (video rating). Primary aim was to assess criterion validity of the OSATS to distinguish experience levels. Results: Kruskal-Wallis test showed significant differences between means of scores between four groups stratified by previous experience in chest tube insertion (level 0: 22.1±3.2 vs. level 1: 26.8±2.8 vs. level 2: 35.4±2.2 vs. level 3: 41.0±2.0; p=0.002; p1,3=0.049, p0,3=0.005). However, if groups were stratified by formal professional level, no statistically significant distinction could be made using OSATS. Hence, the OSATS tool showed criterion validity for differentiation between experience levels. Conclusion: In the pilot study, the modified OSATS for chest tube insertion was apt to standardize expert rating and could be used to measure skill and to depict different experience levels. The OSATS will help facilitate training and assessment of chest drain insertion and could therefore improve surgical training for trauma situations. According to our data, the OSATS might be integrated into modern curricula.
| S-EPMC6278238 | biostudies-other
Project description:PurposeRecently, some attempts have been made to integrate lung ultrasound (LUS) teaching into medical curricula. However, current education studies of LUS are extremely heterogeneous due to the lack of evidence-based guidelines on LUS education. In particular, the assessment of competencies is poorly standardized and mostly relies on non-validated scales. A new validated tool, the objective structured assessment of lung ultrasound skills (LUS-OSAUS), has the potential to overcome these limitations. Therefore, we adopted the LUS-OSAUS tool to assess the competencies of a group of LUS-trained undergraduates. Existing no prior practical applications of the LUS-OSAUS, our aim was to investigate the practical utility of this tool and its applicability in the evaluation of US-trained medical students.MethodsEight undergraduates (two males, six females) were enrolled on a voluntary basis to receive a theoretical and practical training in LUS. Once completed their training, each student performed an LUS examination on a different patient hospitalized for respiratory symptoms. The same eight patients were also scanned by a senior resident in emergency medicine for a comparison with students' results. Students and the senior resident were tested by an examiner using the LUS-OSAUS tool. We compared the scores obtained by operators in all areas of competence of the LUS-OSAUS, the total scores, and the time needed to complete the sonographic task.ResultsMedian students' score in the single items of the scale was significantly lower than the ones obtained by the senior resident (4.0 [3.3-5.0] vs. 5.0 [5.0-5.0]; p < 0.0001). Students scored significantly lower than the senior resident in each item, except for B-line identification, choice of the correct transducer, and suggested focused questions. Median total score was also lower for students compared to the senior resident (70.5 [61.0-74.8] vs. 84.0 [83.5-84.3] (p = 0.0116). Median time required to complete the examination was significantly higher for students (14.1 [12.8-16.1] vs. 4.7 [3.9-5.2] min, p = 0.0117).ConclusionsThe LUS-OSAUS tool allowed for a standardized and comprehensive assessment of student's competencies in lung ultrasound, and helped to discriminate their level of expertise from that of a more experienced operator. The scale also specifically tests the theoretical knowledge of trainees, thus making redundant the use of questionnaires designed for this purpose.
| S-EPMC7223719 | biostudies-literature
Project description:BackgroundPandemic-induced restrictions forced curriculum transformation from on-site education to virtual learning options. This report describes this transition, the challenge of creating technology-enhanced learning for hands-on psychomotor skills teaching in physiotherapy, and students' evaluations of the new technology-enhanced learning approach in Complex Decongestive Physiotherapy.MethodsOn-site theoretical background lectures were replaced with e-learning sessions. Faculty hands-on skills demonstrations for the entire class were replaced with video-recorded demonstrations. Videos included verbal and written instructions and were complemented with checklists guiding the students, training in pairs, through their learning tasks. A cross-sectional observational survey for teaching quality evaluated this new technology-enhanced learning approach and assessed students' preference for traditional or video-based hands-on skills learning.ResultsSurvey return rate was > 50% (46 participating students). Teaching quality was rated between 1.5 ± 0.5 and 1.8 ± 0.4 (Likert scale from - 2 to + 2). Most students (66.7%) preferred the new approach. They appreciated for example that videos were available all the time, enabling self-paced learning, providing an equally good view on skills demonstrations, and the convenience to be able to rewind, re-view, and use speed adjustment options.ConclusionsStudents preferred the new video-based learning of skills for Complex Decongestive Physiotherapy. Because in-class live skills demonstrations were omitted, faculty had more time to provide individual feedback and answer questions. The shift from teacher- to student-centered learning enabled students to control their own learning pace. The innovative program was maintained after pandemic-induced restrictions were lifted. The success of this approach should be tested in other physiotherapy settings and different educational institutions.
| S-EPMC10474678 | biostudies-literature
Project description:BackgroundTeaching and assessment of communication skills have become essential in medical education. The Objective Structured Clinical Examination (OSCE) has been found as an appropriate means to assess communication skills within medical education. Studies have demonstrated the importance of a valid assessment of medical students' communication skills. Yet, the validity of the performance scores depends fundamentally on the quality of the rating scales used in an OSCE. Thus, this systematic review aimed at providing an overview of existing rating scales, describing their underlying definition of communication skills, determining the methodological quality of psychometric studies and the quality of psychometric properties of the identified rating scales.MethodsWe conducted a systematic review to identify psychometrically tested rating scales, which have been applied in OSCE settings to assess communication skills of medical students. Our search strategy comprised three databases (EMBASE, PsycINFO, and PubMed), reference tracking and consultation of experts. We included studies that reported psychometric properties of communication skills assessment rating scales used in OSCEs by examiners only. The methodological quality of included studies was assessed using the COnsensus based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. The quality of psychometric properties was evaluated using the quality criteria of Terwee and colleagues.ResultsData of twelve studies reporting on eight rating scales on communication skills assessment in OSCEs were included. Five of eight rating scales were explicitly developed based on a specific definition of communication skills. The methodological quality of studies was mainly poor. The psychometric quality of the eight rating scales was mainly intermediate.DiscussionOur results reveal that future psychometric evaluation studies focusing on improving the methodological quality are needed in order to yield psychometrically sound results of the OSCEs assessing communication skills. This is especially important given that most OSCE rating scales are used for summative assessment, and thus have an impact on medical students' academic success.
| S-EPMC4816391 | biostudies-literature
Project description:ObjectiveTo investigate the influence of providing online procedural videos to postgraduate chiropractic students preparing for an objective structured clinical examination (OSCE).MethodsEighty-three postgraduate chiropractic students enrolled in a diagnostic unit during 2017 received supplemental video resources prior to their final OSCE. Ninety students enrolled in the 2016 offering of the unit acted as the control group. Two-sample t tests were used to compare OSCE results between groups and paired t tests were used for within-group comparisons. Regression analysis was used to examine the association of age, undergraduate grade point average, and gender with the final OSCE scores. Students were also surveyed regarding their perceptions of the video resources using a purpose-built questionnaire.ResultsA paired t test comparing initial and final OSCE scores found a small but significant increase in scores for the 2017 (mean change 3.6 points; p = .001) but not the 2016 (mean change -1.1 scores; p = .09) cohort. The 2017 cohort had significantly more change than the 2016 cohort (mean difference 4.7 points; p < .001). Analysis of responses to the questionnaire highlighted overall positive feedback for the procedural videos.ConclusionOnline procedural videos as learning resources had a small but positive effect on OSCE performance for a group of postgraduate chiropractic students. Students perceived the resource as being helpful for OSCE preparation.
| S-EPMC7682643 | biostudies-literature
Project description:Undergraduate research can make a positive impact on science education. Unfortunately, the one student-one mentor paradigm of undergraduate research generates a wide range of variability in the student's experience and further limits its availability to a select few students. In contrast, a single faculty member can offer multiple undergraduate teaching positions that provide a consistent experience for the student. We attempted to combine the undergraduate research and teaching experiences in an internship practicum called Peer Instruction and Laboratory Occupational Training (PILOT). Students enrolled in PILOT served as teaching assistants for the upper division Quantitative Biological Methods (QBM) laboratory course. In addition, PILOT students worked on an independent lab project that provided them with hands-on training and supported the QBM course. The development of presentation and teaching skills was also emphasized in PILOT. These activities were designed to improve student communication skills, lab skills, and knowledge of molecular biology content. Here, we describe the PILOT curriculum and report the results of an anonymous assessment survey administered to 75 students who had completed PILOT in the previous five semesters. Our data indicate that PILOT provides an effective format to expand undergraduate opportunities for research and teaching experiences.
| S-EPMC3706164 | biostudies-other
Project description:Background Medical trainees must learn how to provide effective feedback as an essential communication skill, yet few models exist for training and assessing these skills. Objective To develop an observed structured feedback examination (OSFE) to provide feedback training to pediatric fellows and assess changes in skills and self-reported confidence. Methods This educational study was conducted from 2019 to 2020 at an academic children's hospital. Our team developed the OSFE and trained standardized feedback recipients and faculty. Fellows completed baseline self-assessments (31 items) on prior exposure to feedback training, application of skills, and confidence. They then participated in the OSFE, giving feedback to a standardized recipient using a standardized scenario, and were scored by faculty and recipients using a 15-item checklist for performance. Next, fellows participated in feedback training and received individualized feedback, after which they repeated the OSFE and confidence self-assessment. Three months later, fellows completed self-assessments on confidence and application of skills and another OSFE to assess retention. Descriptive statistics and signed rank sum test were used for analysis. Results Of 60 eligible fellows, 19 participated (32%), with 100% follow-up. After training and individualized feedback, all fellows improved feedback skills as measured by OSFE performance (mean change +0.89). All items, measured on a 5-point Likert scale, were sustained 3 months later (mean change +0.92). All fellows reported improved confidence in feedback knowledge (mean change +2.07 post, +1.67 3 months post). Conclusions Feedback training using simulation and individualized feedback moderately improved fellows' performance, confidence, and 3-month retention of feedback skills.
| S-EPMC10829923 | biostudies-literature