Dataset Information

Modules for the Technical Skills Section of the OSCE Component of the American Board of Anesthesiology APPLIED Examination.

ABSTRACT:

Introduction

To assess communication and professionalism, as well as technical skills related to patient care, the American Board of Anesthesiology (ABA) has begun administering an Objective Structured Clinical Examination (OSCE) portion of the APPLIED Examination in addition to the Standard Oral Examination component.

Methods

We created video modules and a curriculum for anesthesiology resident OSCE preparation for the Interpretation of Monitors and Interpretation of Echocardiography components. The modules can be used individually by trainees or included as part of an OSCE workshop led by faculty educators with seven individual stations matching the content of the actual ABA examination. These modules are recommended for all levels of anesthesiology trainees so that they can gain exposure to the format and the fast pace of the examination.

Results

Sixty-six junior and senior anesthesiology residents, fellows, and junior faculty successfully participated in these modules. Seventy-three percent of the participants agreed that after completing these modules, they now had a good understanding of the Interpretation of Monitors and Interpretation of Echocardiography technical skills stations. More than 90% of participants reported that the modules were useful, and more than 70% reported that they now felt prepared for these stations of the OSCE.

Discussion

Developing technical skills stations for deliberate practice and preparation for the ABA OSCE is resource intensive. Finding time and faculty to facilitate OSCE preparation is also challenging. With the video modules and scripts included in this publication, residents can practice independently or as part of larger preparation course.

SUBMITTER: Rochlen LR

PROVIDER: S-EPMC6507923 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Background Simulation is known as an important tool for the learning of technical and non-technical skills without endangering patient safety. In Portugal, a National Pedagogical Plan for Anesthesiology Residents was created based on simulation training. This plan was designed according to the objectives set forth by the Portuguese Board of Anesthesiology. This study aimed to evaluate the impact of simulation training courses on the non-technical skills of medical residents in Anesthesiology. Methods Confidential questionnaires, pre- and post-course, were answered by all the residents that attended the different modules of the simulation training program at Centro Hospitalar e Universitário de Coimbra Biomedical Simulation Centre, Portugal, from February 2011 to March 2018. Results A total of 344 questionnaires were answered. In the group of questions regarding the need for help, mistakes, and self-efficacy over time, students recognized an increase over time in the need for support and the self-assessment of the number of mistakes (p < 0.001). Regarding the self-evaluation of safety culture and communication skills, at the end of the residency, almost all the students recognized that they did not feel bad when asking for help or expressing their opinion, even when they disagreed with the consultant anesthesiologist. This was significantly different from the values of the self-assessment at the beginning of residency (p < 0.001). The evolution of preparation, knowledge, and training also showed a positive evolution over the simulation modules (p < 0.001). Finally, the evaluation of the behavioral component in the clinical setting showed a significant positive evolution over time (p < 0.001): in the end, all the students strongly agreed that behavioral competencies are crucial. Conclusions The impact of simulation on anesthesiology non-technical skills during residency is positive and recognized by the students. Moreover, simulation also helps in the recognition of error, enriching the value of self-confidence and the crucial role of behavioral skills. ABBREVIATIONS BSC-CHUC: Biomedical Simulation Centre from Centro Hospitalar e Universitário de Coimbra.

Project description:In response to the COVID-19 pandemic, the American Board of Anesthesiology transitioned from in-person to virtual administration of its APPLIED Examination, assessing more than 3000 candidates for certification purposes remotely in 2021. Four hundred examiners were involved in delivering and scoring Standardized Oral Examinations (SOEs) and Objective Structured Clinical Examinations (OSCEs). More than 80% of candidates started their exams on time and stayed connected throughout the exam without any problems. Only 74 (2.5%) SOE and 45 (1.5%) OSCE candidates required rescheduling due to technical difficulties. Of those who experienced "significant issues", concerns with OSCE technical stations (interpretation of monitors and interpretation of echocardiograms) were reported most frequently (6% of candidates). In contrast, 23% of examiners "sometimes" lost connectivity during their multiple exam sessions, on a continuum from minor inconvenience to inability to continue. 84% of SOE candidates and 89% of OSCE candidates described "smooth" interactions with examiners and standardized patients/standardized clinicians, respectively. However, only 71% of SOE candidates and 75% of OSCE candidates considered themselves to be able to demonstrate their knowledge and skills without obstacles. When compared with their in-person experiences, approximately 40% of SOE examiners considered virtual evaluation to be more difficult than in-person evaluation and believed the remote format negatively affected their development as an examiner. The virtual format was considered to be less secure by 56% and 40% of SOE and OSCE examiners, respectively. The retirement of exam materials used virtually due to concern for compromise had implications for subsequent exam development. The return to in-person exams in 2022 was prompted by multiple factors, especially concerns regarding standardization and security. The technology is not yet perfect, especially for testing in-person communication skills and displaying dynamic exam materials. Nevertheless, the American Board of Anesthesiology's experience demonstrated the feasibility of conducting large-scale, high-stakes oral and performance exams in a virtual format and highlighted the adaptability and dedication of candidates, examiners, and administering board staff.

Project description:BackgroundPhysicians' communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curriculum. The adoption of reliable measurement instruments is thus essential to evaluate such skills.MethodsUsing Exploratory Factor Analysis (EFA), Multi-Group Confirmatory Factor Analysis (MGCFA) and Item Response Theory analysis (IRT) the current retrospective study tested the factorial validity and reliability of a four-item global rating scale developed by Hodges and McIlroy to measure CS among 296 third- and fourth-year medical students at the Faculty of Medicine in Geneva, Switzerland, during OSCEs.ResultsEFA results at each station showed good reliability scores. However, measurement invariance assessments through MGCFA across different stations (i.e., same students undergoing six or three stations) and across different groups of stations (i.e., different students undergoing groups of six or three stations) were not satisfactory, failing to meet the minimum requirements to establish measurement invariance and thus possibly affecting reliable comparisons between students' communication scores across stations. IRT revealed that the four communication items provided overlapping information focusing especially on high levels of the communication spectrum.ConclusionsUsing this four-item set in its current form it may be difficult to adequately differentiate between students who are poor in CS from those who perform better. Future directions in best-practices to assess CS among medical students in the context of OSCE may thus focus on (1) training examiners so to obtain scores that are more coherent across stations; and (2) evaluating items in terms of their ability to cover a wider spectrum of medical students' CS. In this respect, IRT can prove to be very useful for the continuous evaluation of CS measurement instruments in performance-based assessments.

Project description:ObjectivesThis study investigates the effectiveness of a virtual format of an advanced communication skills observed structured clinical examination (OSCE) for senior medical students in comparison to an in-person format. The study also examines the emotional support students experience in the virtual setting. Our analysis was based on quantitative data collected through objective checklists and post-OSCE survey results.MethodsThe virtual OSCE was a revision of an earlier in-person formative advanced communication skills OSCE for fourth-year medical students. Student performances were assessed by self and peers using objective checklists-the modified Master Interview Rating Scale (mMIRS) and Communication Behavior Checklist (CBC). The mMIRS measured interview process such as avoiding jargon and demonstrating empathy. The CBC examined interview content which included tasks specific to the content of the case. The OSCE was followed by a faculty-led debrief and quantitative survey. The virtual OSCE was conducted in 2021, and the results of the checklists and survey were compared with those collected from two earlier in-person OSCEs.ResultsEighty-three students participated in the virtual OSCE. There was no difference in mMIRS scores between the virtual and in-person OSCE. Overall CBC scores were lower in the virtual OSCE compared to in-person (p < 0.05). Sixty-seven out of 83 (80.7%) students completed the post-OSCE survey. There were no differences between the virtual and in-person OSCE in terms of educational value, whether the OSCE would change the way participants talk to patients, and preparedness to have serious conversations with patients. All respondents somewhat or strongly agreed with feeling emotionally supported during the virtual OSCE.ConclusionThe virtual format was a suitable alternative to an in-person, formative, advanced communication skills OSCE for medical students. The virtual OSCE was educationally effective and was met with student satisfaction and a sense of emotional support. Future virtual iterations must ensure adequate instruction on interview content.

Project description:Background The aim of this cross-sectional survey was to apply the Objective Structured Clinical Examination (OSCE) to evaluate the cardiopulmonary resuscitation (CPR) and endotracheal intubation skills of resident dentists for stage assessment in standardized training. Methods A total of 146 third-year resident dentists were recruited and randomly assigned to perform either CPR or endotracheal intubation. Their performance was scored by experienced anesthesiologists with standardized scoring criteria. Participants were also asked to rated their self-assessed competence, willingness, and perceptions on training status using Likert-type scales in a questionnaire. Student’s ttest was applied to compare scores for CPR and endotracheal intubation performed by resident dentists with different characteristics. The results of the questionnaires were analyzed and visualized by the R package ‘Likert’. Significance was set at the P < 0.05 level. Results The mean OSCE score for endotracheal intubation (59.1 ± 12.5) was lower than that of CPR (72.4 ± 8.8). Participants with Master’s degrees scored higher than those with Bachelor’s degrees and Doctor of Philosophy (PhD) degrees in the assessment of CPR and endotracheal intubation. Low scores of self-assessed competence and willingness were observed, especially for endotracheal intubation. Resident dentists showed poor satisfaction on training volume and frequency of CPR and endotracheal intubation. Conclusion Resident dentists showed poor performance on CPR and endotracheal intubation assessed by the OSCE. Relatively low self-assessed competence and willingness were reported in endotracheal intubation. The medical emergency curriculum for resident dentists should be more consistent and standardized to help resident dentists enhance the proficiency of life support skills. Supplementary Information The online version contains supplementary material available at 10.1186/s12909-022-03775-z.