Dataset Information

A Checklist to Help Faculty Assess ACGME Milestones in a Video-Recorded OSCE.

ABSTRACT: BACKGROUND :Faculty members need to assess resident performance using the Accreditation Council for Graduate Medical Education Milestones. OBJECTIVE :In this randomized study we used an objective structured clinical examination (OSCE) around the disclosure of an adverse event to determine whether use of a checklist improved the quality of milestone assessments by faculty. METHODS :In 2013, a total of 20 anesthesiology faculty members from 3 institutions were randomized to 2 groups to assess 5 videos of trainees demonstrating advancing levels of competency on the OSCE. One group used milestones alone, and the other used milestones plus a 13-item checklist with behavioral anchors based on ideal performance. We classified faculty ratings as either correct or incorrect with regard to the competency level demonstrated in each video, and then used logistic regression analysis to assess the effect of checklist use on the odds of correct classification. RESULTS :Thirteen of 20 faculty members rated assessing performance using milestones alone as difficult or very difficult. Checklist use was associated with significantly greater odds of correct classification at entry level (odds ratio [OR] = 9.2, 95% confidence interval [CI] 4.0-21.2) and at junior level (OR = 2.7, 95% CI 1.3-5.7) performance. For performance at other competency levels checklist use did not affect the odds of correct classification. CONCLUSIONS :A majority of anesthesiology faculty members reported difficulty with assessing a videotaped OSCE of error disclosure using milestones as primary assessment tools. Use of the checklist assisted in correct assessments at the entry and junior levels.

SUBMITTER: Easdown LJ

PROVIDER: S-EPMC5646919 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Background Obesity is a major public health problem, yet residents undergo little formal training and assessment in obesity-related care. Given the recent growth of telehealth, physicians must further learn to apply these skills using a virtual platform. Therefore, we aimed to develop an objective structured clinical examination (OSCE) with reliable checklists to assess resident ability to take a patient-centered obesity-focused history that was feasible over telehealth based on published obesity competencies for medical education. Methods We developed a 15-minute telehealth OSCE to simulate an obesity-related encounter for residents modified from a script used to assess medical student obesity competencies. We designed three checklists to assess resident skills in history taking, communication and professionalism during the obesity-related encounter. Resident performance was assessed as the percentage of obesity-related history taking questions asked during the encounter and as the mean communication and professionalism scores on a scale of 1 through 5 with 1 representing unacceptable/offensive behavior and 5 representing excellent skills. Encounters and assessments were completed by two commissioned actors (standardized patients) and 26 internal medicine residents over a secure online platform. We assessed the reliability of each checklist by calculating the percent agreement between standardized patients and the kappa (κ) statistic on each checklist overall and by each checklist item. Results Overall agreement between standardized patients on the history taking, communication and professionalism checklists were 83.2% (κ = 0.63), 99.5% (κ = 0.72) and 97.8% (κ =0.44), respectively. On average, residents asked 64.8% of questions on the history taking checklist and scored 3.8 and 3.9 out of 5 on the communication and professionalism checklists, respectively. Conclusions Results from this pilot study suggest that our telehealth obesity OSCE and checklists are moderately reliable for assessing key obesity competencies among residents on a virtual platform. Integrating obesity OSCEs and other educational interventions into residency curricula are needed to improve resident ability to take an obesity-focused history. Supplementary Information The online version contains supplementary material available at 10.1186/s12909-022-03672-5.

Project description:ObjectiveThis study aimed to compare two methods of feedback: verbal face-to-face feedback after direct observation (F2F-feedback) versus electronic-written feedback after observation of recorded-VDO of student's performance (VDO-feedback), in terms of effectiveness in improving skill, effects on motivation and satisfaction.BackgroundMedical schools are responsible for teaching and ensuring proficiency of basic surgical skills. Feedback is effective in developing psychomotor skills; by providing information of learner's current performance, how to improve, and enhancing motivation.Materials and methodFifty-eight medical students (3rd- 4th year) were trained to perform vertical mattress suture in small groups. Then, during 6-week period of self-directed practice, students were randomized into group1 VDO-feedback (male:female = 21:8) and group 2 F2F-feedback (male:female = 20:9). Feedbacks were provided once every 2 weeks (Week2, Week4). End-of-rotation OSCE was at Week6, and retention tested was at Week8. Performance checklist (Cronbach's Alpha 0.72) was used to assess skill at 4 timepoints; pre- and post- small group learning, OSCE, and retention phase. Questionnaire was used to assess motivation, learning strategies and satisfaction (Cronbach's Alpha 0.83).ResultAfter in-class learning, further significant improvement of skills could be gained by both F2F- and VDO- feedbacks (p < 0.0001). Both could similarly retain skill for at least 4 weeks later without additional practice. Self-efficacy, test anxiety, and cognitive strategies scores were significantly increased in both groups (p < 0.05). Extrinsic motivation was increased in VDO-feedback group. No difference in satisfaction between groups was observed.Discussion and conclusionVDO-feedback could be alternative to F2F-feedbacks for basic surgical skill training when limitation for simultaneous meeting of teacher and students occurs.Trial registrationThis study has been registered to Thai Clinical Trial Registry (WHO International Clinical Trial Registry Platform) on 11/07/2023 (TCTR20230711005).

Dataset Information

A Checklist to Help Faculty Assess ACGME Milestones in a Video-Recorded OSCE.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets