Dataset Information

Using Transitional Year Milestones to Assess Graduating Medical Students' Skills During a Capstone Course.

ABSTRACT: Undergraduate medical education (UME) follows the lead of graduate medical education (GME) in moving to competency-based assessment. The means for and the timing of competency-based assessments in UME are unclear.We explored the feasibility of using the Accreditation Council for Graduate Medical Education Transitional Year (TY) Milestones to assess student performance during a mandatory, fourth-year capstone course.Our single institution, observational study involved 99 medical students who completed the course in the spring of 2014. Students' skills were assessed by self, peer, and faculty assessment for 6 existing course activities using the TY Milestones. Evaluation completion rates and mean scores were calculated.Students' mean milestone levels ranged between 2.2 and 3.6 (on a 5-level scoring rubric). Level 3 is the performance expected at the completion of a TY. Students performed highest in breaking bad news and developing a quality improvement project, and lowest in developing a learning plan, working in interdisciplinary teams, and stabilizing acutely ill patients. Evaluation completion rates were low for some evaluations, and precluded use of the data for assessing student performance in the capstone course. Students were less likely to complete separate online evaluations. Faculty were less likely to complete evaluations when activities did not include dedicated time for evaluations.Assessment of student competence on 9 TY Milestones during a capstone course was useful, but achieving acceptable evaluation completion rates was challenging. Modifications are necessary if milestone scores from a capstone are intended to be used as a handoff between UME and GME.

SUBMITTER: Clay AS

PROVIDER: S-EPMC4675425 | biostudies-literature | 2015 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Using Transitional Year Milestones to Assess Graduating Medical Students' Skills During a Capstone Course.

Clay Alison S AS Andolsek Kathryn K Grochowski Colleen O'Connor CO Engle Deborah L DL Chudgar Saumil M SM

Journal of graduate medical education 20151201 4

<h4>Background</h4>Undergraduate medical education (UME) follows the lead of graduate medical education (GME) in moving to competency-based assessment. The means for and the timing of competency-based assessments in UME are unclear.<h4>Objective</h4>We explored the feasibility of using the Accreditation Council for Graduate Medical Education Transitional Year (TY) Milestones to assess student performance during a mandatory, fourth-year capstone course.<h4>Methods</h4>Our single institution, obse ...[more]

PMID: 26692982

Similar Datasets

Project description:In the model medical curriculum HannibaL at Hannover Medical School (MHH, Hannover, Germany), communication skills in taking case histories and disclosing diagnoses (breaking bad news) are assessed through an objective structured clinical examination (OSCE). This is part of the examinations which at the MHH represent the equivalent to the First Part of the Medical Examinations. The second year doctor-patient communication course preparing for these examinations was evaluated during the 2009/10 academic year.Using questionnaires specific to the learning objectives, learning needs were assessed, pre-post comparisons of self-assessed competencies were performed and key teaching methods were evaluated (5-point Likert scales, "5"=fully agree). At T0 (start of the course) 267 students participated (response rate: 93.7%), of which 180 filled out the T1 questionnaire during the last session of the course (67.4%). Within-subject analyses of variance and paired t-tests were conducted.The highest learning needs were found for the "to show how"-items regarding history taking and disclosing diagnoses (M=4.4). The T1-T0 comparisons showed the greatest improvements for history taking ("to know how": mean difference = +1.7, "to show how": +1.8, p<.0001 as with all tests) and the "to know how"-item regarding the disclosure of diagnoses (+1.6), followed by the "to show how"-items on disclosing a diagnosis (+1.4), shared decision making (+1.2), self-assessing one's own strengths/weaknesses (+1.0) and confidently approaching new patients (+0.7). Students with T0 values of 1 or 2 on the respective scales improved on average by 2.2 points across all items, students with the value of 3 by 1.1, and from 4 or 5 by 0.1. Methodically, the use of simulated patients was rated the most helpful (M=4.8, 87% with the scale value 5). This doctor-patient communication course is associated with substantial improvements regarding all key learning objectives. Regarding methods, the deployed simulated patients (2-4 per 10-student-course group in 3 of the 7 course sessions, respectively) were rated the most helpful. The present evaluation calls for both further development of the doctor-patient communication curriculum at the MHH and joint activities across medical schools, which are discussed at the end of the paper.

Project description:Introduction"Transitions to residency" programs are designed to maximize quality and safety of patient care, as medical students become residents. However, best instructional or readiness assessment practices are not yet established. We sought to study the impact of a screen-based interactive curriculum designed to prepare interns to address common clinical coverage issues (WISE OnCall) on the clinical skills demonstrated in simulation and hypothesize that performance would improve after completing the module.MethodsSenior medical students were recruited to participate in this single group prestudy/poststudy. Students responded to a call from a standardized nurse (SN) and assessed a standardized patient (SP) with low urine output, interacted with a 45-minute WISE OnCall module on the assessment and management of oliguria, and then evaluated a different SP with low urine output of a different underlying cause. Standardized patients assessed clinical skills with a 37-item, behaviorally anchored checklist measuring clinical skills (intraclass correlation coefficient [ICC], 0.55-0.81). Standardized nurses rated care quality and safety and collaboration and interprofessional communication using a 33-item literature-based, anchored checklist (ICC, 0.47-0.52). Standardized patient and SN ratings of the same student performance were correlated (r, 0.37-0.62; P < 0.01). Physicians assessed clinical reasoning quality based on the students' patient encounter note (ICC, 0.55-0.68), ratings that did not correlate with SP and SN ratings. We compared pre-post clinical skills performance and clinical reasoning. Fifty-two medical students (31%) completed this institutional review board -approved study.ResultsPerformance as measured by the SPs, SNs, and the postencounter note all showed improvement with mostly moderate to large effect sizes (range of Cohen's d, 0.30-1.88; P < 0.05) after completion of the online module. Unexpectedly, professionalism as rated by the SP was poorer after the module (Cohen's d, -0.93; P = 0.000).DiscussionA brief computer-based educational intervention significantly improved graduating medical students' clinical skills needed to be ready for residency.

Project description:BackgroundThe eight main Vietnamese medical schools recently cooperated to produce a book listing the knowledge, attitudes and skills expected of a graduate, including specification of the required level for each skill. The teaching program should ensure that students can reach that level. The objective of this study was to determine the perception of graduating students on whether they had achieved the level set for a selection of clinical and public health skills as a guide for the schools to adjust either the levels or the teaching.MethodsFrom all eight schools, 1136 of the 1528 final year students completed questionnaires just before completed all the requirements for graduation, a response rate of 87% overall (ranging from 74-99% per school). They rated their own competence on a scale of 0-5 for 129 skills selected from the 557 skills listed in the book, and reported where they thought they had learned them. The scores that the students gave themselves were then compared to the levels proposed by the teachers for each skill. The proportions of the self-assessed achievement to the levels expected by the teachers, means self-assessed scores and the coefficients of variation were calculated to make comparisons among disciplines, among schools and among learning sites.ResultsMost students felt they had learned most of the skills for key clinical departments to the required level; this varied little among the schools. Self-assessed skill acquisition in public health and minor clinical disciplines was lower and varied more. Sites outside the classroom were especially important for learning skills. The results revealed key similarities and differences between the teachers and the students in their perception about what could be learned and whereConclusionRevising a curriculum for medical schools demands inputs from all stakeholders. Graduating class students can provide valuable feedback on what they have learned in the existing system. Learning objectives should always be checked with students who have followed their study under existing teaching conditions. The information from the graduates helped to identify potential problem areas where either the objectives or the teaching need adjustment.

Project description:ObjectiveDespite procedural skills being recognized as an important component of medical school education, medical students are not confident in their ability to carry out a range of medical procedures. We conducted an institutional needs assessment and used the results to inform the creation of a procedure-based preclinical elective for first- and second-year students.MethodsWe surveyed second-, third-, and fourth-year medical students at Alpert Medical School as well as select program directors to guide selection of a list of procedures to be taught in the elective. We then created an extracurricular 10-week procedural skills course for preclerkship medical students utilizing a hands-on, flipped classroom practice model. Volunteer preceptors were recruited from the Department of Emergency Medicine to participate with a student-to-faculty ratio not exceeding 5:1. Knowledge and skill acquisition were assessed using a multiple-choice knowledge exam and 4-station practical exam, respectively. Pre- and post-course online surveys were used to assess self-perceived confidence for all procedures.ResultsWe implemented our procedural skills training course for first- and second-year medical students in the fall of 2015. Forty-four students applied for the first iteration of the course and 15 students were selected to participate. Fourteen students ultimately completed the elective as well as the subsequent course surveys, multiple-choice exam, and practical exam. Students who participated in the elective had increased levels of self-reported confidence at the conclusion of the elective and performed better on a practical exam and multiple-choice exam compared to students who participated in only the standard curriculum.ConclusionA longitudinal preclerkship procedural course early during medical school is a feasible method of teaching procedural skills to a cohort of learners. A number of adjustments could be made to the course in order to scale up and include a larger cohort of students at our own or another institution.

Project description:IntroductionEffective learning of physical examination skills (PES) requires suitable teaching and learning techniques and assessment methods. The Tribhuvan University (Nepal) curriculum recommends involving the departments of Medicine and Surgery in PES training (PEST) for second year students as a part of early clinical exposure. The project was developed to make teaching/learning of PES structured, involving eight clinical sciences departments and using appropriate methods for teaching and assessment in KIST Medical College, Nepal.MethodsIrby's three stages of clinical teaching model (Preparation, Teaching, Reflection), was applied for teaching. Skill acquisition was based on Millers' learning pyramid at "show how level" and Dreyfus' competency model at "competent level". Teaching/learning was conducted in small groups. A tutorial, demonstration and practice (TDS) model was developed for teaching/learning techniques based on a simple five-step method for teaching clinical skills. Assessment of effectiveness of training was done at "reaction level" as per Kirkpatrick's model based on students' feedback, "shows how level" as per Miller's pyramid of learning by OSCE and "competent level" as per Dreyfus' model using retro-pre questionnaire.ResultsThe analysis of retro-pre questionnaire based on the Dreyfus model found the average skill score (max score 184), before the introduction of the project module as 15.9 (median = 13.5) and after as 116.5 (median = 116). A paired t-test showed the difference to be statistically significant (100.5±23 and 95% CI 95.45 - 105.59). The average overall feedback score for the students on PES training based on seven items on a five point Likert scale was found to be 4.30. The mean total objective structured clinical examination (OSCE) score was 3.77 (SD+/- 0.33) out of 5; 80% of students scored more than 70%.ConclusionStudents learned most of the skills with the implementation of the structured PES module and did well in the OSCE. Students and faculty were satisfied with the training and assessment.

Dataset Information

Using Transitional Year Milestones to Assess Graduating Medical Students' Skills During a Capstone Course.

Publications

Using Transitional Year Milestones to Assess Graduating Medical Students' Skills During a Capstone Course.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets