Dataset Information

Are we generating more assessments without added value? Surgical trainees' perceptions of and receptiveness to cross-specialty assessment.

ABSTRACT: INTRODUCTION:Competency-based medical education (CBME) hinges on robust assessment. However, integrating regular workplace-based assessment within demanding and sometimes chaotic clinical environments remains challenging. Many faculty lack assessment expertise, and some programs lack the infrastructure and faculty numbers to fulfill CBME's mandate. Recognizing this, we designed and implemented an assessment innovation that trains and deploys a cadre of faculty to assess in specialties outside their own. Specifically, we explored trainees' perceptions of and receptiveness to this novel assessment approach. METHODS:Within Western University's Surgical Foundations program, 27 PGY?1 trainees were formatively assessed by trained non-surgeons on a basic laparoscopic surgical skill. These assessments did not impact trainees' progression. Four focus groups were conducted to gauge residents' sentiments about the experience of cross-specialty assessment. Data were then analyzed using a thematic analysis approach. RESULTS:While a few trainees found the experience motivating, more often trainees questioned the feedback they received and the practicality of this assessment approach to advance their procedural skill acquisition. What trainees wanted were strategies for improvement, not merely an assessment of performance. DISCUSSION:Trainees' trepidation at the idea of using outside assessors to meet increased assessment demands appeared grounded in their expectations for assessment. What trainees appeared to desire was a coach-someone who could break their performance into its critical individual components-as opposed to an assessor whose role was limited to scoring their performance. Understanding trainees' receptivity to new assessment approaches is crucial; otherwise training programs run the risk of generating more assessments without added value.

SUBMITTER: Burm S

PROVIDER: S-EPMC7459015 | biostudies-literature | 2020 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Are we generating more assessments without added value? Surgical trainees' perceptions of and receptiveness to cross-specialty assessment.

Burm Sarah S Sebok-Syer Stefanie S SS Van Koughnett Julie Ann JA Watling Christopher J CJ

Perspectives on medical education 20200801 4

<h4>Introduction</h4>Competency-based medical education (CBME) hinges on robust assessment. However, integrating regular workplace-based assessment within demanding and sometimes chaotic clinical environments remains challenging. Many faculty lack assessment expertise, and some programs lack the infrastructure and faculty numbers to fulfill CBME's mandate. Recognizing this, we designed and implemented an assessment innovation that trains and deploys a cadre of faculty to assess in specialties ou ...[more]

PMID: 32504448

Similar Datasets

Project description:Introduction ?In the United Kingdom, work-based assessments (WBAs) including procedure-based assessments (PBAs), case-based discussions (CBDs), clinical evaluation exercises (CEXs), and direct observation of procedural skills (DOPS) have been used in Higher General Surgical Training Program (HGSTP) since the introduction of Modernising Medical Careers. Although the Intercollegiate Surgical Curriculum Project states that they should be used for the formative development of trainees using feedback and reflection, there is no study to look at the perception of their usefulness and barriers in using them, particularly in HGSTP. The aim of this study is to investigate trainer's and trainee's perception of their usefulness, barriers in using them, and way forward for their improvement in HGSTP. Methods ?This was a mixed method study. In phase I, after ethics committee approval, an online survey was sent to 83 trainers and 104 trainees, with a response rate of 33 and 37%, respectively, using Online Surveys (formerly Bristol Online Survey) from July 2018 to December 2018. After analysis of this result, in phase II, semistructured interviews were conducted with five trainees and five trainers who had volunteered to take part from phase I. Thematic analysis was performed to develop overarching themes. Results ?For professional formative development, 15% of the trainers and 53% of the trainees felt that WBAs had a low value. Among 4 WBAs-CEX, CBD, PBA, and DOPS-PBA was thought to be the most useful WBA by 52% trainers and 74% trainees. More trainers than trainees felt that it was being used as a formative tool (33 vs. 16%). The total number of WBAs thought to be required was between 20 and 40 per year, with 46% of the trainers and 53% of the trainees preferring these numbers. The thematic analysis generated four themes with subthemes in each: theme 1, "factors affecting usefulness," including the mode of validation, trainer/trainee engagement, and time spent in validating; theme 2, "doubt on utility" due to doubt on validity and being used as a tick-box exercise; theme 3, "pitfalls/difficulties" due to lack of time to validate, late validation, e-mail rather than face-to-face validation, trainer and trainee behavior, variability in feedback given, and emphasis on number than quality; and theme 4, "improvement strategies." Conclusions ?The WBAs are not being used in a way they are supposed to be used. The perception of educational impact (Kirkpatrick levels 1 and 2) by trainers was more optimistic than by trainees. Improvements can be made by giving/finding more time, trainer training, more face-to-face validation, and better trainer trainee interactions.

Project description:Although medical school programs oriented toward postgraduate specialty training have the potential to reduce the duration and cost of medical school for US medical students, success depends on the ability of students to predict their postgraduate specialties. It is clear that first-year choices are poorly predictive, but it is not known when predictions become sufficiently reliable to support specialty-oriented learning programs. We therefore examined the predictive value of specialty preferences expressed at the ends of the first, second and third years of medical school and asked whether concurrent expressions of confidence in choices improved predictive ability. We also investigated the possibility that discrepancies between predicted and actual postgraduate specialty training were related to scores on an examination of knowledge in basic biomedical sciences required for US medical school graduation (the United States Medical Licensing Examination (USLME) Step 1 examination).We calculated positive and negative predictive values (PPV and NPV, respectively) for specialty choices and the sensitivity and specificity of asking for choices for 634 University of Colorado School of Medicine students who trained in 23 accredited residencies from 2011 through 2015. We examined the effect of confidence in first choices in 609 students, and in 334 students, sought an association between USMLE Step 1 scores and switching from postgraduate training specialties predicted at the end of year 2.The PPV of first choices improved from years 1 through 3. NPV was high throughout. PPVs of year 3 first choices ranged from 79% in Anesthesiology to 95% in Psychiatry. Expressions of confidence in first choices did not improve PPV. Sensitivity of asking for first choices increased with time; specificity was consistently high. USLME Step 1 scores were higher for students who ultimately trained in specialties more competitive than first-choice specialties at the end of year 2.Specialty-oriented learning programs during medical school must accommodate students who change career plans. The PPV of specialty first choices improves each year, but even year 3 predictions can be inaccurate with potential loss of students from specialty-specific programs. USMLE Step 1 scores appeared to affect career plans expressed at the end of year 2.

Project description:BackgroundAlthough pathology informatics (PI) is essential to modern pathology practice, the field is often poorly understood. Pathologists who have received little to no exposure to informatics, either in training or in practice, may not recognize the roles that informatics serves in pathology. The purpose of this study was to characterize perceptions of PI by noninformatics-oriented pathologists and to do so at two large centers with differing informatics environments.MethodsPathology trainees and staff at Cleveland Clinic (CC) and Massachusetts General Hospital (MGH) were surveyed. At MGH, pathology department leadership has promoted a pervasive informatics presence through practice, training, and research. At CC, PI efforts focus on production systems that serve a multi-site integrated health system and a reference laboratory, and on the development of applications oriented to department operations. The survey assessed perceived definition of PI, interest in PI, and perceived utility of PI.ResultsThe survey was completed by 107 noninformatics-oriented pathologists and trainees. A majority viewed informatics positively. Except among MGH trainees, confusion of PI with information technology (IT) and help desk services was prominent, even in those who indicated they understood informatics. Attendings and trainees indicated desire to learn more about PI. While most acknowledged that having some level of PI knowledge would be professionally useful and advantageous, only a minority plan to utilize it.ConclusionsInformatics is viewed positively by the majority of noninformatics pathologists at two large centers with differing informatics orientations. Differences in departmental informatics culture can be attributed to the varying perceptions of PI by different individuals. Incorrect perceptions exist, such as conflating PI with IT and help desk services, even among those who claim to understand PI. Further efforts by the PI community could address such misperceptions, which could help enable a better understanding of what PI is and is not, and potentially lead to increased acceptance by non-informaticist pathologists.

Dataset Information

Are we generating more assessments without added value? Surgical trainees' perceptions of and receptiveness to cross-specialty assessment.

Publications

Are we generating more assessments without added value? Surgical trainees' perceptions of and receptiveness to cross-specialty assessment.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets