An Innovative Feedback Tool Leading to Improved Faculty Feedback and Positive Reception by Residents.
Ontology highlight
ABSTRACT:
SUBMITTER: Harrison R
PROVIDER: S-EPMC6948690 | biostudies-literature | 2019 Dec
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:IntroductionFormative evaluations of clinical teaching for emergency medicine (EM) faculty are limited. The goal of this study was to develop a behaviorally-based tool for evaluating and providing feedback to EM faculty based on their clinical teaching skills during a shift.MethodsWe used a three-phase structured development process. Phase 1 used the nominal group technique with a group of faculty first and then with residents to generate potential evaluation items. Phase 2 included separate focus groups and used a modified Delphi technique with faculty and residents, as well as a group of experts to evaluate the items generated in Phase 1. Following this, residents classified the items into novice, intermediate, and advanced educator skills. Once items were determined for inclusion and subsequently ranked they were built into the tool by the investigators (Phase 3).ResultsThe final instrument, the "Faculty Shift Card," is a behaviorally-anchored evaluation and feedback tool used to facilitate feedback to EM faculty about their teaching skills during a shift. The tool has four domains: teaching clinical decision-making; teaching interpersonal skills; teaching procedural skills; and general teaching strategies. Each domain contains novice, intermediate, and advanced sections with 2-5 concrete examples for each level of performance.ConclusionThis structured process resulted in a well-grounded and systematically developed evaluation tool for EM faculty that can provide real-time actionable feedback to faculty and support improved clinical teaching.
| S-EPMC6324693 | biostudies-literature
Project description:IntroductionProviding feedback is a fundamental principle in medical education; however, as educators, our community lacks the necessary skills to give meaningful, impactful feedback to those under our supervision. By improving our feedback-giving skills, we provide concrete ways for trainees to optimize their performance, ultimately leading to better patient care.MethodsIn this faculty development workshop, faculty groups used six feedback video vignettes scripted, enacted, and produced by residents to arrive at a shared mental model of feedback. During workshop development, we used qualitative analysis for faculty narratives combined with the findings from a focused literature review to define dimensions of feedback.ResultsTwenty-three faculty (physical medicine and rehabilitation and neurology) participated in seven small-group workshops. Analysis of group discussion notes yielded 343 codes that were collapsed into 25 coding categories. After incorporating the results of a focused literature review, we identified 48 items grouped into 10 dimensions of feedback. Online session evaluation indicated that faculty members liked the workshop's format and thought they were better at providing feedback to residents as a result of the workshop.DiscussionSmall faculty groups were able to develop a shared mental model of dimensions of feedback that was also grounded in medical education literature. The theme of specificity of feedback was prominent and echoed recent medical education research findings. Defining performance expectations for feedback providers in the form of a practical and psychometrically sound rubric can enhance reliable scoring of feedback performance assessments and should be the next step in our work.
| S-EPMC6519682 | biostudies-literature
Project description:Introduction:Senior resident-led morning report (MR) occurs in many residency programs, but residents rarely receive training on how to facilitate MR or feedback on their facilitation. I created and implemented a required 2-week resident-as-teacher curriculum pairing senior residents with faculty mentors. Methods:The curriculum allowed individualization to each specific resident's needs. The faculty mentor helped the resident set curricular goals, observed and provided feedback on resident facilitation of four MRs and one noontime conference, and reviewed adult learning principles with the resident. The curriculum guided the faculty and resident pair through leading MR and applied evidence to guide resident teaching. I surveyed resident teachers, the residents who attend MR, and faculty mentors to determine the curriculum's perceived educational impact. Results:Over the 2010-2016 academic years, 124 senior residents participated. Senior residents self-reported significantly more confidence, interest, and preparedness for teaching after the curriculum. Trainees attending MR rated the quality of equal value after curriculum implementation, and responded that senior residents leading MR were more likely to give clear explanations, teach at an appropriate level, and were less likely to run out of time. Faculty mentors enjoyed participating and found the one-on-one mentorship relationship important for their satisfaction with the experience. Discussion:Outcome data suggested that the mentorship relationship was the most important element of the curriculum and that flexibility was key to allowing individual needs to be met. The intervention is applicable to other residency programs and specialties, and faculty mentors are not required to have a medical education background.
| S-EPMC7431183 | biostudies-literature
Project description:BACKGROUND: While the Accreditation Council for Graduate Medical Education recommends multisource feedback (MSF) of resident performance, there is no uniformly accepted MSF tool for emergency medicine (EM) trainees, and the process of obtaining MSF in EM residencies is untested. OBJECTIVE: To determine the feasibility of an MSF program and evaluate the intraclass and interclass correlation of a previously reported resident professionalism evaluation, the Humanism Scale (HS). METHODS: To assess 10 third-year EM residents, we distributed an anonymous 9-item modified HS (EM-HS) to emergency department nursing staff, faculty physicians, and patients. The evaluators rated resident performance on a 1 to 9 scale (needs improvement to outstanding). Residents were asked to complete a self-evaluation of performance, using the same scale. ANALYSIS: Generalizability coefficients (Eρ(2)) were used to assess the reliability within evaluator classes. The mean score for each of the 9 questions provided by each evaluator class was calculated for each resident. Correlation coefficients were used to evaluate correlation between rater classes for each question on the EM-HS. Eρ(2) and correlation values greater than 0.70 were deemed acceptable. RESULTS: EM-HSs were obtained from 44 nurses and 12 faculty physicians. The residents had an average of 13 evaluations by emergency department patients. Reliability within faculty and nurses was acceptable, with Eρ(2) of 0.79 and 0.83, respectively. Interclass reliability was good between faculty and nurses. CONCLUSIONS: An MSF program for EM residents is feasible. Intraclass reliability was acceptable for faculty and nurses. However, reliable feedback from patients requires a larger number of patient evaluations.
| S-EPMC3179243 | biostudies-other
Project description:Background/Objectives: The serious illness conversation (SIC) is an evidence-based framework for conversations with patients about a serious illness diagnosis. The objective of our study was to develop and validate a novel tool, the SIC-evaluation exercise (SIC-Ex), to facilitate assessment of resident-led conversations with oncology patients. Design: We developed the SIC-Ex based on SIC and on the Royal College of Canada Medical Oncology milestones. Seven resident trainees and 10 evaluators were recruited. Each trainee conducted an SIC with a patient, which was videotaped. The evaluators watched the videos and evaluated each trainee by using the novel SIC-Ex and the reference Calgary-Cambridge guide (CCG) at months zero and three. We used Kane's validity framework to assess validity. Results: Intra-class correlation using average SIC-Ex scores showed a moderate level of inter-evaluator agreement (range 0.523-0.822). Most evaluators rated a particular resident similar to the group average, except for one to two evaluator outliers in each domain. Test-retest reliability showed a moderate level of consistency among SIC-Ex scores at months zero and three. Global rating at zero and three months showed fair to good/very good inter-evaluator correlation. Pearson correlation coefficients comparing total SIC-Ex and CCG scores were high for most evaluators. Self-scores by trainees did not correlate well with scores by evaluators. Conclusions: SIC-Ex is the first assessment tool that provides evidence for incorporating the SIG guide framework for evaluation of resident competence. SIC-Ex is conceptually related to, but more specific than, CCG in evaluating serious illness conversation skills.
| S-EPMC8241377 | biostudies-literature
Project description:ObjectivesThe vast majority of residents' working time is spent away from patients. In hospital practice, many factors may influence the resident's working day structure.Using an innovative method, we aimed to compare working time allocation among internal medicine residents using time-motion observations. The first study goal was to describe how the method could be used for inter-hospital comparison. The secondary goal was to learn about specific differences in the resident's working day structure in university and non-university hospital settings.DesignTwo separate time-motion studies. Trained peer-observers followed the residents during weekday day shifts with a tablet, able to record 22 different activities and corresponding context (with patient, phone, colleague or computer).SettingInternal medicine residencies at a university (May-July 2015) and a non-university (September-October 2016) community hospital.Participants28 residents (mean age: 29 years, average postgraduate training: 30 months) at university hospital, 21 residents (mean age: 30 years, average postgraduate training: 17 months) at non-university hospital.OutcomesTime spent with patients and time dedicated to activities directly related to patients; description of main differences of time allocation between hospitals.ResultsCumulatively 1051 hours of observation (566 (university hospital)+486 (non-university hospital)) and 92 day shifts (49+43) were evaluated. Daily working time was 11.5 versus 11.3 hours. A median daily period of 195 min (IQR 179-211, 27.9%) and 116 min (IQR 98-134, 17.2%) (p<0.001) was dedicated to direct patient care, respectively.ConclusionsWe successfully identified differences potentially related to each hospital structure and organisation. Inter-hospital comparisons could help set up interventions aiming to improve workday structure and experience of residents.
| S-EPMC7044966 | biostudies-literature
Project description:INTRODUCTION/AIMS: Internists care for older adults and teach geriatrics to trainees, but they often feel ill-prepared for these tasks. The aims of our 1-day Continuing Medical Education workshop were to improve the knowledge and self-perceived competence of general internists in their care of older adults and to increase their geriatrics teaching for learners. SETTING: Two internal medicine training programs encompassing University, Veterans Affairs, and a community-based hospital in Portland, OR, USA. PROGRAM DESCRIPTION: Course faculty identified gaps in assessment of cognition, function, and decisional capacity; managing care transitions; and treatment of behavioral symptoms. To address these gaps, our workshop provided geriatric content discussions followed by small group role plays to apply newly learned content. Forty teaching faculty participated. PROGRAM EVALUATION: Participants completed 13-item multiple-choice pre- and post-workshop geriatric knowledge tests, pre- and post-workshop surveys of self-perceived competence to care for older adults, and completed an open-ended 'commitment to change' prompt after the intervention. Knowledge scores improved following the intervention (61% to 72%, p < .0001), as did self-perceived competence (11 of 14 items significant). Seventy-one percent of participants reported success in meeting their commitment to change goals. DISCUSSION: A 1-day intervention improved teaching faculty knowledge and self-perceived competence to care for older patients and led to self-perceived changes in teaching behaviors.
| S-EPMC2517934 | biostudies-literature
Project description:ObjectivesMultisource feedback (MSF) has potential value in learner assessment, but has not been broadly implemented nor studied in emergency medicine (EM). This study aimed to adapt existing MSF instruments for emergency department implementation, measure feasibility, and collect initial validity evidence to support score interpretation for learner assessment.MethodsResidents from eight U.S. EM residency programs completed a self-assessment and were assessed by eight physicians, eight nonphysician colleagues, and 25 patients using unique instruments. Instruments included a five-point rating scale to assess interpersonal and communication skills, professionalism, systems-based practice, practice-based learning and improvement, and patient care. MSF feasibility was measured by percentage of residents who collected the target number of instruments. To develop internal structure validity evidence, Cronbach's alpha was calculated as a measure of internal consistency.ResultsA total of 125 residents collected a mean of 7.0 physician assessments (n = 752), 6.7 nonphysician assessments (n = 775), and 17.8 patient assessments (n = 2,100) with respective response rates of 67.2, 75.2, and 77.5%. Cronbach's alpha values for physicians, nonphysicians, patients, and self were 0.97, 0.97, 0.96, and 0.96, respectively.ConclusionsThis study demonstrated that MSF implementation is feasible, although challenging. The tool and its scale demonstrated excellent internal consistency. EM educators may find the adaptation process and tools applicable to their learners.
| S-EPMC6001492 | biostudies-literature
Project description:IntroductionReliable team assessment has become a priority because of growing emphasis on interprofessional education and team-based care. Objective rating scales are needed to evaluate interprofessional student teams and individuals and provide real-time feedback.MethodsIn response to a need for behavioral rating scales, we modified the McMaster-Ottawa Scale from a 9-point to a 3-point scale and added descriptive behavioral anchors to define three levels of competency (i.e., below, at, and above expected). This modification is intended to provide consistent rating of individuals and teams in patient settings. We then developed a demonstration video using actors representing four professions to demonstrate the three levels of performance within the team. Our faculty rater tool, consisting of the modified scale and video, is designed to provide standardized ratings in interprofessional educational settings that involve patient care.ResultsWe conducted training sessions with 40 faculty members from seven professions (medicine, dentistry, occupational therapy, nursing, pharmacy, physician assistant, and psychology) over a 2-year period. Immediately after each training session, two trained faculty observers rated interprofessional student teams as they conducted history and assessments on standardized patients. Observer scores were compared with one another and with standard expert ratings of the same teams. Trained observer ratings were consistent across the pairs. The observer training can be conducted within 60-90 minutes with the tool.DiscussionResults of our implementation of the faculty rater tool confirm that the modified McMaster-Ottawa Scale is feasible to administer in clinical settings and that the demonstration video can be easily adopted for standardizing observer ratings.
| S-EPMC6464413 | biostudies-literature
Project description:ObjectiveTo explore how residents experienced the application of the Positive Health dialogue tool (PH-tool) during outpatient consultations and its influence on the delivery of value-based healthcare (VBHC).DesignQualitative study using non-participant observations of outpatient consultations during which residents used the PH-tool, followed by longitudinal individual, semistructured interviews. To analyse the data from observations and interviews, observational form notes' summarisation and categorisation, and an iterative-inductive thematic approach was used.ParticipantsEight residents-five from the ear, nose, and throat-department and three from the gastroenterology-hepatology-department-were selected through convenience sampling, accounting for 79 observations and 79 interviews.ResultsResidents had bivalent experiences with using the PH-tool. Residents discussed three main benefits: a gained insight into the individual patient's context and functioning, a changed dynamics in resident-patient communication, and an increased awareness regarding value in terms of patient-related outcomes and healthcare costs. Three barriers became apparent: doubts regarding the PH-tool's relevance and scope, boundaries of superspecialised medical professionals, and a lack of demarcation in clinical practice.ConclusionThe PH-tool use can be beneficial for residents during outpatient consultations with new patients and follow-up in cases of multidimensional problems, particularly in cases of chronic conditions and generalist care. In these situations, the tool yielded valuable patient information beyond physical health, helped foster patient engagement, and enabled tailoring the treatment plan to individual patients' needs. On the other hand, the PH-tool was not a good fit for simple problems, clearly demarcated help requests, periodic follow-up consultations, or verbose patients. In addition, it was not suitable for superspecialised care, because it yielded an abundance of general information. For particular patients and problems, using the PH-tool seems a promising strategy to increase VBHC delivery. Nevertheless, further research and detailing is needed to better align the PH-tool's broad intent and clinical practice.
| S-EPMC8614141 | biostudies-literature