Dataset Information

Workplace-based assessment of communication skills: A pilot project addressing feasibility, acceptance and reliability.

ABSTRACT: Background: Imparting communication skills has been given great importance in medical curricula. In addition to standardized assessments, students should communicate with real patients in actual clinical situations during workplace-based assessments and receive structured feedback on their performance. The aim of this project was to pilot a formative testing method for workplace-based assessment. Our investigation centered in particular on whether or not physicians view the method as feasible and how high acceptance is among students. In addition, we assessed the reliability of the method. Method: As part of the project, 16 students held two consultations each with chronically ill patients at the medical practice where they were completing GP training. These consultations were video-recorded. The trained mentoring physician rated the student's performance and provided feedback immediately following the consultations using the Berlin Global Rating scale (BGR). Two impartial, trained raters also evaluated the videos using BGR. For qualitative and quantitative analysis, information on how physicians and students viewed feasibility and their levels of acceptance was collected in written form in a partially standardized manner. To test for reliability, the test-retest reliability was calculated for both of the overall evaluations given by each rater. The inter-rater reliability was determined for the three evaluations of each individual consultation. Results: The formative assessment method was rated positively by both physicians and students. It is relatively easy to integrate into daily routines. Its significant value lies in the personal, structured and recurring feedback. The two overall scores for each patient consultation given by the two impartial raters correlate moderately. The degree of uniformity among the three raters in respect to the individual consultations is low. Discussion: Within the scope of this pilot project, only a small sample of physicians and students could be surveyed to a limited extent. There are indications that the assessment can be improved by integrating more information on medical context and student self-assessments. Despite the current limitations regarding test criteria, it is clear that workplace-based assessment of communication skills in the clinical setting is a valuable addition to the communication curricula of medical schools.

SUBMITTER: Weyers S

PROVIDER: S-EPMC5135413 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Workplace-based assessment of communication skills: A pilot project addressing feasibility, acceptance and reliability.

Weyers Simone S Jemi Iman I Karger André A Raski Bianca B Rotthoff Thomas T Pentzek Michael M Mortsiefer Achim A

GMS journal for medical education 20161115 5

<b>Background:</b> Imparting communication skills has been given great importance in medical curricula. In addition to standardized assessments, students should communicate with real patients in actual clinical situations during workplace-based assessments and receive structured feedback on their performance. The aim of this project was to pilot a formative testing method for workplace-based assessment. Our investigation centered in particular on whether or not physicians view the method as feas ...[more]

PMID: 27990466

Similar Datasets

Project description:BackgroundDeveloping communication skills is a key competency for residents. Coaching, broadly accepted as a training modality in medical education, has been proven a successful tool for teaching communication skills. Little research is available thus far to investigate virtual coaching on communication skills for telemedicine encounters. The purpose of the study was to test the hypothesis that virtually coaching residents on communication skills is feasible and acceptable. We surveyed 21 resident-faculty pairs participating in a "fully virtual" coaching session (patient, coach, and resident were virtual).MethodsWe asked 50 neurology resident-faculty coach pairs to complete one "fully virtual" coaching session between May 20 and August 31, 2020. After each session, the resident and coach completed a 15-item survey, including Likert-style scale and open-ended questions, assessing feasibility and acceptability. Descriptive statistics and qualitative content and thematic analyses were performed.ResultsForty-two percent (21/50) of all eligible residents completed "fully virtual" coaching sessions. The overall survey response rate was 91 % (38/42). The majority of respondents agreed that the direct observation and debriefing conversation were easy to schedule and occurred without technical difficulties and that debriefing elements (self-reflection, feedback, takeaways) were useful for residents. Ninety-five percent of respondents rated the coach's virtual presence to be not at all disruptive to the resident-patient interaction. Virtual coaching alleviated resident stress associated with observation and was perceived as an opportunity for immediate feedback and a unique approach for resident education that will persist into the future.ConclusionsIn this pilot study, residents and faculty coaches found virtual coaching on communication skills feasible and acceptable for telemedicine encounters. Many elements of our intervention may be adoptable by other residency programs. For example, residents may share their communication goals with clinic faculty supervisors and then invite them to directly observe virtual encounters what could facilitate targeted feedback related to the resident's goals. Moreover, virtual coaching on communication skills in both the in-person and telemedicine settings may particularly benefit residents in challenging encounters such as those with cognitively impaired patients or with surrogate decision-makers.

Project description:BackgroundResidents may benefit from simulated practice with personalized feedback to prepare for high-stakes disclosure conversations with patients after harmful errors and to meet American Council on Graduate Medical Education mandates. Ideally, feedback would come from patients who have experienced communication after medical harm, but medical researchers and leaders have found it difficult to reach this community, which has made this approach impractical at scale. The Video-Based Communication Assessment app is designed to engage crowdsourced laypeople to rate physician communication skills but has not been evaluated for use with medical harm scenarios.ObjectiveWe aimed to compare the reliability of 2 assessment groups (crowdsourced laypeople and patient advocates) in rating physician error disclosure communication skills using the Video-Based Communication Assessment app.MethodsInternal medicine residents used the Video-Based Communication Assessment app; the case, which consisted of 3 sequential vignettes, depicted a delayed diagnosis of breast cancer. Panels of patient advocates who have experienced harmful medical error, either personally or through a family member, and crowdsourced laypeople used a 5-point scale to rate the residents' error disclosure communication skills (6 items) based on audiorecorded responses. Ratings were aggregated across items and vignettes to create a numerical communication score for each physician. We used analysis of variance, to compare stringency, and Pearson correlation between patient advocates and laypeople, to identify whether rank order would be preserved between groups. We used generalizability theory to examine the difference in assessment reliability between patient advocates and laypeople.ResultsInternal medicine residents (n=20) used the Video-Based Communication Assessment app. All patient advocates (n=8) and 42 of 59 crowdsourced laypeople who had been recruited provided complete, high-quality ratings. Patient advocates rated communication more stringently than crowdsourced laypeople (patient advocates: mean 3.19, SD 0.55; laypeople: mean 3.55, SD 0.40; P<.001), but patient advocates' and crowdsourced laypeople's ratings of physicians were highly correlated (r=0.82, P<.001). Reliability for 8 raters and 6 vignettes was acceptable (patient advocates: G coefficient 0.82; crowdsourced laypeople: G coefficient 0.65). Decision studies estimated that 12 crowdsourced layperson raters and 9 vignettes would yield an acceptable G coefficient of 0.75.ConclusionsCrowdsourced laypeople may represent a sustainable source of reliable assessments of physician error disclosure skills. For a simulated case involving delayed diagnosis of breast cancer, laypeople correctly identified high and low performers. However, at least 12 raters and 9 vignettes are required to ensure adequate reliability and future studies are warranted. Crowdsourced laypeople rate less stringently than raters who have experienced harm. Future research should examine the value of the Video-Based Communication Assessment app for formative assessment, summative assessment, and just-in-time coaching of error disclosure communication skills.

Project description:BackgroundProfessionalism and communication skills constitute important components of the integral formation of physicians which has repercussion on the quality of health care and medical education. The objective of this study was to assess medical graduates' professionalism and communication skills from the patients' perspective and to examine its association with patients' socio-demographic variables.MethodsThis is a hospital based cross-sectional study. It involved 315 patients and 105 medical graduates selected by convenient sampling method. A modified and validated version of the American Board of Internal Medicine's (ABIM) Patient Assessment survey questionnaire was used for data collection through a face to face interview. Data processing and analysis were performed using the Statistical Package for Social Science (SPSS) 16.0. Mean, frequency distribution, and percentage of the variables were calculated. A non-parametric Kruskal Wallis test was applied to verify whether the patients' assessment was influenced by variables such as age, gender, education, at a level of significance, p ≤ 0.05.ResultsFemale patients constituted 46% of the sample, whereas males constituted 54%. The mean age was 36 ± 16. Patients' scoring of the graduate's skills ranged from 3.29 to 3.83 with a mean of 3.64 on a five-point Likert scale. Items assessing the "patient involvement in decision-making" were assigned the minimum mean values, while items dealing with "establishing adequate communication with patient" assigned the maximum mean values. Patients, who were older than 45 years, gave higher scores than younger ones (p < 0.001). Patients with higher education reported much lower scores than those with lower education (p = 0.003). Patients' gender did not show any statistically significant influence on the rating level.ConclusionGenerally patients rated the medical graduates' professionalism and communication skills at a good level. Patients' age and educational level were significantly associated with the rating level.

Project description:ObjectiveTo meet increasing demands to train neuroendovascular techniques, we developed a dedicated simulator applying individualized three-dimensional intracranial aneurysm models ('HANNES'; Hamburg Anatomic Neurointerventional Endovascular Simulator). We hypothesized that HANNES provides a realistic and reproducible training environment to practice coil embolization and to exemplify disparities between neurointerventionalists, thus objectively benchmarking operators at different levels of experience.MethodsSix physicians with different degrees of neurointerventional procedural experience were recruited into a standardized training protocol comprising catheterization of two internal carotid artery (ICA) aneurysms and one basilar tip aneurysm, followed by introduction of one framing coil into each aneurysm and finally complete coil embolization of one determined ICA aneurysm. The level of difficulty increased with every aneurysm. Fluoroscopy was recorded and assessed for procedural characteristics and adverse events.ResultsPhysicians were divided into inexperienced and experienced operators, depending on their experience with microcatheter handling. Mean overall catheterization times increased with difficulty of the aneurysm model. Inexperienced operators showed longer catheterization times (median; IQR: 47; 30-84s) than experienced operators (21; 13-58s, p = 0.011) and became significantly faster during the course of the attempts (rho = -0.493, p = 0.009) than the experienced physicians (rho = -0.318, p = 0.106). Number of dangerous maneuvers throughout all attempts was significantly higher for inexperienced operators (median; IQR: 1.0; 0.0-1.5) as compared to experienced operators (0.0; 0.0-1.0, p = 0.014).ConclusionHANNES represents a modular neurointerventional training environment for practicing aneurysm coil embolization in vitro. Objective procedural metrics correlate with operator experience, suggesting that the system could be useful for assessing operator proficiency.

Project description:BackgroundCommunication skills learned in the classroom do not transfer easily into clinical practice because they are not reinforced by teachers in the workplace setting and because lack of faculty training restricts the transfer of communication skills in real patient encounters. Trained university-based communication skills teachers often work simultaneously as doctors in clinics. This study explored if and how the skills of these teachers play a role in communication skills training in the clinical workplace.MethodsWe used an exploratory sequential design: a mixed method approach that combined a survey with communication skills teachers, and qualitative individual interviews with these teachers and their educational leaders in clinical departments. The questionnaire was analysed using descriptive statistics. The interviews were analysed using content analysis.ResultsThe response rate was 34 %. A majority (93 %) used their communication skills when communicating with patients and relatives. Less than half taught communication in clinical departments. Approximately half of the respondents stated that encouragement from their leaders or colleagues would inspire them to use their teaching skills in the workplace. However, only 20 % had told their leaders about their competencies in teaching communication. One third thought that they needed further teacher training to teach in the clinical workplace. Qualitative analysis showed that teaching opportunities existed but mainly consisted of random, one-off sessions that came about through the initiative of the communication skills teachers themselves. The teachers described several barriers, such as the challenge of teaching colleagues, as communication relates to identity and hierarchical structures, as well as a lack of requests from colleagues or management, and department culture prioritizing topics relating to medical expertise. None of the educational leaders made use of the teachers' specific communication skills in a structured way: some saw it as unimportant, while others saw it as a potential resource.ConclusionTransfer of the teaching skills of communication skills teachers trained for university-based clinical communication training happened, but to a limited degree. Although both opportunities and barriers for transferring communication skills existed, barriers seemed to dominate, and opportunities for communication skills training in the workplace setting were not used to their full potential.

Dataset Information

Workplace-based assessment of communication skills: A pilot project addressing feasibility, acceptance and reliability.

Publications

Workplace-based assessment of communication skills: A pilot project addressing feasibility, acceptance and reliability.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets