Dataset Information

Assessing communication skills during OSCE: need for integrated psychometric approaches.

ABSTRACT:

Background

Physicians' communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curriculum. The adoption of reliable measurement instruments is thus essential to evaluate such skills.

Methods

Using Exploratory Factor Analysis (EFA), Multi-Group Confirmatory Factor Analysis (MGCFA) and Item Response Theory analysis (IRT) the current retrospective study tested the factorial validity and reliability of a four-item global rating scale developed by Hodges and McIlroy to measure CS among 296 third- and fourth-year medical students at the Faculty of Medicine in Geneva, Switzerland, during OSCEs.

Results

EFA results at each station showed good reliability scores. However, measurement invariance assessments through MGCFA across different stations (i.e., same students undergoing six or three stations) and across different groups of stations (i.e., different students undergoing groups of six or three stations) were not satisfactory, failing to meet the minimum requirements to establish measurement invariance and thus possibly affecting reliable comparisons between students' communication scores across stations. IRT revealed that the four communication items provided overlapping information focusing especially on high levels of the communication spectrum.

Conclusions

Using this four-item set in its current form it may be difficult to adequately differentiate between students who are poor in CS from those who perform better. Future directions in best-practices to assess CS among medical students in the context of OSCE may thus focus on (1) training examiners so to obtain scores that are more coherent across stations; and (2) evaluating items in terms of their ability to cover a wider spectrum of medical students' CS. In this respect, IRT can prove to be very useful for the continuous evaluation of CS measurement instruments in performance-based assessments.

SUBMITTER: Piumatti G

PROVIDER: S-EPMC7887794 | biostudies-literature | 2021 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing communication skills during OSCE: need for integrated psychometric approaches.

Piumatti Giovanni G Cerutti Bernard B Perron Noëlle Junod NJ

BMC medical education 20210216 1

<h4>Background</h4>Physicians' communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curriculum. The adoption of reliable measurement instruments is thus essential to evaluate such skills.<h4>Methods</h4>Using Exploratory Factor Analysis (EFA), Multi-Group Confirmatory Factor Analysis ...[more]

PMID: 33593345

Similar Datasets

Project description:BackgroundTeaching and assessment of communication skills have become essential in medical education. The Objective Structured Clinical Examination (OSCE) has been found as an appropriate means to assess communication skills within medical education. Studies have demonstrated the importance of a valid assessment of medical students' communication skills. Yet, the validity of the performance scores depends fundamentally on the quality of the rating scales used in an OSCE. Thus, this systematic review aimed at providing an overview of existing rating scales, describing their underlying definition of communication skills, determining the methodological quality of psychometric studies and the quality of psychometric properties of the identified rating scales.MethodsWe conducted a systematic review to identify psychometrically tested rating scales, which have been applied in OSCE settings to assess communication skills of medical students. Our search strategy comprised three databases (EMBASE, PsycINFO, and PubMed), reference tracking and consultation of experts. We included studies that reported psychometric properties of communication skills assessment rating scales used in OSCEs by examiners only. The methodological quality of included studies was assessed using the COnsensus based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. The quality of psychometric properties was evaluated using the quality criteria of Terwee and colleagues.ResultsData of twelve studies reporting on eight rating scales on communication skills assessment in OSCEs were included. Five of eight rating scales were explicitly developed based on a specific definition of communication skills. The methodological quality of studies was mainly poor. The psychometric quality of the eight rating scales was mainly intermediate.DiscussionOur results reveal that future psychometric evaluation studies focusing on improving the methodological quality are needed in order to yield psychometrically sound results of the OSCEs assessing communication skills. This is especially important given that most OSCE rating scales are used for summative assessment, and thus have an impact on medical students' academic success.

Project description:ObjectivesThis study investigates the effectiveness of a virtual format of an advanced communication skills observed structured clinical examination (OSCE) for senior medical students in comparison to an in-person format. The study also examines the emotional support students experience in the virtual setting. Our analysis was based on quantitative data collected through objective checklists and post-OSCE survey results.MethodsThe virtual OSCE was a revision of an earlier in-person formative advanced communication skills OSCE for fourth-year medical students. Student performances were assessed by self and peers using objective checklists-the modified Master Interview Rating Scale (mMIRS) and Communication Behavior Checklist (CBC). The mMIRS measured interview process such as avoiding jargon and demonstrating empathy. The CBC examined interview content which included tasks specific to the content of the case. The OSCE was followed by a faculty-led debrief and quantitative survey. The virtual OSCE was conducted in 2021, and the results of the checklists and survey were compared with those collected from two earlier in-person OSCEs.ResultsEighty-three students participated in the virtual OSCE. There was no difference in mMIRS scores between the virtual and in-person OSCE. Overall CBC scores were lower in the virtual OSCE compared to in-person (p < 0.05). Sixty-seven out of 83 (80.7%) students completed the post-OSCE survey. There were no differences between the virtual and in-person OSCE in terms of educational value, whether the OSCE would change the way participants talk to patients, and preparedness to have serious conversations with patients. All respondents somewhat or strongly agreed with feeling emotionally supported during the virtual OSCE.ConclusionThe virtual format was a suitable alternative to an in-person, formative, advanced communication skills OSCE for medical students. The virtual OSCE was educationally effective and was met with student satisfaction and a sense of emotional support. Future virtual iterations must ensure adequate instruction on interview content.

Project description:BackgroundThe National Academy of Medicine recently identified improving clinicians' serious illness communication skills as a necessary step in improving patient and family outcomes near the end of life, but there is not an accepted set of core communication skills for engaging with surrogate decision makers.ObjectiveTo determine the core serious illness communication skills clinicians should acquire to care for incapacitated, hospitalized patients with acute, life-threatening illness, including patients with Alzheimer's disease and related dementias.MethodsFrom January 2019 to July 2020, we conducted a modified Delphi study with a panel of 79 experts in the field of serious illness communication. We developed a preliminary list of candidate communication skills through a structured literature review. We presented the candidate skills to the panelists in the context of three prototypical serious illness conversations. Over three rounds, panelists first augmented the list of candidate skills, then voted on the skills. The final set included skills deemed "very important" or "essential" by 70% of panelists. For external validation, we engaged 11 practicing clinicians and 7 community stakeholders for their perspectives on the expert-endorsed list of skills.ResultsThe panelists' ratings indicate the importance of a diverse set of communication skills related to providing clear information exchange as well as emotional and psychological support to surrogates. The final set included 33 skills, 12 of which were endorsed for all three prototypical serious illness conversations. Practicing clinicians and community stakeholders supported the expert-endorsed framework with only minor additions.ConclusionWe generated a stakeholder-endorsed list of skills that can inform the content of communication skills training programs for clinicians who care for incapacitated patients in the inpatient setting. The skills go beyond those required to provide traditional cognitive decision support and suggest the need for a paradigm shift in curricular content for communication training.

Project description:BackgroundVaccine hesitancy is challenging for clinicians and of increasing concern since COVID-19 vaccination rollout began. Standardized patients (SPs) provide an ideal method for assessing resident physicians' current skills, providing opportunity to practice and gain immediate feedback, while also informing evaluation of curriculum and training. As such, we designed and implemented an OSCE station where residents were tasked with engaging and educating a vaccine-hesitant patient.AimDescribe residents' vaccine counseling practices, core communication and interpersonal skills, and effectiveness in meeting the objectives of the case. Explore how effectiveness in overcoming vaccine hesitancy may be associated with communication and interpersonal skills in order to inform educational efforts.SettingAnnual OSCE at a simulation center.Participants106 internal medicine residents (51% PGY1, 49% PGY2).Program descriptionResidents participated in an annual residency-wide, multi-station OSCE, one of which included a Black, middle-aged, vaccine-hesitant male presenting for a routine video visit. Residents had 10 min to complete the encounter, during which they sought to educate, explore concerns, and make a recommendation. After each encounter, faculty gave residents feedback on their counseling skills and reviewed best practices for effective communication on the topic. SPs completed a behaviorally anchored checklist (30 items across 7 clinical skill domains and 2 measures of trust in the vaccine's safety and resident) which will inform future curriculum.Program evaluationFifty-five percent (SD: 43%) of the residents performed well on the vaccine-specific education domain. PGY2 residents scored significantly higher on two of the seven domains compared to PGY1s (patient education/counseling-PGY1: 35% (SD: 36%) vs. PGY2: 52% (SD: 41%), p = 0.044 and activation-PGY1: 37% (SD: 45%) vs. PGY2: 59% (SD: 46%), p = 0.016). In regression analyses, education/counseling and vaccine-specific communication skills were strongly, positively associated with trust in the resident and in the vaccine's safety. A review of qualitative data from the SPs' perspective suggested that low performers did not use patient-centered communication skills.DiscussionThis needs assessment suggests that many residents needed in-the-moment feedback, additional education, and vaccine-specific communication practice. Our program plans to reinforce evidence-based practices physicians can implement for vaccine hesitancy through ongoing curriculum, practice, and feedback. This type of needs assessment is replicable at other institutions and can be used, as we have, to ultimately shed light on next steps for programmatic improvement.

Project description:Objective Participation in qualitative research, particularly analysis of recorded medical dialogue, offers real-time, longitudinal immersion that can strengthen clinical trainee communication skills. The study objective was to explore how qualitative research participation impacts clinical trainees’ self-perceived communication skills development and practice. Methods In this study, a 17-member multidisciplinary working group of child life specialists, advanced practice providers, undergraduate/medical students, residents, fellows, attending physicians, social scientists, and career researchers with recent qualitative and communication research experience assembled to discuss this topic using a structured discussion guide. Content analysis was used to identify concepts and themes. Results Three key themes characterizing the impact of qualitative research participation on aspiring clinicians’ communication skills development and practice arose – the 3Cs: (1) Connection, therapeutic alliance, and accompaniment; (2) Clarity and prognostic communication; (3) Compassion, empathy, and understanding. Participants emphasized that qualitative research learning improved their understanding of patient/family lived experiences, preparing them for future clinical encounters, strengthening their emotional intelligence, and promoting self-care, resilience, and professional affirmation. Conclusions Immersion in clinical communication through participation in qualitative research is an under-utilized resource for supporting clinical trainees in communication skills development. Innovation The process of collaborative knowledge production through the collective exploration of an a priori question related to group members’ collective experiences is methodologically innovative. Further, re-thinking qualitative research participation as an underutilized educational opportunity is pedagogically novel, and leaders in medical education and qualitative research should collaborate to realize the potential of this teaching tool. Highlights • Qualitative research participation offers immersion in clinical communication.• Participation impact characterized by the 3 C’s: 1) Connection, 2) Clarity, 3) Compassion.• This is an under-utilized medical education resource for communication skills development.• Medical education and qualitative research leaders should collaborate.

Project description:BackgroundPoor communication skills can potentially compromise patient care. However, as communication skills training (CST) programs are not seen as a priority to many clinical departments, there is a discernible absence of a standardised, recommended framework for these programs to be built upon. This systematic scoping review (SSR) aims to gather prevailing data on existing CSTs to identify key factors in teaching and assessing communication skills in the postgraduate medical setting.MethodsIndependent searches across seven bibliographic databases (PubMed, PsycINFO, EMBASE, ERIC, CINAHL, Scopus and Google Scholar) were carried out. Krishna's Systematic Evidence-Based Approach (SEBA) was used to guide concurrent thematic and content analysis of the data. The themes and categories identified were compared and combined where possible in keeping with this approach and then compared with the tabulated summaries of the included articles.ResultsTwenty-five thousand eight hundred ninety-four abstracts were identified, and 151 articles were included and analysed. The Split Approach revealed similar categories and themes: curriculum design, teaching methods, curriculum content, assessment methods, integration into curriculum, and facilitators and barriers to CST. Amidst a wide variety of curricula designs, efforts to develop the requisite knowledge, skills and attitudes set out by the ACGME current teaching and assessment methods in CST maybe categorised into didactic and interactive methods and assessed along Kirkpatrick's Four Levels of Learning Evaluation.ConclusionsA major flaw in existing CSTs is a lack of curriculum structure, focus and standardisation. Based upon the findings and current design principles identified in this SSR in SEBA, we forward a stepwise approach to designing CST programs. These involve 1) defining goals and learning objectives, 2) identifying target population and ideal characteristics, 3) determining curriculum structure, 4) ensuring adequate resources and mitigating barriers, 5) determining curriculum content, and 6) assessing learners and adopting quality improvement processes.

Project description:BackgroundThe outbreak of coronavirus disease 2019 (COVID-19) and its quick progression to a global pandemic has urged medical schools to shift from didactic to distance learning and assessment approaches. The quality of clinical training and assessment have been jeopardized due to the regulatory restrictions and potential hazards to human lives. The aim of this paper is to evaluate the utility and efficacy of an electronic Objective Structured Clinical Examination (e-OSCE), which attempted to transform the format of a face-to-face OSCE to an e-OSCE.MethodsWe conducted three end of clerkship e-OSCEs for final year medical students in Surgery, Medicine and Family Medicine using the teleconferencing application of Microsoft Teams (MST). The e-OSCE blueprint included the assessment of all clinical skills except physical examination and procedural skills. Examiners supervised e-OSCE from the college campus, while all students were remotely assessed through the MST channels. During the exam, the students stayed in their specified MST channel and examiners rotated across all students. The utility and efficacy of e-OSCE was evaluated using a self-administered questionnaire for students, examiners and e-OSCE team.ResultsThe data analysis showed that 93.4% students and 92.2% examiners agreed with the quality and process of e-OSCE. Similarly, 83.6% students and 98% examiners agreed with the seamless organization of e-OSCE. As many as 45.9% students and 74.5% examiners agreed that e-OSCE was close to real life practice. Approximately one fifth of students and one third of examiners preferred e-OSCE over the face-to-face OSCE. The analysis of qualitative data generated the themes of e-OSCE structure and technology. While majority of participants were satisfied with e-OSCE, students were concerned about examiners' training and e-OSCE contents. Examiners and e-OSCE team recognized the paper-less, tech-savy, fast and reliable format of e-OSCE.ConclusionDuring and beyond COVID- 19 era, e-OSCE is a strong substitute to standard OSCE for assessing clinical competence except for physical examination and procedural skills. The planning and implementation of e-OSCE reflects an ingenuity in the assessment of clinical competencies of medical students.

Dataset Information

Assessing communication skills during OSCE: need for integrated psychometric approaches.

Background

Methods

Results

Conclusions

Publications

Assessing communication skills during OSCE: need for integrated psychometric approaches.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets