Dataset Information

Added value of assessing medical students' reflective writings in communication skills training: a longitudinal study in four academic centres.

ABSTRACT: OBJECTIVES:This study describes the development and implementation of a model to assess students' communication skills highlighting the use of reflective writing. We aimed to evaluate the usefulness of the students' reflections in the assessment of communication skills. DESIGN:Third-year and fourth-year medical students enrolled in an elective course on clinical communication skills development were assessed using different assessment methods. SETTING AND PARTICIPANTS:The communication skills course was offered at four universities (three in Brazil and one in Portugal) and included 69 students. OUTCOME MEASURES:The students were assessed by a Multiple-Choice Questionnaire (MCQ), an objective structured clinical examination (OSCE) and reflective writing narratives. The Cronbach's alpha, dimensionality and the person's correlation were applied to evaluate the reliability of the assessment methods and their correlations. Reflective witting was assessed by applying the Reflection Evaluation for Enhanced Competencies Tool Rubric (Reflect Score (RS)) to measure reflections' depth, and the Thematic Score (TS) to map and grade reflections' themes. RESULTS:The Cronbach alpha for the MCQ, OSCE global score, TS and RS were, respectively, 0.697, 0.633, 0.784 and 0.850. The interobserver correlation for the TS and RS were, respectively, 0.907 and 0.816. The assessment of reflection using the TS was significantly correlated with the MCQ (r=0.412; p=0.019), OSCE (0.439; p=0.012) and RS (0.410; p=0.020). The RS did not correlate with the MCQ and OSCE. CONCLUSIONS:Assessing reflection through mapping the themes and analysing the depth of reflective writing expands the assessment of communication skills. While the assessment of reflective themes is related to the cognitive and behavioural domains of learning, the reflective depth seems to be a specific competence, not correlated with other assessment methods-possibly a metacognitive domain.

SUBMITTER: Ament Giuliani Franco C

PROVIDER: S-EPMC7651724 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Added value of assessing medical students' reflective writings in communication skills training: a longitudinal study in four academic centres.

Ament Giuliani Franco Camila C Franco Renato Soleiman RS Cecilio-Fernandes Dario D Severo Milton M Ferreira Maria Amélia MA de Carvalho-Filho Marco Antonio MA

BMJ open 20201106 11

<h4>Objectives</h4>This study describes the development and implementation of a model to assess students' communication skills highlighting the use of reflective writing. We aimed to evaluate the usefulness of the students' reflections in the assessment of communication skills.<h4>Design</h4>Third-year and fourth-year medical students enrolled in an elective course on clinical communication skills development were assessed using different assessment methods.<h4>Setting and participants</h4>The c ...[more]

PMID: 33158823

Similar Datasets

Project description:BackgroundReflective practice (RP) plays a crucial role in encouraging learners to think critically and consciously about their performances. Providing constructive feedback can further enhance RP. But non-Western learners might face different learning barriers compared to learners in the West, where RP originated.MethodsIn this retrospective study, we assessed RP and feedback performances on Thai medical students' patient history-taking skills. We applied RP and peer feedback, along with feedback from the instructors, during the history-taking sessions of the ten-week introduction course for fourth-year medical students. Twelve history-taking sessions were used for the analysis. Two instructors assessed students' reflective performance and categorised them into one of the six stages of Gibbs' reflective cycle; their feedback performances were analysed using Pendleton's model. We investigated the correlations between students' overall grade point average (GPAX) and patient history-taking scores on the Objective Structured Clinical Examination (OSCE). Students' opinions of the RP teaching method were also collected.ResultsAll (n = 48) students participated in our study. The students' mean age was 21.2 ± 0.5 years. The majority of the students were female (64.6%). The data indicated that 33 and 4% of the participants were categorised into the evaluation stage and action plan stage of Gibbs' reflective cycle, respectively. In addition, 22 and 15% of the participants were able to state what their peers did well and suggest how peers could improve their skills, respectively. All students passed the minimum passing level of four history-taking OSCE stations. Participants agreed that RP was a useful tool (mean 9.0, SD 0.1), which enhanced their thought processes (mean 8.4, SD 0.2) and future performances (mean 8.2, SD 0.2). However, there was no correlation between the students' highest Gibbs' reflection levels and their history-taking OSCE scores.ConclusionsRP, together with feedback, proved to be a useful technique to help fourth-year Thai medical students improve their reflection skills, enhance their medical knowledge, and improve patient history-taking skills. Further study with longer monitoring is required to further explore negative and positive influential factors affecting students' achievement of better reflection performances.

Project description:BackgroundTeaching and assessment of communication skills have become essential in medical education. The Objective Structured Clinical Examination (OSCE) has been found as an appropriate means to assess communication skills within medical education. Studies have demonstrated the importance of a valid assessment of medical students' communication skills. Yet, the validity of the performance scores depends fundamentally on the quality of the rating scales used in an OSCE. Thus, this systematic review aimed at providing an overview of existing rating scales, describing their underlying definition of communication skills, determining the methodological quality of psychometric studies and the quality of psychometric properties of the identified rating scales.MethodsWe conducted a systematic review to identify psychometrically tested rating scales, which have been applied in OSCE settings to assess communication skills of medical students. Our search strategy comprised three databases (EMBASE, PsycINFO, and PubMed), reference tracking and consultation of experts. We included studies that reported psychometric properties of communication skills assessment rating scales used in OSCEs by examiners only. The methodological quality of included studies was assessed using the COnsensus based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. The quality of psychometric properties was evaluated using the quality criteria of Terwee and colleagues.ResultsData of twelve studies reporting on eight rating scales on communication skills assessment in OSCEs were included. Five of eight rating scales were explicitly developed based on a specific definition of communication skills. The methodological quality of studies was mainly poor. The psychometric quality of the eight rating scales was mainly intermediate.DiscussionOur results reveal that future psychometric evaluation studies focusing on improving the methodological quality are needed in order to yield psychometrically sound results of the OSCEs assessing communication skills. This is especially important given that most OSCE rating scales are used for summative assessment, and thus have an impact on medical students' academic success.

Project description:Objectives: Training communication skills has come to be recognized as a vital aspect of medical school education. A medical communication course based on the COMSKIL Communication Skills Training (CST) Program was developed, integrated into the core curriculum, and evaluated at the Leipzig University Medical School. Methods: Between October 2016 and July 2017, 312 medical students (mean age 21.80 years; 62% male) participated in the medical communication course. Each course unit was evaluated via questionnaires specifically designed to address the theoretical and practical content of the curriculum. The items correspond to the material covered in each course unit. Students responded using a 5-point-Likert scale (1="not at all helpful", 5="extremely helpful") to rate the degree to which the course helped them learn about the subject matter and train the skills covered in the curriculum. Results: The average score for the first part of the course (theoretical foundations) was M=3.69 (SD=0.35). The second part received a similar rating (M=3.84; SD=0.73). The role play exercises with actor-patients received a score of M=4.27 (SD=0.62). In an overall evaluation at the end of the course, students rated the administration of the course (setting, etc), knowledge gained, and skills trained with a score of M=4.11 (SD=0.66). The role play exercises received an overall score of M=4.36 (SD=0.61). Conclusion: A new curriculum for teaching medical students patient-physician communication skills based on the COMSKIL CST program was established at the University of Leipzig. The goal of this course is to teach students about the kinds of communication scenarios they will encounter in their future working lives as care providers and equip them with the fundamental communication techniques and skills they need to successfully handle those situations. A formal evaluation of the program resulted in satisfactory findings, indicating that it is well suited for use in medical universities.

Project description:BackgroundPhysicians' communication skills (CS) are known to significantly affect the quality of health care. Communication skills training programs are part of most undergraduate medical curricula and are usually assessed in Objective Structured Clinical Examinations (OSCE) throughout the curriculum. The adoption of reliable measurement instruments is thus essential to evaluate such skills.MethodsUsing Exploratory Factor Analysis (EFA), Multi-Group Confirmatory Factor Analysis (MGCFA) and Item Response Theory analysis (IRT) the current retrospective study tested the factorial validity and reliability of a four-item global rating scale developed by Hodges and McIlroy to measure CS among 296 third- and fourth-year medical students at the Faculty of Medicine in Geneva, Switzerland, during OSCEs.ResultsEFA results at each station showed good reliability scores. However, measurement invariance assessments through MGCFA across different stations (i.e., same students undergoing six or three stations) and across different groups of stations (i.e., different students undergoing groups of six or three stations) were not satisfactory, failing to meet the minimum requirements to establish measurement invariance and thus possibly affecting reliable comparisons between students' communication scores across stations. IRT revealed that the four communication items provided overlapping information focusing especially on high levels of the communication spectrum.ConclusionsUsing this four-item set in its current form it may be difficult to adequately differentiate between students who are poor in CS from those who perform better. Future directions in best-practices to assess CS among medical students in the context of OSCE may thus focus on (1) training examiners so to obtain scores that are more coherent across stations; and (2) evaluating items in terms of their ability to cover a wider spectrum of medical students' CS. In this respect, IRT can prove to be very useful for the continuous evaluation of CS measurement instruments in performance-based assessments.

Project description:BackgroundMany academic medical centres have introduced strategies to assess the productivity of faculty as part of compensation schemes. We conducted a systematic review of the effects of such strategies on faculty productivity.MethodsWe searched the MEDLINE, Healthstar, Embase and PsycInfo databases from their date of inception up to October 2011. We included studies that assessed academic productivity in clinical, research, teaching and administrative activities, as well as compensation, promotion processes and satisfaction.ResultsOf 531 full-text articles assessed for eligibility, we included 9 articles reporting on eight studies. The introduction of strategies for assessing academic productivity as part of compensation schemes resulted in increases in clinical productivity (in six of six studies) in terms of clinical revenue, the work component of relative-value units (these units are nonmonetary standard units of measure used to indicate the value of services provided), patient satisfaction and other departmentally used standards. Increases in research productivity were noted (in five of six studies) in terms of funding and publications. There was no change in teaching productivity (in two of five studies) in terms of educational output. Such strategies also resulted in increases in compensation at both individual and group levels (in three studies), with two studies reporting a change in distribution of compensation in favour of junior faculty. None of the studies assessed effects on administrative productivity or promotion processes. The overall quality of evidence was low.InterpretationStrategies introduced to assess productivity as part of a compensation scheme appeared to improve productivity in research activities and possibly improved clinical productivity, but they had no effect in the area of teaching. Compensation increased at both group and individual levels, particularly among junior faculty. Higher quality evidence about the benefits and harms of such assessment strategies is needed.

Dataset Information

Added value of assessing medical students' reflective writings in communication skills training: a longitudinal study in four academic centres.

Publications

Added value of assessing medical students' reflective writings in communication skills training: a longitudinal study in four academic centres.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets