Dataset Information

A Comparison of Assessment Tools: Is Direct Observation an Improvement Over Objective Structured Clinical Examinations for Communications Skills Evaluation?

ABSTRACT: Background :Evaluation of resident physicians' communications skills is a challenging task and is increasingly accomplished with standardized examinations. There exists a need to identify the effective, efficient methods for assessment of communications skills. Objective :We compared objective structured clinical examination (OSCE) and direct observation as approaches for assessing resident communications skills. Methods :We conducted a retrospective cohort analysis of orthopaedic surgery resident physicians at a single tertiary care academic institution, using the Institute for Healthcare Communication "4 Es" model for effective communication. Data were collected between 2011 and 2015. A total of 28 residents, each with OSCE and complete direct observation assessment checklists, were included in the analysis. Residents were included if they had 1 OSCE assessment and 2 or more complete direct observation assessments. Results :There were 28 of a possible 59 residents (47%) included. A total of 89% (25 of 28) of residents passed the communications skills OSCE; only 54% (15 of 28) of residents passed the direct observation communications assessment. There was a positive, moderate correlation between OSCE and direct observation scores overall (r = 0.415, P = .028). There was no agreement between OSCE and direct observation in categorizing residents into passing and failing scores (κ = 0.205, P = .16), after adjusting for chance agreement. Conclusions :Our results suggest that OSCE and direct observation tools provide different insights into resident communications skills (simulation of rare and challenging situations versus real-life daily encounters), and may provide useful perspectives on resident communications skills in different contexts.

SUBMITTER: Goch AM

PROVIDER: S-EPMC5901804 | biostudies-other | 2018 Apr

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

A Comparison of Assessment Tools: Is Direct Observation an Improvement Over Objective Structured Clinical Examinations for Communications Skills Evaluation?

Goch Abraham M AM Karia Raj R Taormina David D Kalet Adina A Zuckerman Joseph J Egol Kenneth A KA Phillips Donna D

Journal of graduate medical education 20180401 2

<h4>Background</h4>Evaluation of resident physicians' communications skills is a challenging task and is increasingly accomplished with standardized examinations. There exists a need to identify the effective, efficient methods for assessment of communications skills.<h4>Objective</h4>We compared objective structured clinical examination (OSCE) and direct observation as approaches for assessing resident communications skills.<h4>Methods</h4>We conducted a retrospective cohort analysis of orthopa ...[more]

PMID: 29686764

Similar Datasets

Project description:BackgroundTeaching and assessment of communication skills have become essential in medical education. The Objective Structured Clinical Examination (OSCE) has been found as an appropriate means to assess communication skills within medical education. Studies have demonstrated the importance of a valid assessment of medical students' communication skills. Yet, the validity of the performance scores depends fundamentally on the quality of the rating scales used in an OSCE. Thus, this systematic review aimed at providing an overview of existing rating scales, describing their underlying definition of communication skills, determining the methodological quality of psychometric studies and the quality of psychometric properties of the identified rating scales.MethodsWe conducted a systematic review to identify psychometrically tested rating scales, which have been applied in OSCE settings to assess communication skills of medical students. Our search strategy comprised three databases (EMBASE, PsycINFO, and PubMed), reference tracking and consultation of experts. We included studies that reported psychometric properties of communication skills assessment rating scales used in OSCEs by examiners only. The methodological quality of included studies was assessed using the COnsensus based Standards for the selection of health status Measurement INstruments (COSMIN) checklist. The quality of psychometric properties was evaluated using the quality criteria of Terwee and colleagues.ResultsData of twelve studies reporting on eight rating scales on communication skills assessment in OSCEs were included. Five of eight rating scales were explicitly developed based on a specific definition of communication skills. The methodological quality of studies was mainly poor. The psychometric quality of the eight rating scales was mainly intermediate.DiscussionOur results reveal that future psychometric evaluation studies focusing on improving the methodological quality are needed in order to yield psychometrically sound results of the OSCEs assessing communication skills. This is especially important given that most OSCE rating scales are used for summative assessment, and thus have an impact on medical students' academic success.

Dataset Information

A Comparison of Assessment Tools: Is Direct Observation an Improvement Over Objective Structured Clinical Examinations for Communications Skills Evaluation?

Publications

A Comparison of Assessment Tools: Is Direct Observation an Improvement Over Objective Structured Clinical Examinations for Communications Skills Evaluation?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure