Dataset Information

Psychometric characterization of the obstetric communication assessment tool for medical education: a pilot study.

ABSTRACT:

Objective

To characterize the psychometric properties of a novel Obstetric Communication Assessment Tool (OCAT) in a pilot study of standardized difficult OB communication scenarios appropriate for undergraduate medical evaluation.

Methods

We developed and piloted four challenging OB Standardized Patient (SP) scenarios in a sample of twenty-one third year OB/GYN clerkship students: Religious Beliefs (RB), Angry Father (AF), Maternal Smoking (MS), and Intimate Partner Violence (IPV). Five trained Standardized Patient Reviewers (SPRs) independently scored twenty-four randomized video-recorded encounters using the OCAT. Cronbach's alpha and Intraclass Correlation Coefficient-2 (ICC-2) were used to estimate internal consistency (IC) and inter-rater reliability (IRR), respectively. Systematic variation in reviewer scoring was assessed using the Stuart-Maxwell test.

Results

IC was acceptable to excellent with Cronbach's alpha values (and 95% Confidence Intervals [CI]): RB 0.91 (0.86, 0.95), AF 0.76 (0.62, 0.87), MS 0.91 (0.86, 0.95), and IPV 0.94 (0.91, 0.97). IRR was unacceptable to poor with ICC-2 values: RB 0.46 (0.40, 0.53), AF 0.48 (0.41, 0.54), MS 0.52 (0.45, 0.58), and IPV 0.67 (0.61, 0.72). Stuart-Maxwell analysis indicated systematic differences in reviewer stringency.

Conclusions

Our initial characterization of the OCAT demonstrates important issues in communications assessment. We identify scoring inconsistencies due to differences in SPR rigor that require enhanced training to improve assessment reliability. We outline a rational process for initial communication tool validation that may be useful in undergraduate curriculum development, and acknowledge that rigorous validation of OCAT training and implementation is needed to create a valuable OB communication assessment tool.

SUBMITTER: Rodriguez AN

PROVIDER: S-EPMC4912696 | biostudies-literature | 2016 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Psychometric characterization of the obstetric communication assessment tool for medical education: a pilot study.

Rodriguez A Noel AN DeWitt Peter P Fisher Jennifer J Broadfoot Kirsten K Hurt K Joseph KJ

International journal of medical education 20160611

<h4>Objective</h4>To characterize the psychometric properties of a novel Obstetric Communication Assessment Tool (OCAT) in a pilot study of standardized difficult OB communication scenarios appropriate for undergraduate medical evaluation.<h4>Methods</h4>We developed and piloted four challenging OB Standardized Patient (SP) scenarios in a sample of twenty-one third year OB/GYN clerkship students: Religious Beliefs (RB), Angry Father (AF), Maternal Smoking (MS), and Intimate Partner Violence (IPV ...[more]

PMID: 27289202

Similar Datasets

Project description:In self-directed learning environments, students must monitor and assess their learning progress themselves. Formative assessments play a central role as instructional tools to train students in these self-assessment skills. Key to the effectiveness of a formative assessment as a training instrument is the degree of student involvement in the design and implementation of the formative assessment used to train them. Five characteristics are essential to secure a sufficient level of student involvement: (1) congruence (i.e., the tasks of a formative assessment must reflect the instructional content), (2) authenticity (i.e., the tasks of a formative assessment should be related to students' background and study context), (3) consultation (i.e., students should have a say in how their answers are evaluated based on which criteria), (4) transparency (i.e., no "mystery items"-the wording of the items must clearly addresses the targeted content), and (5) accommodation (i.e., all students should have the same chance to complete the items of a formative assessment). These five characteristics of student involvement were used to evaluate the quality of formative assessments that are employed to train undergraduates in a medical school in self-assessment skills. Different from extant approaches that typically rely on instructors' judgments to infer the degree of student involvement, this study uses a paradigm in higher educational research asking students directly about their perceptions of an instructional tool. Thus, here, students had to rate their level of involvement in the design and implementation of the formative assessments used to train them. A total of 140 medical students participated in the study. Multivariate statistical methods were used to analyze the data. The majority of medical students appreciate the use of formative assessments to teach them self-directed learning skills; but the results also show that there is still room for increasing student involvement-especially, for minority students.

Project description:BackgroundQuality improvement (QI) is an essential component of modern clinical practice. Front-line professionals offer valuable perspectives on areas for improvement and are motivated to deliver change. In the UK, all junior doctors are expected to participate in QI in order to advance to the next stage of their training. However, UK undergraduates receive no standardized training in QI methods. This is perpetuated within medical schools by a lack of teaching capacity and competing priorities, and may lead to tokenistic engagement with future QI projects.MethodsWe describe a near-peer teaching programme designed to introduce students to QI methods. This pilot study was conceived and delivered in full by junior doctors and used existing resources to ensure high quality teaching content. 111 fifth-year medical students from the University of Cambridge were taught in interactive, participative workshops that encourage them to develop their own QI change ideas and projects. Core topics included the model for improvement, driver diagrams, stakeholder engagement, measurement for improvement and analysing and presenting data. Students completed surveys before and immediately after this intervention to assess their understanding of and confidence in utilizing QI methods. Questionnaires were also completed by junior doctor tutors.ResultsAnalysis of questionnaires completed before and immediately after the intervention revealed statistically significant improvements in students' self-reported understanding of QI (p < 0.05) and confidence in applying techniques to their own work (p < 0.05). Students expressed a preference for QI teaching delivered by junior doctors, citing a relaxed learning environment and greater relevance to their stage of training. Tutors reported increased confidence in using QI techniques and a greater willingness to engage with QI in future.ConclusionsIn this single-centre study, near-peer teaching produced significant improvements in students' self-reported understanding of QI and confidence in applying QI methods. Near-peer teaching may constitute a sustainable means of teaching essential QI skills at undergraduate level. Future work must evaluate objective measures of student engagement with and competence in conducting QI.

Dataset Information

Psychometric characterization of the obstetric communication assessment tool for medical education: a pilot study.

Objective

Methods

Results

Conclusions

Publications

Psychometric characterization of the obstetric communication assessment tool for medical education: a pilot study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets