Dataset Information

Voluntary vs. compulsory student evaluation of clerkships: effect on validity and potential bias.

ABSTRACT:

Background

Students evaluations of their learning experiences can provide a useful source of information about clerkship effectiveness in undergraduate medical education. However, low response rates in clerkship evaluation surveys remain an important limitation. This study examined the impact of increasing response rates using a compulsory approach on validity evidence.

Methods

Data included 192 responses obtained voluntarily from 49 third-year students in 2014-2015, and 171 responses obtained compulsorily from 49 students in the first six months of the consecutive year at one medical school in Lebanon. Evidence supporting internal structure and response process validity was compared between the two administration modalities. The authors also tested for potential bias introduced by the use of the compulsory approach by examining students' responses to a sham item that was added to the last survey administration.

Results

Response rates increased from 56% in the voluntary group to 100% in the compulsory group (P ConclusionsWhile increasing response rates is often a policy requirement aimed to improve the credibility of ratings, using authority to enforce responses may not increase reliability and can raise concerns over the meaningfulness of the evaluation. Administrators are urged to consider not only response rates, but also representativeness and quality of responses in administering evaluation surveys.

SUBMITTER: Aoun Bahous S

PROVIDER: S-EPMC5756350 | biostudies-literature | 2018 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Voluntary vs. compulsory student evaluation of clerkships: effect on validity and potential bias.

Aoun Bahous Sola S Salameh Pascale P Salloum Angelique A Salameh Wael W Park Yoon Soo YS Tekian Ara A

BMC medical education 20180105 1

<h4>Background</h4>Students evaluations of their learning experiences can provide a useful source of information about clerkship effectiveness in undergraduate medical education. However, low response rates in clerkship evaluation surveys remain an important limitation. This study examined the impact of increasing response rates using a compulsory approach on validity evidence.<h4>Methods</h4>Data included 192 responses obtained voluntarily from 49 third-year students in 2014-2015, and 171 respo ...[more]

PMID: 29304800

Similar Datasets

Project description:BACKGROUND:Objective structured clinical examinations (OSCEs) are a useful method to evaluate medical students' performance in the clerkship years. OSCEs are designed to assess skills and knowledge in a standardized clinical setting and through use of a preset standard grading sheet, so that clinical knowledge can be evaluated at a high level and in a reproducible way. OBJECTIVE:This study aimed to present our OSCE assessment tool designed specifically for radiology clerkship medical students, which we called the objective structured radiology examination (OSRE), with the intent to advance the assessment of clerkship medical students by providing an objective, structured, reproducible, and low-cost method to evaluate medical students' radiology knowledge and the reproducibility of this assessment tool. METHODS:We designed 9 different OSRE cases for radiology clerkship classes with participating third- and fourth-year medical students. Each examination comprises 1 to 3 images, a clinical scenario, and structured questions, along with a standardized scoring sheet that allows for an objective and low-cost assessment. Each medical student completed 3 of 9 random examination cases during their rotation. To evaluate for reproducibility of our scoring sheet assessment tool, we used 5 examiners to grade the same students. Reproducibility for each case and consistency for each grader were assessed with a two-way mixed effects intraclass correlation coefficient (ICC). An ICC below 0.4 was deemed poor to fair, an ICC of 0.41 to 0.60 was moderate, an ICC of 0.6 to 0.8 was substantial, and an ICC greater than 0.8 was almost perfect. We also assessed the correlation of scores and the students' clinical experience with a linear regression model and compared mean grades between third- and fourth-year students. RESULTS:A total of 181 students (156 third- and 25 fourth-year students) were included in the study for a full academic year. Moreover, 6 of 9 cases demonstrated average ICCs more than 0.6 (substantial correlation), and the average ICCs ranged from 0.36 to 0.80 (P<.001 for all the cases). The average ICC for each grader was more than 0.60 (substantial correlation). The average grade among the third-year students was 11.9 (SD 4.9), compared with 12.8 (SD 5) among the fourth-year students (P=.005). There was no correlation between clinical experience and OSRE grade (-0.02; P=.48), adjusting for the medical school year. CONCLUSIONS:Our OSRE is a reproducible assessment tool with most of our OSRE cases showing substantial correlation, except for 3 cases. No expertise in radiology is needed to grade these examinations using our scoring sheet. There was no correlation between scores and the clinical experience of the medical students tested.

Project description:PurposeMedical school curricula have focused more on early clinical exposure with compressed didactic curricula, raising questions on how pathology can be effectively integrated into clinically relevant medical education. This study highlights how a required 1-week pathology rotation embedded within a surgery clerkship can impact students' knowledge base and perspectives of pathology.MethodsOne hundred ninety-two medical students rotated through a newly designed mandatory 1-week pathology rotation during surgery clerkship. Post-rotation feedback and survey data from students were collected to evaluate their perspectives of pathology. Pathology residents and faculty were surveyed about changes on workflow imposed by the new rotation.ResultsEighty percent of student respondents agreed the rotation improved understanding of pathology workflow and its integration into the larger picture of healthcare delivery. 62% and 66% reported the rotation had a positive impact on their perspectives of pathology and pathologists, respectively. However, a significant number pathology resident respondents noted that integration of students into clinical activities either slightly (42%) or significantly (5%) decreased their own learning. Both pathology faculty and residents also noted medical student presence either slightly (19% and 37%, respectively) or significantly (63% and 58%, respectively) decreased workflow efficiency.ConclusionsIntegration of pathology rotations into surgical clerkships is a viable strategy to remedy decreased pathology contact and education due to curricular restructuring that condenses preclinical time while offering medical students a more integrated and practical perspective of pathology as a field. It is essential for pathology departments to prioritize and actively participate in both preclinical and clinical curricular development.Supplementary informationThe online version contains supplementary material available at 10.1007/s40670-022-01569-y.

Project description:BackgroundElective introductory clerkships in dermatology serve a critical function in providing formative experiences to medical students interested in the field. Although dermatology clerkships play a pivotal role in students' career choices and residency preparation, the assessment systems used to evaluate students on these clerkships are widely different and likely affect student experiences.ObjectiveThis study aimed to explore the relationship between dermatology clerkship assessment systems and student experiences through interviews with students about their clerkship reflections and perceptions of assessment.MethodsThe authors contacted clerkship directors via the Association of Professors of Dermatology mailing list and invited them to provide a description of the assessment system at their institution. The authors, via contacted clerkship directors, then invited students who had completed an introductory dermatology clerkship in between 2018 and 2019 to provide a description of the assessment system at their institution and to participate in a qualitative interview about their experiences with assessment systems. The authors then iteratively synthesized interview transcripts using phenomenological analysis, in which a templated approach was used to achieve comprehensive thematic categorization.ResultsPrior to clerkship onset, students expressed a limited understanding of their clinical role and the assessment system. During the clerkship, students endorsed variable expectations across preceptors, limited feedback experiences, and pressures to perform for evaluators. After their clerkship, students continued to perceive assessment systems as nontransparent, subjective, and preordained.ConclusionMedical students perceived assessment systems on introductory dermatology clerkships to be unclear and arbitrary. Encouragingly, students also viewed these challenges in assessment as malleable, identifying several opportunities for educational reform in dermatology clerkships.

Project description:BackgroundImplicit bias instruction is becoming more prevalent in health professions education, with calls for skills-based curricula moving from awareness and recognition to management of implicit bias. Evidence suggests that health professionals and students learning about implicit bias ("learners") have varying attitudes about instruction in implicit bias, including the concept of implicit bias itself. Assessing learner attitudes could inform curriculum development and enable instructional designs that optimize learner engagement. To date, there are no instruments with evidence for construct validity that assess learner attitudes about implicit bias instruction and its relevance to clinical care.MethodsThe authors developed a novel instrument, the Attitude Towards Implicit Bias Instrument (ATIBI) and gathered evidence for three types of construct validity- content, internal consistency, and relationship to other variables.ResultsAuthors utilized a modified Delphi technique with an interprofessional team of experts, as well as cognitive interviews with medical students leading to item refinement to improve content validity. Seven cohorts of medical students, N = 1072 completed the ATIBI. Psychometric analysis demonstrated high internal consistency (α = 0.90). Exploratory factor analysis resulted in five factors. Analysis of a subset of 100 medical students demonstrated a moderate correlation with similar instruments, the Integrative Medicine Attitude Questionnaire (r = 0.63, 95% CI: [0.59, 0.66]) and the Internal Motivation to Respond Without Prejudice Scale (r = 0.36, 95% CI: [0.32, 0.40]), providing evidence for convergent validity. Scores on our instrument had low correlation to the External Motivation to Respond Without Prejudice Scale (r = 0.15, 95% CI: [0.09, 0.19]) and the Groningen Reflection Ability Scale (r = 0.12, 95% CI: [0.06, 0.17]) providing evidence for discriminant validity. Analysis resulted in eighteen items in the final instrument; it is easy to administer, both on paper form and online.ConclusionThe Attitudes Toward Implicit Bias Instrument is a novel instrument that produces reliable and valid scores and may be used to measure medical student attitudes related to implicit bias recognition and management, including attitudes toward acceptance of bias in oneself, implicit bias instruction, and its relevance to clinical care.

Dataset Information

Voluntary vs. compulsory student evaluation of clerkships: effect on validity and potential bias.

Background

Methods

Results

Publications

Voluntary vs. compulsory student evaluation of clerkships: effect on validity and potential bias.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets