Dataset Information

Training Medical Students to Create and Collaboratively Review Multiple-Choice Questions: A Comprehensive Workshop.

ABSTRACT:

Introduction

Multiple-choice question (MCQ) creation is an infrequently used active-learning strategy. Previous studies demonstrated that medical students find value in the process, but have minimal training, which may limit potential learning benefits. We therefore developed a process for question-creation that required students to complete in-depth training, in addition to collaborative question-writing and editing.

Methods

We created a question-writing workshop consisting of three components: (1) training in MCQ writing utilizing NBME online modules, a practice MCQ-writing session, and a training session, (2) writing MCQs independently after choosing topics from an institutionally generated blueprint, and (3) reviewing and editing MCQs via an in-person session. To understand students' perceptions, we held two four-student focus groups and recorded/transcribed the data. We iteratively reviewed the transcripts to generate a codebook and corresponding themes. We used the focus group data to generate a survey with Likert-scale questions, which we sent to the remaining 10 students and analyzed using Microsoft Excel.

Results

Eighteen second-year medical students participated in this workshop. Students perceived that question-writing training (3.7/5.0±0.5) and question writing (3.9/5.0±0.3) benefitted their learning. Students perceived that MCQ writing required concept integration (4.1/5.0±0.6). Students described how question writing allowed them to recognize subtle distinctions between therapies and diagnoses. Each MCQ required about 1.5 hours to write and collaboratively edit.

Discussion

Our results demonstrated that students perceived question writing to benefit their learning. More importantly, students felt that question writing actively engaged them to integrate content and compare concepts; students' engagement suggests that they learned from this question-writing activity.

SUBMITTER: Kurtz J

PROVIDER: S-EPMC7549389 | biostudies-literature | 2020 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Training Medical Students to Create and Collaboratively Review Multiple-Choice Questions: A Comprehensive Workshop.

Kurtz Josh J Holman Beth B Monrad Seetha U SU

MedEdPORTAL : the journal of teaching and learning resources 20201006

<h4>Introduction</h4>Multiple-choice question (MCQ) creation is an infrequently used active-learning strategy. Previous studies demonstrated that medical students find value in the process, but have minimal training, which may limit potential learning benefits. We therefore developed a process for question-creation that required students to complete in-depth training, in addition to collaborative question-writing and editing.<h4>Methods</h4>We created a question-writing workshop consisting of th ...[more]

PMID: 33083537

Similar Datasets

Project description:Multiple choice questions (MCQs) offer high reliability and easy machine-marking, but allow for cueing and stimulate recognition-based learning. Very short answer questions (VSAQs), which are open-ended questions requiring a very short answer, may circumvent these limitations. Although VSAQ use in medical assessment increases, almost all research on reliability and validity of VSAQs in medical education has been performed by a single research group with extensive experience in the development of VSAQs. Therefore, we aimed to validate previous findings about VSAQ reliability, discrimination, and acceptability in undergraduate medical students and teachers with limited experience in VSAQs development. To validate the results presented in previous studies, we partially replicated a previous study and extended results on student experiences. Dutch undergraduate medical students (n = 375) were randomized to VSAQs first and MCQs second or vice versa in a formative exam in two courses, to determine reliability, discrimination, and cueing. Acceptability for teachers (i.e., VSAQ review time) was determined in the summative exam. Reliability (Cronbach's α) was 0.74 for VSAQs and 0.57 for MCQs in one course. In the other course, Cronbach's α was 0.87 for VSAQs and 0.83 for MCQs. Discrimination (average Rir) was 0.27 vs. 0.17 and 0.43 vs. 0.39 for VSAQs vs. MCQs, respectively. Reviewing time of one VSAQ for the entire student cohort was ±2 minutes on average. Positive cueing occurred more in MCQs than in VSAQs (20% vs. 4% and 20.8% vs. 8.3% of questions per person in both courses). This study validates the positive results regarding VSAQs reliability, discrimination, and acceptability in undergraduate medical students. Furthermore, we demonstrate that VSAQ use is reliable among teachers with limited experience in writing and marking VSAQs. The short learning curve for teachers, favourable marking time and applicability regardless of the topic suggest that VSAQs might also be valuable beyond medical assessment.

Project description:Objectives: Physicians with different professional backgrounds often diagnose a patients' problem collaboratively. In this article, we first introduce a process model for collaborative diagnosing (CDR model), describe the development of a simulation used to empirically examine the facilitation of collaborative diagnostic reasoning. Based on a contemporary validity framework [1], we further suggest indicators for validity and collect initial evidence with respect to the scoring, generalization, extrapolation, and implication inferences to assess the validity of the simulation when used to assess effects of learning interventions. Method: In a quasi-experimental study, we assessed objectivity and reliability of the simulation and compared medical students with low and advanced prior knowledge to practitioners with high prior knowledge with respect to their diagnostic accuracy, diagnostic efficiency, information sharing skills, and their intrinsic cognitive load. Additionally, we obtained authenticity ratings from practitioners with high prior knowledge. Results: The results yielded satisfying initial evidence for the validity of the scoring and the extrapolation inferences as ratings are objective, and the simulation and the collaborative process is perceived as rather authentic. Additionally, participants on different levels of prior knowledge differ with respect to their diagnostic accuracy, diagnostic efficiency, information sharing skills, and their reported intrinsic cognitive load. With one exception (information sharing skills), the generalization inference seems to be valid as well. Conclusions: We conclude that collecting validity evidence for the simulation was an important step towards a better interpretation of the simulation. We found that the simulation is an authentic and valid representation of the chosen collaborative situation and that the collected validity evidence offers sufficient evidence for an initial validation of the simulation. Nevertheless, the validation process highlighted some important gaps that need further consideration. We further conclude that applying a validation model to the context of empirical research is promising and encourage other researchers to follow the example.

Project description:During the preclinical years, single-best-answer multiple-choice questions (SBA-MCQs) are often used to test the higher-order cognitive processes of medical students (such as application and analysis) while simultaneously assessing lower-order processes (like knowledge and comprehension). Consequently, it can be difficult to pinpoint which learning outcome has been achieved or needs improvement. We developed a new scoring system for SBA-MCQs using a step-by-step methodology to evaluate each learning outcome independently. Enrolled in this study were third-year medical students (n = 316) who had registered in the basic microbiology course at the Faculty of Medicine, Siriraj Hospital, Mahidol University during the academic year 2017. A step-by-step SBA-MCQ with a new scoring system was created and used as a tool to evaluate the validity of the traditional SBA-MCQs that assess two separate outcomes simultaneously. The scores for the two methods, in percentages, were compared using two different questions (SBA-MCQ1 and SBA-MCQ2). SBA-MCQ1 tested the students' knowledge of the causative agent of a specific infectious disease and the basic characteristics of the microorganism, while SBA-MCQ2 tested their knowledge of the causative agent of a specific infectious disease and the pathogenic mechanism of the microorganism. The mean score obtained with the traditional SBA-MCQs was significantly lower than that obtained with the step-by-step SBA-MCQs (85.9% for the traditional approach versus 90.9% for step-by-step SBA-MCQ1; p < 0.001; and 81.5% for the traditional system versus 87.4% for step-by-step SBA-MCQ2; p < 0.001). Moreover, 65.8% and 87.8% of the students scored lower with the traditional SBA-MCQ1 and the traditional SBA-MCQ2, respectively, than with the corresponding sets of step-by-step SBA-MCQ questions. These results suggest that traditional SBA-MCQ scores need to be interpreted with caution because they have the potential to underestimate the learning achievement of students. Therefore, the step-by-step SBA-MCQ is preferable to the traditional SBA-MCQs and is recommended for use in examinations during the preclinical years.

Dataset Information

Training Medical Students to Create and Collaboratively Review Multiple-Choice Questions: A Comprehensive Workshop.

Introduction

Methods

Results

Discussion

Publications

Training Medical Students to Create and Collaboratively Review Multiple-Choice Questions: A Comprehensive Workshop.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets