Dataset Information

Accuracy of script concordance tests in fourth-year medical students.

ABSTRACT:

Objectives

This investigation aimed to determine the validity of script concordance test (SCT), compared with clinical-case-related short-answer management problems (SAMP), in fourth-year medical students.

Methods

This retrospective study was conducted at the Medical School of Lille University. Cardiology and gynecology examinations both included 3 SCT and 2 clinical-case-related SAMP. Final score did not include SCT results, and was out of 20 points. The passing score was ?10/20. Wilcoxon and McNemar tests were used to compare quantitative and qualitative variables, respectively. Correlation between scores was also analyzed.

Results

A total of 519 and 521 students completed SAMP and SCT in cardiology and gynecology, respectively. Cardiology score was significantly higher in SCT than SAMP (mean ± SD 13.5±2.4 versus 11.4±2.6, Wilcoxon test, p<0.001). In gynecology, SCT score was significantly lower than SAMP score (10.8±2.6 versus 11.4±2.7, Wilcoxon test, p=0.001). SCT and SAMP scores were significantly correlated (p <0.05, Pearson's correlation). However, percentage of students with SCT score ? 10/20 was similar among those who passed or failed cardiology (327 of 359 (91%) vs 146 of 160 (91%), ?²=0.004, df =1, p=0.952), or gynecology (274 of 379 (65%) vs 84 of 142 (59%), ?²=1.614, df=1, p=0.204) SAMP test. Cronbach alpha coefficient was 0.31 and 0.92 for all SCT and SAMP, respectively.

Conclusions

Although significantly correlated, the scores obtained in SCT and SAMP were significantly different in fourth-year medical students. These findings suggest that SCT should not be used for summative purposes in fourth-year medical students.

SUBMITTER: Nseir S

PROVIDER: S-EPMC5339020 | biostudies-literature | 2017 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Accuracy of script concordance tests in fourth-year medical students.

Nseir Saad S Elkalioubie Ahmed A Deruelle Philippe P Lacroix Dominique D Gosset Didier D

International journal of medical education 20170225

<h4>Objectives</h4>This investigation aimed to determine the validity of script concordance test (SCT), compared with clinical-case-related short-answer management problems (SAMP), in fourth-year medical students.<h4>Methods</h4>This retrospective study was conducted at the Medical School of Lille University. Cardiology and gynecology examinations both included 3 SCT and 2 clinical-case-related SAMP. Final score did not include SCT results, and was out of 20 points. The passing score was ≥10/20 ...[more]

PMID: 28237977

Similar Datasets

Project description:BackgroundUndergraduate medical studies represent a wide range of learning opportunities served in the form of various teaching-learning modalities for medical learners. A clinical scenario is frequently used as a modality, followed by multiple-choice and open-ended questions among other learning and teaching methods. As such, script concordance tests (SCTs) can be used to promote a higher level of clinical reasoning. Recent technological developments have made generative artificial intelligence (AI)-based systems such as ChatGPT (OpenAI) available to assist clinician-educators in creating instructional materials.ObjectiveThe main objective of this project is to explore how SCTs generated by ChatGPT compared to SCTs produced by clinical experts on 3 major elements: the scenario (stem), clinical questions, and expert opinion.MethodsThis mixed method study evaluated 3 ChatGPT-generated SCTs with 3 expert-created SCTs using a predefined framework. Clinician-educators as well as resident doctors in psychiatry involved in undergraduate medical education in Quebec, Canada, evaluated via a web-based survey the 6 SCTs on 3 criteria: the scenario, clinical questions, and expert opinion. They were also asked to describe the strengths and weaknesses of the SCTs.ResultsA total of 102 respondents assessed the SCTs. There were no significant distinctions between the 2 types of SCTs concerning the scenario (P=.84), clinical questions (P=.99), and expert opinion (P=.07), as interpretated by the respondents. Indeed, respondents struggled to differentiate between ChatGPT- and expert-generated SCTs. ChatGPT showcased promise in expediting SCT design, aligning well with Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria, albeit with a tendency toward caricatured scenarios and simplistic content.ConclusionsThis study is the first to concentrate on the design of SCTs supported by AI in a period where medicine is changing swiftly and where technologies generated from AI are expanding much faster. This study suggests that ChatGPT can be a valuable tool in creating educational materials, and further validation is essential to ensure educational efficacy and accuracy.

Project description:ProblemThe World Health Organization calls on all with quality medical information to share it with the public and combat health misinformation; however, U.S. medical schools do not currently teach students effective communication with lay audiences about health. Most physicians have inadequate training in mass communication strategies.ApproachIn August 2018, a novel 90-minute class at the University of Minnesota Medical School introduced fourth-year medical students to basic skills for communicating with lay audiences through mass media. Instructors were physicians with experience interacting with the general public via radio, op-ed articles, social media, print media, television, and community and legislative advocacy. After a 20-minute lecture and sharing of instructors' personal experiences, students completed two 30-minute small-group activities. They identified communications objectives and talking points for a health topic, drafting these as Tweets or an op-ed article outline, then presented talking points in a mock press conference with their peers, practicing skills just learned. Pre- and postsurveys documented students' previous engagement and comfort with future engagement with mass media messaging.OutcomesOver 1 week, 142 students participated in 6 separate classes, and 127 completed both pre- and postsurveys. Before the course, only 6% (7/127) of students had comfortably engaged with social media and 14% (18/127) had engaged with traditional media in their professional roles. After the course, students self-reported an increase in their comfort, perceived ability, and likelihood of using specific communications skills to advocate for their patients (all P < .001).Next stepsThe course will be expanded into a 5-session thread for third- and fourth-year medical students spread over 2 years. This thread will include meeting physicians who engage with lay audiences, identifying best practices for mass health communication, identifying bias and misinformation, "dos and don'ts" of social media, and communication skills for legislative advocacy.

Dataset Information

Accuracy of script concordance tests in fourth-year medical students.

Objectives

Methods

Results

Conclusions

Publications

Accuracy of script concordance tests in fourth-year medical students.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets