Dataset Information

Medical licensing examinations in both Sweden and the US favor pharmacology over lifestyle.

ABSTRACT: Low priority of disease prevention and health promotion in medical education may contribute to lack of lifestyle-counseling in clinical practice. Pharmacology-related knowledge is valued 5 times higher compared to lifestyle-related knowledge in examinations on noncommunicable diseases (NCDs) in undergraduate medical education in Sweden. This study aims to establish (i) whether medical licensing examinations are biased to favor pharmacology- over lifestyle-related knowledge and (ii) whether such a bias is present in both Sweden and the US. We identified 204 NCD-related questions from previous Swedish licensing examinations, and 77 cases from a U.S. question bank commonly used to prepare for the United States Medical Licensing Examination® (USMLE®) Step 3. With the help of expected correct answers, we determined distribution of points attainable for knowledge in the respective category (lifestyle / pharmacology / other) for 5 major NCDs: coronary heart disease (CHD), chronic obstructive pulmonary disease (COPD), diabetes, hypertension, and stroke. The percentage of points attainable for lifestyle-related knowledge was 6.7 (95% CI 4.1-9.3) in Sweden and 4.6 (95%CI 0.0-9.1) in the U.S. The respective percentages for pharmacology-related knowledge were 32.6 (95% CI 26.3-38.8) and 44.5 (95% CI 33.2-55.8) percent. The pharmacology vs. lifestyle-quotas were 4.9 in Sweden and 9.8 in the U.S. Likelihoods of equal emphasis on lifestyle and pharmacology in NCDs was < 0.001 in both countries. There is a marked preference for pharmacology over lifestyle in medical licensing examinations in both Sweden and the U.S. Newly qualified doctors may be inadequately prepared to address preventable causes of NCDs.

SUBMITTER: Krachler B

PROVIDER: S-EPMC8227799 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundOver the past 2 years, researchers have used various medical licensing examinations to test whether ChatGPT (OpenAI) possesses accurate medical knowledge. The performance of each version of ChatGPT on the medical licensing examination in multiple environments showed remarkable differences. At this stage, there is still a lack of a comprehensive understanding of the variability in ChatGPT's performance on different medical licensing examinations.ObjectiveIn this study, we reviewed all studies on ChatGPT performance in medical licensing examinations up to March 2024. This review aims to contribute to the evolving discourse on artificial intelligence (AI) in medical education by providing a comprehensive analysis of the performance of ChatGPT in various environments. The insights gained from this systematic review will guide educators, policymakers, and technical experts to effectively and judiciously use AI in medical education.MethodsWe searched the literature published between January 1, 2022, and March 29, 2024, by searching query strings in Web of Science, PubMed, and Scopus. Two authors screened the literature according to the inclusion and exclusion criteria, extracted data, and independently assessed the quality of the literature concerning Quality Assessment of Diagnostic Accuracy Studies-2. We conducted both qualitative and quantitative analyses.ResultsA total of 45 studies on the performance of different versions of ChatGPT in medical licensing examinations were included in this study. GPT-4 achieved an overall accuracy rate of 81% (95% CI 78-84; P<.01), significantly surpassing the 58% (95% CI 53-63; P<.01) accuracy rate of GPT-3.5. GPT-4 passed the medical examinations in 26 of 29 cases, outperforming the average scores of medical students in 13 of 17 cases. Translating the examination questions into English improved GPT-3.5's performance but did not affect GPT-4. GPT-3.5 showed no difference in performance between examinations from English-speaking and non-English-speaking countries (P=.72), but GPT-4 performed better on examinations from English-speaking countries significantly (P=.02). Any type of prompt could significantly improve GPT-3.5's (P=.03) and GPT-4's (P<.01) performance. GPT-3.5 performed better on short-text questions than on long-text questions. The difficulty of the questions affected the performance of GPT-3.5 and GPT-4. In image-based multiple-choice questions (MCQs), ChatGPT's accuracy rate ranges from 13.1% to 100%. ChatGPT performed significantly worse on open-ended questions than on MCQs.ConclusionsGPT-4 demonstrates considerable potential for future use in medical education. However, due to its insufficient accuracy, inconsistent performance, and the challenges posed by differing medical policies and knowledge across countries, GPT-4 is not yet suitable for use in medical education.Trial registrationPROSPERO CRD42024506687; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=506687.

Dataset Information

Medical licensing examinations in both Sweden and the US favor pharmacology over lifestyle.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets