Automated Test Assembly Using SAS Operations Research Software in a Medical Licensing Examination
Ontology highlight
ABSTRACT:
SUBMITTER: Shao C
PROVIDER: S-EPMC7174804 | biostudies-literature | 2019 May
REPOSITORIES: biostudies-literature
Similar Datasets
Project description: Not available
| S-EPMC11345959 | biostudies-literature
Project description:BackgroundMaintaining data quality and integrity is important for research studies involving prospective data collection. Data must be entered, erroneous or missing data must be identified and corrected if possible, and an audit trail created.MethodsUsing as an example a large prospective study, the Missouri Lower Respiratory Infection (LRI) Project, we present an approach to data management predominantly using SAS software. The Missouri LRI Project was a prospective cohort study of nursing home residents who developed an LRI. Subjects were enrolled, data collected, and follow-ups occurred for over three years. Data were collected on twenty different forms. Forms were inspected visually and sent off-site for data entry. SAS software was used to read the entered data files, check for potential errors, apply corrections to data sets, and combine batches into analytic data sets. The data management procedures are described.ResultsStudy data collection resulted in over 20,000 completed forms. Data management was successful, resulting in clean, internally consistent data sets for analysis. The amount of time required for data management was substantially underestimated.ConclusionData management for prospective studies should be planned well in advance of data collection. An ongoing process with data entered and checked as they become available allows timely recovery of errors and missing data.
| S-EPMC2546431 | biostudies-literature
Project description: Not available
| S-EPMC6839191 | biostudies-literature
Project description:The purpose of this study was to examine the opinions of medical students and physician writers regarding the medical humanities as a subject and its inclusion in the medical school curriculum. Furthermore, we addressed whether an assessment test should be added to the National Medical Licensing Examination of Korea (KMLE). A total of 192 medical students at Inha University and 39 physician writers registered with the Korean Association of Physician Essayists and the Korean Association of Physician Poets participated in this study. They were asked to answer a series of questionnaires. Most medical students (59%) and all physician writers (100%) answered that the medical humanities should be included in the medical school curriculum to train good physicians. They thought that the KMLE did not currently include an assessment of the medical humanities (medical students 69%, physician writers 69%). Most physician writers (87%; Likert scale, 4.38±0.78) felt that an assessment of the medical humanities should be included in the KMLE. Half of the medical students (51%; Likert scale, 2.51±1.17) were against including it in the KMLE, which they would have to pass after several years of study. For the preferred field of assessment, medical ethics was the most commonly endorsed subject (medical students 59%, physician writers 39%). The most frequently preferred evaluation method was via an interview (medical students 45%, physician writers 33%). In terms of the assessment of the medical humanities and the addition of this subject to the KMLE, an interview-based evaluation should be developed.
| S-EPMC4309938 | biostudies-literature
Project description: Not available
| S-EPMC8520127 | biostudies-literature
Project description:PurposeThe goal of this study was to characterize the difficulty index of the items in the skills test components of the class I and II Korean emergency medical technician licensing examination (KEMTLE), which requires examinees to select items randomly.MethodsThe results of 1,309 class I KEMTLE examinations and 1,801 class II KEMTLE examinations in 2013 were subjected to analysis. Items from the basic and advanced skills test sections of the KEMTLE were compared to determine whether some were significantly more difficult than others.ResultsIn the class I KEMTLE, all 4 of the items on the basic skills test showed significant variation in difficulty index (P<0.01), as well as 4 of the 5 items on the advanced skills test (P<0.05). In the class II KEMTLE, 4 of the 5 items on the basic skills test showed significantly different difficulty index (P<0.01), as well as all 3 of the advanced skills test items (P<0.01).ConclusionIn the skills test components of the class I and II KEMTLE, the procedure in which examinees randomly select questions should be revised to require examinees to respond to a set of fixed items in order to improve the reliability of the national licensing examination.
| S-EPMC4751291 | biostudies-literature
Project description:Exploration of the underlying biological mechanisms of disease is useful for many purposes such as the development of novel treatment modalities in addition to informing on-going risk factor research. DNA-microarray technology is a relatively recent and novel approach to conducting genome-wide gene expression studies to identify previously unknown biological pathways associated with disease. The copious data arising from microarray experiments is not conducive to traditional analytical approaches. Beyond the analytical challenges, there are equally important issues related to the interpretation and presentation of results. This chapter outlines appropriate techniques for analyzing microarray data in a fashion that also yields a list of top genes with differential expression in a given experiment. Derivatives of the top genes list can be used as a starting point for the presentation of study results. The list also serves as the basis for additional techniques related to enhanced interpretation and presentation of results. All analyses described in this chapter can be performed using relatively limited computational resources such as a lap top PC with at least 2.0 GB of memory and 2.0 GHz of processing speed.
| S-EPMC6824719 | biostudies-literature
Project description:BackgroundChatGPT (OpenAI) has gained considerable attention because of its natural and intuitive responses. ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers, as stated by OpenAI as a limitation. However, considering that ChatGPT is an interactive AI that has been trained to reduce the output of unethical sentences, the reliability of the training data is high and the usefulness of the output content is promising. Fortunately, in March 2023, a new version of ChatGPT, GPT-4, was released, which, according to internal evaluations, was expected to increase the likelihood of producing factual responses by 40% compared with its predecessor, GPT-3.5. The usefulness of this version of ChatGPT in English is widely appreciated. It is also increasingly being evaluated as a system for obtaining medical information in languages other than English. Although it does not reach a passing score on the national medical examination in Chinese, its accuracy is expected to gradually improve. Evaluation of ChatGPT with Japanese input is limited, although there have been reports on the accuracy of ChatGPT's answers to clinical questions regarding the Japanese Society of Hypertension guidelines and on the performance of the National Nursing Examination.ObjectiveThe objective of this study is to evaluate whether ChatGPT can provide accurate diagnoses and medical knowledge for Japanese input.MethodsQuestions from the National Medical Licensing Examination (NMLE) in Japan, administered by the Japanese Ministry of Health, Labour and Welfare in 2022, were used. All 400 questions were included. Exclusion criteria were figures and tables that ChatGPT could not recognize; only text questions were extracted. We instructed GPT-3.5 and GPT-4 to input the Japanese questions as they were and to output the correct answers for each question. The output of ChatGPT was verified by 2 general practice physicians. In case of discrepancies, they were checked by another physician to make a final decision. The overall performance was evaluated by calculating the percentage of correct answers output by GPT-3.5 and GPT-4.ResultsOf the 400 questions, 292 were analyzed. Questions containing charts, which are not supported by ChatGPT, were excluded. The correct response rate for GPT-4 was 81.5% (237/292), which was significantly higher than the rate for GPT-3.5, 42.8% (125/292). Moreover, GPT-4 surpassed the passing standard (>72%) for the NMLE, indicating its potential as a diagnostic and therapeutic decision aid for physicians.ConclusionsGPT-4 reached the passing standard for the NMLE in Japan, entered in Japanese, although it is limited to written questions. As the accelerated progress in the past few months has shown, the performance of the AI will improve as the large language model continues to learn more, and it may well become a decision support system for medical professionals by providing more accurate information.
| S-EPMC10612006 | biostudies-literature
Project description:This article aims to describe the training and medical licensing system (uieop) for becoming a physician officer (uigwan) during Korea's Goryeo Dynasty (918-1392). In the Goryeo Dynasty, although no license was necessary to provide medical services to the common people, there was a licensing examination to become a physician officer. No other national licensing system for healthcare professionals existed in Korea at that time. The medical licensing examination was administered beginning in 958. Physician officers who passed the medical licensing examination worked in two main healthcare institutions: the Government Hospital (Taeuigam) and Pharmacy for the King (Sangyakguk). The promotion and expansion of medical education differed depending on the historical period. Until the reign of King Munjong (1046-1083), medical education as a path to licensure was encouraged in order to increase the number of physician officers qualifying for licensure by examination; thus, the number of applicants sitting for the examination increased. However, in the late Goryeo Dynasty, after the officer class of the local authorities (hyangri) showed a tendency to monopolize the examination, the Goryeo government limited the examination applications by this group. The medical licensing examination was divided into two parts: medicine and 'feeling the pulse and acupuncture' (jugeumeop). The Goryeo Dynasty followed the Chinese Dang Dynasty's medical system while also taking a strong interest in the Chinese Song Dynasty's ideas about medicine.
| S-EPMC4536363 | biostudies-literature
Project description:BackgroundChatGPT has shown impressive performance in national medical licensing examinations, such as the United States Medical Licensing Examination (USMLE), even passing it with expert-level performance. However, there is a lack of research on its performance in low-income countries' national licensing medical examinations. In Peru, where almost one out of three examinees fails the national licensing medical examination, ChatGPT has the potential to enhance medical education.ObjectiveWe aimed to assess the accuracy of ChatGPT using GPT-3.5 and GPT-4 on the Peruvian National Licensing Medical Examination (Examen Nacional de Medicina [ENAM]). Additionally, we sought to identify factors associated with incorrect answers provided by ChatGPT.MethodsWe used the ENAM 2022 data set, which consisted of 180 multiple-choice questions, to evaluate the performance of ChatGPT. Various prompts were used, and accuracy was evaluated. The performance of ChatGPT was compared to that of a sample of 1025 examinees. Factors such as question type, Peruvian-specific knowledge, discrimination, difficulty, quality of questions, and subject were analyzed to determine their influence on incorrect answers. Questions that received incorrect answers underwent a three-step process involving different prompts to explore the potential impact of adding roles and context on ChatGPT's accuracy.ResultsGPT-4 achieved an accuracy of 86% on the ENAM, followed by GPT-3.5 with 77%. The accuracy obtained by the 1025 examinees was 55%. There was a fair agreement (κ=0.38) between GPT-3.5 and GPT-4. Moderate-to-high-difficulty questions were associated with incorrect answers in the crude and adjusted model for GPT-3.5 (odds ratio [OR] 6.6, 95% CI 2.73-15.95) and GPT-4 (OR 33.23, 95% CI 4.3-257.12). After reinputting questions that received incorrect answers, GPT-3.5 went from 41 (100%) to 12 (29%) incorrect answers, and GPT-4 from 25 (100%) to 4 (16%).ConclusionsOur study found that ChatGPT (GPT-3.5 and GPT-4) can achieve expert-level performance on the ENAM, outperforming most of our examinees. We found fair agreement between both GPT-3.5 and GPT-4. Incorrect answers were associated with the difficulty of questions, which may resemble human performance. Furthermore, by reinputting questions that initially received incorrect answers with different prompts containing additional roles and context, ChatGPT achieved improved accuracy.
| S-EPMC10570896 | biostudies-literature