Dataset Information

Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study.

ABSTRACT: INTRODUCTION:Progress tests provide students feedback on their level of proficiency over the course of their medical studies. Peer-assisted learning and competency-based education have become increasingly important in medical education. Although progress tests have been proven to be useful as a longitudinal feedback instrument, there are currently no progress tests that have been created in cooperation with students or that focus on competency in medical education. In this study, we investigated the extent to which students can be included in the development of a progress test and demonstrated that aspects of knowledge related to competency can be represented on a competency-based progress test. METHODS:A two-dimensional blueprint for 144 multiple-choice questions (MCQs) covering groups of medical subjects and groups of competency areas was generated by three expert groups for developing the competency-based progress test. A total of 31 students from seven medical schools in Germany actively participated in this exercise. After completing an intensive and comprehensive training programme, the students generated and reviewed the test questions for the competency-based progress test using a separate platform of the ItemManagementSystem (IMS). This test was administered as a formative test to 469 students in a pilot study in November 2013 at eight medical schools in Germany. The scores were analysed for the overall test and differentiated according to the subject groups and competency areas. RESULTS:A pool of more than 200 MCQs was compiled by the students for pilot use, of which 118 student-generated MCQs were used in the progress test. University instructors supplemented this pool with 26 MCQs, which primarily addressed the area of scientific skills. The post-review showed that student-generated MCQs were of high quality with regard to test statistic criteria and content. Overall, the progress test displayed a very high reliability. When the academic years were compared, the progress test mapped out over the course of study not only by the overall test but also in terms of the subject groups and competency areas. OUTLOOK:Further development in cooperation with students will be continued. Focus will be on compiling additional questions and test formats that can represent competency at a higher skill level, such as key feature questions, situational judgement test questions and OSCE. In addition, the feedback formats will be successively expanded. The intention is also to offer the formative competency-based progress test online.

SUBMITTER: Wagener S

PROVIDER: S-EPMC4606478 | biostudies-other | 2015

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study.

Wagener Stefan S Möltner Andreas A Tımbıl Sevgi S Gornostayeva Maryna M Schultz Jobst-Hendrik JH Brüstle Peter P Mohr Daniela D Vander Beken Anna A Better Julian J Fries Martin M Gottschalk Marc M Günther Janine J Herrmann Laura L Kreisel Christian C Moczko Tobias T Illg Claudius C Jassowicz Adam A Müller Andreas A Niesert Moritz M Strübing Felix F Jünger Jana J

GMS Zeitschrift fur medizinische Ausbildung 20151015 4

<h4>Introduction</h4>Progress tests provide students feedback on their level of proficiency over the course of their medical studies. Peer-assisted learning and competency-based education have become increasingly important in medical education. Although progress tests have been proven to be useful as a longitudinal feedback instrument, there are currently no progress tests that have been created in cooperation with students or that focus on competency in medical education. In this study, we inve ...[more]

PMID: 26483859

Similar Datasets

Project description:Radiomics is an emerging technique for the quantification of imaging data that has recently shown great promise for deeper phenotyping of cardiovascular disease. Thus far, the technique has been mostly applied in single-centre studies. However, one of the main difficulties in multi-centre imaging studies is the inherent variability of image characteristics due to centre differences. In this paper, a comprehensive analysis of radiomics variability under several image- and feature-based normalisation techniques was conducted using a multi-centre cardiovascular magnetic resonance dataset. 218 subjects divided into healthy (n = 112) and hypertrophic cardiomyopathy (n = 106, HCM) groups from five different centres were considered. First and second order texture radiomic features were extracted from three regions of interest, namely the left and right ventricular cavities and the left ventricular myocardium. Two methods were used to assess features' variability. First, feature distributions were compared across centres to obtain a distribution similarity index. Second, two classification tasks were proposed to assess: (1) the amount of centre-related information encoded in normalised features (centre identification) and (2) the generalisation ability for a classification model when trained on these features (healthy versus HCM classification). The results showed that the feature-based harmonisation technique ComBat is able to remove the variability introduced by centre information from radiomic features, at the expense of slightly degrading classification performance. Piecewise linear histogram matching normalisation gave features with greater generalisation ability for classification ( balanced accuracy in between 0.78 ± 0.08 and 0.79 ± 0.09). Models trained with features from images without normalisation showed the worst performance overall ( balanced accuracy in between 0.45 ± 0.28 and 0.60 ± 0.22). In conclusion, centre-related information removal did not imply good generalisation ability for classification.

Project description:Background To ascertain if undergraduate medical students attain adequate knowledge to practice in paediatrics, we designed the minimum accepted competency (MAC) examination. This was a set of MCQ’s designed to test the most basic, ‘must know’ knowledge as determined by non-faculty paediatric clinicians. Only two-thirds of undergraduate students passed this exam, despite 96% of the same cohort passing their official university paediatric examination. We aim to describe the psychometric properties of the MAC examination to explore why there was a difference in student performance between these two assessments which should, in theory, be testing the same subject area. We will also investigate if the MAC examination is a potentially reliable method of assessing undergraduate knowledge. Methods The MAC examination was sat by three groups of undergraduate medical students and paediatric trainee doctors. Test item analysis was performed using facility index, discrimination index and Cronbach’s alpha. Results Test item difficulty on the MAC between each group was positively correlated. Correlation of item difficulty with the standard set for each item showed a statistically significant positive relationship. However, for 10 of the items, the mean score achieved by the candidates did not even reach two standard deviations below the standard set by the faculty. Medical students outperformed the trainee doctors on three items. 18 of 30 items achieved a discrimination index > 0.2. Cronbach’s alpha ranged from 0.22–0.59. Conclusion Despite faculty correctly judging that this would be a difficult paper for the candidates, there were a significant number of items on which students performed particularly badly. It is possible that the clinical emphasis in these non-faculty derived questions was juxtaposed with the factual recall often required for university examinations. The MAC examination highlights the difference in the level of knowledge expected of a junior doctor starting work in paediatrics between faculty and non-faculty clinicians and can identify gaps between the current curriculum and the ‘hidden curriculum’ required for real world clinical practice. The faculty comprises physicians in employment by the University whose role it is to design the paediatric curriculum and deliver teaching to undergraduate students. Non-faculty clinicians are paediatric physicians who work soley as clinicians with no affiliation to an educational institution. The concept of a MAC examination to test basic medical knowledge is feasible and the study presented is an encouraging first step towards this method of assessment. Supplementary information The online version contains supplementary material available at 10.1186/s12909-022-03475-8.

Project description:An emerging field of research describes the role of preoperative health behaviours, known as prehabilitation. The preoperative period may be a more physically and emotionally salient time to introduce and foster chronic adherence to health behaviours, such as exercise, in patients compared to post-treatment during recovery. Moreover, physical and psychosocial improvements during the preoperative period may translate into an enhanced recovery trajectory with reduced operative complications and postoperative adverse effects. No studies have assessed prehabilitation for men with prostate cancer undergoing radical prostatectomy.This is a multi-centre, pilot randomized control trial conducted at two Canadian urban teaching hospitals. 100 men undergoing radical prostatectomy for prostate cancer with no contraindications to exercise will be recruited and randomized to the prehabiliation program or usual care. Prehabilitation participants will engage in a preoperative, individualized exercise program including pelvic floor muscle strengthening instructions and a healthy lifestyle guide for men with prostate cancer. These participants will be asked to engage in 60 minutes of home-based, unsupervised, moderate-intensity exercise on 3-4 days per week. Usual care participants will receive the same pelvic floor muscle strengthening instructions and healthy lifestyle guide only. We will assess the feasibility of conducting an adequately powered trial of the same design via recruitment rate, programmatic adherence/contamination, attrition, and safety. Estimates of intervention efficacy will be captured through measurements at baseline (4-8 weeks preoperatively), within 1 week prior to surgery, and postoperatively at 4, 12, and 26 weeks. Efficacy outcomes include: fatigue, quality of life, urinary incontinence, physical fitness, body composition, aerobic fitness, pain, and physical activity volume.The primary outcome of this study is to determine the feasibility of conducting a full-scale, randomized controlled trial of prehabilitation versus usual care and to estimate effect sizes that will inform sample size determinations for subsequent trials in this field. To our knowledge, this is the first study to examine a structured presurgical exercise program for men undergoing radical prostatectomy for prostate cancer. This trial will advance our understanding of strategies to efficiently and effectively use the preoperative period to optimize postoperative recovery.Clinicaltrials.gov Identifier: NCT02036684.

Dataset Information

Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study.

Publications

Development of a competency-based formative progress test with student-generated MCQs: Results from a multi-centre pilot study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets