Dataset Information

Practice effects in medical school entrance testing with the undergraduate medicine and health sciences admission test (UMAT).

ABSTRACT:

Background

The UMAT is widely used for selection into undergraduate medical and dental courses in Australia and New Zealand (NZ). It tests aptitudes thought to be especially relevant to medical studies and consists of 3 sections - logical reasoning and problem solving (UMAT-1), understanding people (UMAT-2) and non-verbal reasoning (UMAT-3). A substantial proportion of all candidates re-sit the UMAT. Re-sitting raises the issue as to what might be the precise magnitude and determinants of any practice effects on the UMAT and their implications for equity in subsequent selection processes.

Methods

Between 2000 and 2012, 158,909 UMAT assessments were completed. From these, 135,833 cases were identified where a candidate had sat once or more during that period with 117,505 cases (86.5%) having sat once, 14,739 having sat twice (10.9%), 2,752 thrice (2%) and 837, 4 or more times (0.6%). Subsequent analyses determined predictors of multiple re-sits as well as the magnitude and socio-demographic determinants of any practice effects.

Results

Increased likelihood of re-sitting the UMAT twice or more was predicted by being male, of younger age, being from a non-English language speaking background and being from NZ and for Australian candidates, being urban rather than rurally based. For those who sat at least twice, the total UMAT score between a first and second attempt improved by 10.7 ± 0.2 percentiles, UMAT-1 by 8.3 ± 0.2 percentiles, UMAT-2 by 8.3 ± 0.2 percentiles and UMAT-3 by 7.7 ± 0.2 percentiles. An increase in total UMAT percentile score on re-testing was predicted by a lower initial score and being a candidate from NZ rather than from Australia while a decrease was related to increased length of time since initially sitting the test, older age and non-English language background.

Conclusions

Re-sitting the UMAT augments performance in each of its components together with the total UMAT percentile score. Whether this increase represents just an improvement in performance or an improvement in understanding of the variables and therefore competence needs to be further defined. If only the former, then practice effects may be introducing inequity in student selection for medical or dental schools in Australia or NZ.

SUBMITTER: Puddey IB

PROVIDER: S-EPMC4007585 | biostudies-literature | 2014 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Practice effects in medical school entrance testing with the undergraduate medicine and health sciences admission test (UMAT).

Puddey Ian B IB Mercer Annette A Andrich David D Styles Irene I

BMC medical education 20140312

<h4>Background</h4>The UMAT is widely used for selection into undergraduate medical and dental courses in Australia and New Zealand (NZ). It tests aptitudes thought to be especially relevant to medical studies and consists of 3 sections - logical reasoning and problem solving (UMAT-1), understanding people (UMAT-2) and non-verbal reasoning (UMAT-3). A substantial proportion of all candidates re-sit the UMAT. Re-sitting raises the issue as to what might be the precise magnitude and determinants o ...[more]

PMID: 24618021

Similar Datasets

Project description:BackgroundKnowledge in natural sciences generally predicts study performance in the first two years of the medical curriculum. In order to reduce delay and dropout in the preclinical years, Hamburg Medical School decided to develop a natural science test (HAM-Nat) for student selection. In the present study, two different approaches to scale construction are presented: a unidimensional scale and a scale composed of three subject specific dimensions. Their psychometric properties and relations to academic success are compared.Methods334 first year medical students of the 2006 cohort responded to 52 multiple choice items from biology, physics, and chemistry. For the construction of scales we generated two random subsamples, one for development and one for validation. In the development sample, unidimensional item sets were extracted from the item pool by means of weighted least squares (WLS) factor analysis, and subsequently fitted to the Rasch model. In the validation sample, the scales were subjected to confirmatory factor analysis and, again, Rasch modelling. The outcome measure was academic success after two years.ResultsAlthough the correlational structure within the item set is weak, a unidimensional scale could be fitted to the Rasch model. However, psychometric properties of this scale deteriorated in the validation sample. A model with three highly correlated subject specific factors performed better. All summary scales predicted academic success with an odds ratio of about 2.0. Prediction was independent of high school grades and there was a slight tendency for prediction to be better in females than in males.ConclusionsA model separating biology, physics, and chemistry into different Rasch scales seems to be more suitable for item bank development than a unidimensional model, even when these scales are highly correlated and enter into a global score. When such a combination scale is used to select the upper quartile of applicants, the proportion of successful completion of the curriculum after two years is expected to rise substantially.

Project description:Objective: The concept of perfectionism comprises high standards of performance as needed in medicine, but also concerns about making mistakes and dealing with social reactions about not being perfect. Perfectionism is associated with motivation and deep learning strategies but high expression of perfectionism has been found to be associated with symptoms of stress and anxiety in students. We aim to gain insights into the longitudinal development of different dimensions of perfectionism in medical students with respect to their way of medical school admission. Methods: At the Medical Faculty of Hamburg University, 167 undergraduate medical students completed validated questionnaires (MPS-H and MPS-F) of different dimensions of perfectionism and sociodemographic data including medical school admission procedures, personality traits (BSI-10 and GSE), and symptoms of depression and anxiety (PHQ-9 and GAD-7) at the start of their first year and at half term of their second year. Results: On average, after controlling for baseline and age, a significant decrease (p?0.05) in Self-Oriented Perfectionism was found during the first two years in students who were admitted after a waiting period (M: -12.57; 95% CI: [-21.94 - -3.35]), by other ways of medical school entrance (M: -6.36; 95% CI: [-12.71 - -0.02]), by multiple mini-interviews (HAM-Int) (M: -5.52; 95% CI: [-9.90 - -1.14]), and by a natural science test (HAM-Nat) (M: -3.41; 95% CI: [-6.71 - -0.11]. Waiting period students also showed a significant longitudinal decline in the scale Personal Standards (M: -4.62; 95% CI: [-8.04 - -1.21]. Conclusions: Since medical students from all admission groups except from the high school degree group showed a significant longitudinal decrease in Self-Oriented Perfectionism, high levels of aspects of perfectionism associated with intrinsic motivation or deep learning strategies could be included medical school admission processes. Additionally, particular attention needs to be paid not to induce a loss of intrinsic motivation or deep learning strategies during undergraduate medical education.

Project description:Introduction:Emergency Medicine is a medical specialty based on knowledge and skills required for the prevention, diagnosis and management of the acute and urgent aspects of illness and injury affecting patients of all age groups with a full spectrum of undifferentiated physical and behavioural disorders. Addis Ababa University School of Medicine started its Emergency Medicine Residency in 2010 and Emergency Medicine training for fourth-year medical students started in 2013. This study aims to assess attitudes of fifth year medical students towards Emergency Medicine training and its contribution to their final year of medical school training. Methods:Two hundred fifth year medical students participated in the study by convenience sampling. Self-administered questionnaires and Likert scales were used for data collection. Descriptive frequencies and chi-square analysis were done for categorical data. Ethical oversight was provided by the Institutional Review Board of the Addis Ababa University College of Health Sciences. Results:Of the 200 participants, 150 were male and 50 were female. 80% agreed its relevance for undergraduates. Relevance was significantly associated with recommendation to other medical schools (? 2 = 8.34, Pr = 0.004). 72% of respondents agreed lectures are appropriate teaching methods, 70% agreed group activity, 68.5% skill sessions, 67.5% morning discussions, 64% diagnostic session, 60% duty exposures and 45% seminars. Difficulties faced during internship are primarily attributed to lack of facilities, ranging from the setup of the emergency centre to instruments and emergency drugs. 60% of respondents agreed that Emergency Medicine training is important to future careers. 65% agreed recommending training to other medical schools. Conclusion:An Emergency Medicine rotation during the final year of medical school provides opportunities to learn about undifferentiated medical emergencies and it should be included for other medical schools in the country. Participants suggest that leadership aspects of Emergency Medicine need more emphasis as the curriculum is further developed in the future.

Project description:Objectives: A potential new avenue to address the shortage of country doctors is to change the rules for admission to medical school. We therefore study the link between high-school grade point average and prospective physicians' choice to work in rural areas. To further inform the discussion about rules for admission, we also study the effects of other predictors: a measure of students' attitudes towards risk; whether they waited for their place of study (Wartesemester); whether their parents worked as medical doctors; and whether they have some practical experience in the medical sector. Methods: We conducted two internet surveys in 2012 and 2014. In the first survey, the sample comprised 701 students and in the second, 474 students. In both surveys, we asked students for their regional preferences; in the 2014 survey, we additionally asked students for their first, second, and third preferences among a comprehensive set of specializations, including becoming a general practitioner. In both surveys, we asked students for basic demographic information (age and gender), their parents' occupation, a measure of subjective income expectations, a measure of risk attitudes, and their high-school grade point average (Abiturnote), and First National Boards Examination grade (Physikum). In 2014, we additionally asked for waiting periods (Wartesemester) as well as for prior professional experience in the health-care sector. Results: We find that three factors increase the probability of having a preference for working in a rural area significantly, holding constant all other influences: having a medical doctor among the parents, having worse grades in the high-school grade point average, and being more risk averse. Moreover, we find that those willing to work in the countryside have significantly more experience in the medical sector before admission to medical school. Discussion: Our results suggest that a change in the selection process for medical school may increase the supply of country doctors. Instead of focusing on the high-school grade point average, universities could even more intensely screen for study motivation through interviews or by taking into account students' background, extracurricular activities, or waiting periods.

Project description:Introduction: In 2012 safety strategies were defined in five intervention areas to improve patient safety in Austria. Regarding policy development, patient safety should be mandatory part of education of all healthcare sectors, and measures to improve hygiene standards are to be included in organizational development. The aim of this project was to achieve sustained improvement in routine procedures and anchor patient safety in the undergraduate medical curriculum by making online instructional videos on clinical skills and hygiene procedures permanently available as preparation for the first clinical clerkship. Method: Short films explaining how to insert urinary catheters in women and men were produced and provided online. These videos were shown to medical students shortly before the practical Objective Structured Clinical Examination (OSCE). After viewing the videos, all of the students were surveyed using an online questionnaire with 15 questions regarding quality and acceptance. The effect of the videos on learning success was determined by the assessment outcome through red cards in the practical exam. A red card for behavior endangering the doctor or others meant zero points and discontinuation of the assessment at that particular OSCE station. Results: A total of 647 students viewed one of the two videos on urinary catheters, 623 responded to the online Moodle questionnaire completly. 551 (85.2%) reported being better able to recall individual steps and procedures, 626 students (96.7%) positively rated the fact that instructional videos were available on the Medical University of Vienna's website. More than half of the respondents (56.6%) were better able to remember critical hygiene practices. The comparison of the assessment outcomes on the OSCE for 2016 and 2013, a year in which the instructional videos were not yet available, shows no significant (chi2=3.79; p>0.05) but a trend towards improvement. The chance of getting a red card in 2013 was 3.36 times higher than in 2016. Conclusion: Even if our study was unable to show significant improvements in the OSCE as a result of viewing the videos, it appears that clearly imparting medical skills and hygiene standards-including in visual form-is still important prior to the first clerkship to ensure the highest level of patient safety possible. The combination of teaching and learning formats, such as videos on online platforms with textbooks or lecture notes, is well suited to increase effectiveness and efficiency in learning. There is a need for further studies to investigate and analyze the effects of instructional videos in more detail.

Project description:Background and objective: Integrative Medicine and Health (IMH) is a theory-based paradigm shift for health, disease and health care, which can probably only be achieved by supplementing medical roles and competences. Definition of IMH: The definitions of the Academic Consortium for Integrative Medicine and Health 2015 and the so-called Berlin Agreement: Self-Responsibility and Social Action in Practicing and Fostering Integrative Medicine and Health Globally are used. The basic features of evidence-based Integrative Medicine and Health (EB-IMH) are based on the recommendations on EBM by David L. Sackett. Global State of Undergraduate and Postgraduate Medical Education (UG-PGME) for IMH: The USA and Canada are most advanced in the development of IMH regarding practice, teaching and research worldwide. Despite socio-cultural peculiarities, they can provide guidance for Europe and especially for Germany. Of interest here are competences for UG-PGME in IMH in primary care and in some specialist disciplines (e.g. internal medicine, gynecology, pediatrics, geriatrics, oncology, palliative care). For these specialties, the need for an interprofessional UG-PGME for IMH was shown in the early stages of development. UG-PGME for IMH in Germany: In the course of the development of the new Medical Licensure Act in Germany (ÄApprO), based on a revision of the National Competence-based Catalogue of Learning Objectives for Medicine (NKLM 2.0) and new regulations for Postgraduate Medical Education in Germany, suggestions for an extension of UG-PGME are particularly topical. To some extent there are already approaches to IMH. Old and new regulations are set out and are partly compared. As a result, some essential elements of IMH are mapped in the new ÄApprO. The new regulations for Postgraduate Medical Education do not mention IMH. Conclusion: The development of medical competences for IMH in the continuum of the UG-PGME could be supported by the coordinated introduction of appropriate entrustable professional activities (EPA) and IMH sub-competences combined with appropriate assessment.

Dataset Information

Practice effects in medical school entrance testing with the undergraduate medicine and health sciences admission test (UMAT).

Background

Methods

Results

Conclusions

Publications

Practice effects in medical school entrance testing with the undergraduate medicine and health sciences admission test (UMAT).

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets