Dataset Information

Dimensionality and predictive validity of the HAM-Nat, a test of natural sciences for medical school admission.

ABSTRACT: BACKGROUND: Knowledge in natural sciences generally predicts study performance in the first two years of the medical curriculum. In order to reduce delay and dropout in the preclinical years, Hamburg Medical School decided to develop a natural science test (HAM-Nat) for student selection. In the present study, two different approaches to scale construction are presented: a unidimensional scale and a scale composed of three subject specific dimensions. Their psychometric properties and relations to academic success are compared. METHODS: 334 first year medical students of the 2006 cohort responded to 52 multiple choice items from biology, physics, and chemistry. For the construction of scales we generated two random subsamples, one for development and one for validation. In the development sample, unidimensional item sets were extracted from the item pool by means of weighted least squares (WLS) factor analysis, and subsequently fitted to the Rasch model. In the validation sample, the scales were subjected to confirmatory factor analysis and, again, Rasch modelling. The outcome measure was academic success after two years. RESULTS: Although the correlational structure within the item set is weak, a unidimensional scale could be fitted to the Rasch model. However, psychometric properties of this scale deteriorated in the validation sample. A model with three highly correlated subject specific factors performed better. All summary scales predicted academic success with an odds ratio of about 2.0. Prediction was independent of high school grades and there was a slight tendency for prediction to be better in females than in males. CONCLUSIONS: A model separating biology, physics, and chemistry into different Rasch scales seems to be more suitable for item bank development than a unidimensional model, even when these scales are highly correlated and enter into a global score. When such a combination scale is used to select the upper quartile of applicants, the proportion of successful completion of the curriculum after two years is expected to rise substantially.

SUBMITTER: Hissbach JC

PROVIDER: S-EPMC3261109 | biostudies-literature | 2011

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Dimensionality and predictive validity of the HAM-Nat, a test of natural sciences for medical school admission.

Hissbach Johanna C JC Klusmann Dietrich D Hampe Wolfgang W

BMC medical education 20111014

<h4>Background</h4>Knowledge in natural sciences generally predicts study performance in the first two years of the medical curriculum. In order to reduce delay and dropout in the preclinical years, Hamburg Medical School decided to develop a natural science test (HAM-Nat) for student selection. In the present study, two different approaches to scale construction are presented: a unidimensional scale and a scale composed of three subject specific dimensions. Their psychometric properties and rel ...[more]

PMID: 21999767

Similar Datasets

Project description:AIMS: Tests with natural-scientific content are predictive of the success in the first semesters of medical studies. Some universities in the German speaking countries use the 'Test for medical studies' (TMS) for student selection. One of its test modules, namely "medical and scientific comprehension", measures the ability for deductive reasoning. In contrast, the Hamburg Assessment Test for Medicine, Natural Sciences (HAM-Nat) evaluates knowledge in natural sciences. In this study the predictive power of the HAM-Nat test will be compared to that of the NatDenk test, which is similar to the TMS module "medical and scientific comprehension" in content and structure. METHODS: 162 medical school beginners volunteered to complete either the HAM-Nat (N=77) or the NatDenk test (N=85) in 2007. Until spring 2011, 84.2% of these successfully completed the first part of the medical state examination in Hamburg. Via different logistic regression models we tested the predictive power of high school grade point average (GPA or "Abiturnote") and the test results (HAM-Nat and NatDenk) with regard to the study success criterion "first part of the medical state examination passed successfully up to the end of the 7(th) semester" (Success7Sem). The Odds Ratios (OR) for study success are reported. RESULTS: For both test groups a significant correlation existed between test results and study success (HAM-Nat: OR=2.07; NatDenk: OR=2.58). If both admission criteria are estimated in one model, the main effects (GPA: OR=2.45; test: OR=2.32) and their interaction effect (OR=1.80) are significant in the HAM-Nat test group, whereas in the NatDenk test group only the test result (OR=2.21) significantly contributes to the variance explained. CONCLUSIONS: On their own both HAM-Nat and NatDenk have predictive power for study success, but only the HAM-Nat explains additional variance if combined with GPA. The selection according to HAM-Nat and GPA has under the current circumstances of medical school selection (many good applicants and only a limited number of available spaces) the highest predictive power of all models.

Project description:BackgroundThe UMAT is widely used for selection into undergraduate medical and dental courses in Australia and New Zealand (NZ). It tests aptitudes thought to be especially relevant to medical studies and consists of 3 sections - logical reasoning and problem solving (UMAT-1), understanding people (UMAT-2) and non-verbal reasoning (UMAT-3). A substantial proportion of all candidates re-sit the UMAT. Re-sitting raises the issue as to what might be the precise magnitude and determinants of any practice effects on the UMAT and their implications for equity in subsequent selection processes.MethodsBetween 2000 and 2012, 158,909 UMAT assessments were completed. From these, 135,833 cases were identified where a candidate had sat once or more during that period with 117,505 cases (86.5%) having sat once, 14,739 having sat twice (10.9%), 2,752 thrice (2%) and 837, 4 or more times (0.6%). Subsequent analyses determined predictors of multiple re-sits as well as the magnitude and socio-demographic determinants of any practice effects.ResultsIncreased likelihood of re-sitting the UMAT twice or more was predicted by being male, of younger age, being from a non-English language speaking background and being from NZ and for Australian candidates, being urban rather than rurally based. For those who sat at least twice, the total UMAT score between a first and second attempt improved by 10.7 ± 0.2 percentiles, UMAT-1 by 8.3 ± 0.2 percentiles, UMAT-2 by 8.3 ± 0.2 percentiles and UMAT-3 by 7.7 ± 0.2 percentiles. An increase in total UMAT percentile score on re-testing was predicted by a lower initial score and being a candidate from NZ rather than from Australia while a decrease was related to increased length of time since initially sitting the test, older age and non-English language background.ConclusionsRe-sitting the UMAT augments performance in each of its components together with the total UMAT percentile score. Whether this increase represents just an improvement in performance or an improvement in understanding of the variables and therefore competence needs to be further defined. If only the former, then practice effects may be introducing inequity in student selection for medical or dental schools in Australia or NZ.

Project description:BackgroundAlthough used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools.MethodsThis was a retrospective cohort study of four ICSM (1188 students, entering 2010-2013) and three LKCMedicine cohorts (222 students, 2013-2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A-levels) in a subset of ICSM students.ResultsWhen BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08-0.19; LKCMedicine: 0.22-0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09-0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A-level grades.DiscussionSchools who wish to assess scientific knowledge independently of A-levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

Project description:Background/aimIn the 2012/13 winter semester, the Magdeburg Medical Faculty introduced a test of knowledge for the selection of applicants. The Hamburg Assessment Test for Medicine - Natural Sciences (HAM-Nat) comprises a multiple-choice test with questions on the aspects of biology, physics, chemistry and mathematics relevant to medicine, which was specifically developed for the selection of medicine applicants. The aim is to study how the HAM-Nat influences student selection, the reasons why students decide to take the test as part of their application procedure and what expectations they have of their course of study.MethodsThe selection procedures applied at the university in 2011 (without HAM-Nat) and in 2012-2014 (with HAM-Nat) are compared. On the basis of the results of exploratory interviews, university entrants in winter semester 2013/2014 participated in a written survey on why they chose their subject and place of study and their expectations of their course of study.ResultsNo problems were encountered in introducing the extended selection procedure that included the HAM-Nat Test. The HAM-Nat had a great influence on the selection decision. About 65% of the students admitted would not have obtained a place if the decision had been based exclusively on their Abitur grade [grade obtained in the German school-leaving examination]. On average, male applicants obtained better HAM-Nat results than female ones. The questionnaire was answered by 147 out of 191 university entrants (77%). In the case of applicants from Saxony-Anhalt, the principle reasons for choosing the regional capital are its proximity, the social environment offered, good conditions for studying and the feel-good factor at the university. For the majority of applicants, however, particularly applicants from other federal states, the relatively good chances of admission in Magdeburg were the main reason.ConclusionThe Magdeburg Medical Faculty regards the HAM-Nat as a suitable tool for selecting applicants with outstanding knowledge of natural sciences and thus of increasing and harmonising levels of knowledge at the start of the course. Completion of the standard period of study and success in the 1st part of the German Medical Examination will be the subject of further observation of the students. The HAM-Nat, as a performance-related selection procedure, is not suitable for giving active preference to natives of Saxony-Anhalt in the application procedure but their number has increased since it was introduced. Applicants primarily use the selection procedure tactically to obtain the university place they want to study medicine. Specifics relating to curricula and university profile and research areas are not critical to their choice.

Project description:Objectives: A potential new avenue to address the shortage of country doctors is to change the rules for admission to medical school. We therefore study the link between high-school grade point average and prospective physicians' choice to work in rural areas. To further inform the discussion about rules for admission, we also study the effects of other predictors: a measure of students' attitudes towards risk; whether they waited for their place of study (Wartesemester); whether their parents worked as medical doctors; and whether they have some practical experience in the medical sector. Methods: We conducted two internet surveys in 2012 and 2014. In the first survey, the sample comprised 701 students and in the second, 474 students. In both surveys, we asked students for their regional preferences; in the 2014 survey, we additionally asked students for their first, second, and third preferences among a comprehensive set of specializations, including becoming a general practitioner. In both surveys, we asked students for basic demographic information (age and gender), their parents' occupation, a measure of subjective income expectations, a measure of risk attitudes, and their high-school grade point average (Abiturnote), and First National Boards Examination grade (Physikum). In 2014, we additionally asked for waiting periods (Wartesemester) as well as for prior professional experience in the health-care sector. Results: We find that three factors increase the probability of having a preference for working in a rural area significantly, holding constant all other influences: having a medical doctor among the parents, having worse grades in the high-school grade point average, and being more risk averse. Moreover, we find that those willing to work in the countryside have significantly more experience in the medical sector before admission to medical school. Discussion: Our results suggest that a change in the selection process for medical school may increase the supply of country doctors. Instead of focusing on the high-school grade point average, universities could even more intensely screen for study motivation through interviews or by taking into account students' background, extracurricular activities, or waiting periods.

Dataset Information

Dimensionality and predictive validity of the HAM-Nat, a test of natural sciences for medical school admission.

Publications

Dimensionality and predictive validity of the HAM-Nat, a test of natural sciences for medical school admission.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets