Dataset Information

The value of the UK Clinical Aptitude Test in predicting pre-clinical performance: a prospective cohort study at Nottingham Medical School.

ABSTRACT:

Background

The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Quantitative Reasoning, Verbal Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admissions panels in advance of the selection process. As yet the predictive validity of the test against course performance is largely unknown.The study objective was to determine whether UKCAT scores predict performance during the first two years of the 5-year undergraduate medical course at Nottingham.

Methods

We studied a single cohort of students, who entered Nottingham Medical School in October 2007 and had taken the UKCAT. We used linear regression analysis to identify independent predictors of marks for different parts of the 2-year preclinical course.

Results

Data were available for 204/260 (78%) of the entry cohort. The UKCAT total score had little predictive value. Quantitative Reasoning was a significant independent predictor of course marks in Theme A ('The Cell'), (p = 0.005), and Verbal Reasoning predicted Theme C ('The Community') (p < 0.001), but otherwise the effects were slight or non-existent.

Conclusion

This limited study from a single entry cohort at one medical school suggests that the predictive value of the UKCAT, particularly the total score, is low. Section scores may predict success in specific types of course assessment.The ultimate test of validity will not be available for some years, when current cohorts of students graduate. However, if this test of mental ability does not predict preclinical performance, it is arguably less likely to predict the outcome in the clinical years. Further research from medical schools with different types of curriculum and assessment is needed, with longitudinal studies throughout the course.

SUBMITTER: Yates J

PROVIDER: S-EPMC2922293 | biostudies-literature | 2010 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The value of the UK Clinical Aptitude Test in predicting pre-clinical performance: a prospective cohort study at Nottingham Medical School.

Yates Janet J James David D

BMC medical education 20100728

<h4>Background</h4>The UK Clinical Aptitude Test (UKCAT) was introduced in 2006 as an additional tool for the selection of medical students. It tests mental ability in four distinct domains (Quantitative Reasoning, Verbal Reasoning, Abstract Reasoning, and Decision Analysis), and the results are available to students and admissions panels in advance of the selection process. As yet the predictive validity of the test against course performance is largely unknown.The study objective was to determ ...[more]

PMID: 20667093

Similar Datasets

Project description:The 2-hour long United Kingdom Clinical Aptitude Test (UKCAT) is used by many universities in the United Kingdom as part of their selection process for undergraduate medical and dentistry degrees. We aimed to compare the performance of senior doctors in primary and secondary care and across a range of specialties, in a modified version of the medical school entrance examination-the mUKCAT. Lay people were also included in the study. Despite its widespread use, this is the first study that examines the performance of senior clinicians in the UKCAT.The study used a prospective cross-sectional design. It used mock questions from the UKCAT website to generate an mUKCAT that was anticipated to take 15?minutes to complete. In all, 167 doctors at consultant, general practitioner (GP), or specialty trainee grade and 26 lay people took part.The overall mean mUKCAT score of all participants was 2486 (69.1%). Of the total cohort, 126 (65.3%) scored above our designated threshold of 2368 and were deemed to have passed the mUKCAT. Excluding lay people, 113 (67.7%) of the 167 doctors scored above that threshold. Medical specialty was associated with overall score (P?=?0.003), with anesthetists/intensive care physicians scoring highest (n?=?20, mean score 2660) and GPs scoring lowest (n?=?38, mean score 2302). Academics outperformed nonacademics (mean score of academics, n?=?44 vs nonacademics, n?=?123: 2750 vs 2406; P?<?0.001). Those clinicians in senior management positions scored lower than those in "standard" roles (mean score of senior management, n?=?31 vs standard roles, n?=?136: 2332 vs 2534, mean difference 202, 95% confidence interval 67-337, P?=?0.004).In the situational judgement section, there was no evidence that specialty was associated with score (P?=?0.15). Academics exhibited greater situational judgement than their nonacademic colleagues (academics vs nonacademics: 69.8 vs 63.6%; P?=?0.01).The majority of senior clinicians passed our mUKCAT. Academics and anesthetists were found to be the best performers, with GPs and those in senior management positions performing the worst.

Project description:Admission to undergraduate medical training in Germany occurs by central and local pathways. Central admission includes two distinct groups: Students with top school-leaving grades (best-SLG group) and students with inferior school-leaving grades who are admitted with a delay of up to seven years (delayed admission group). Students with academic difficulties and early dropouts are present in both groups. Local admission at our university involves the German Test for Medical Studies (TMS) and allows the admission by merit of students with a wide range of school-leaving grades.To examine the justification of a TMS-based strategy to reduce the admission of potentially weak best school-leavers and enhance the admission of potentially able candidates with mediocre school-leaving grades.The prognostic contribution of the school-leaving (SL) GPA and the TMS to academic performance and to continuity in the pre-clinical part of the undergraduate medical program was examined in two study groups: best school leavers (SL grade 1.0, SL-GPA 823-900 points) and mediocre school leavers (SL grades 2.0-2.3, SL-GPA 689-660 points). The outcomes in both groups were compared in relation to their TMS results. The prospective study included four consecutive cohorts.In each study group the TMS predicted the academic performance (β=0.442-0.446) and the continuity of studies (OR=0.890-0.853) better than the SL-GPA (β=0.238-0.047; OR=1.009-0.998). Attrition was most strongly associated with failing to take the TMS (OR=0.230-0.380). Mediocre school leavers with TMS scores ≥125 performed as well as the best school leavers. Mediocre school leavers with TMS scores between 110-124 performed on average less well but within the required standards. Best school leavers with mediocre TMS scores and 30% of the best school leavers who hadn't taken the TMS performed less well than most mediocre school leavers with high TMS scores.The TMS appears to differentiate between potentially successful and less successful students in both GPA categories. Mediocre school leavers (SLG 2.0-2.3) with exceptionally high TMS results reach better pre-clinical examination results than best school leavers (SLG 1.0) with mediocre TMS results. Thus, the present data justify the use of the TMS to facilitate the participation of mediocre school leavers in the competition for admission slots.

Project description:BackgroundSelection of medical students in the UK is still largely based on prior academic achievement, although doubts have been expressed as to whether performance in earlier life is predictive of outcomes later in medical school or post-graduate education. This study analyses data from five longitudinal studies of UK medical students and doctors from the early 1970s until the early 2000s. Two of the studies used the AH5, a group test of general intelligence (that is, intellectual aptitude). Sex and ethnic differences were also analyzed in light of the changing demographics of medical students over the past decades.MethodsData from five cohort studies were available: the Westminster Study (began clinical studies from 1975 to 1982), the 1980, 1985, and 1990 cohort studies (entered medical school in 1981, 1986, and 1991), and the University College London Medical School (UCLMS) Cohort Study (entered clinical studies in 2005 and 2006). Different studies had different outcome measures, but most had performance on basic medical sciences and clinical examinations at medical school, performance in Membership of the Royal Colleges of Physicians (MRCP(UK)) examinations, and being on the General Medical Council Specialist Register.ResultsCorrelation matrices and path analyses are presented. There were robust correlations across different years at medical school, and medical school performance also predicted MRCP(UK) performance and being on the GMC Specialist Register. A-levels correlated somewhat less with undergraduate and post-graduate performance, but there was restriction of range in entrants. General Certificate of Secondary Education (GCSE)/O-level results also predicted undergraduate and post-graduate outcomes, but less so than did A-level results, but there may be incremental validity for clinical and post-graduate performance. The AH5 had some significant correlations with outcome, but they were inconsistent. Sex and ethnicity also had predictive effects on measures of educational attainment, undergraduate, and post-graduate performance. Women performed better in assessments but were less likely to be on the Specialist Register. Non-white participants generally underperformed in undergraduate and post-graduate assessments, but were equally likely to be on the Specialist Register. There was a suggestion of smaller ethnicity effects in earlier studies.ConclusionsThe existence of the Academic Backbone concept is strongly supported, with attainment at secondary school predicting performance in undergraduate and post-graduate medical assessments, and the effects spanning many years. The Academic Backbone is conceptualized in terms of the development of more sophisticated underlying structures of knowledge ('cognitive capital' and 'medical capital'). The Academic Backbone provides strong support for using measures of educational attainment, particularly A-levels, in student selection.

Project description:BackgroundMeasures used for medical student selection should predict future performance during training. A problem for any selection study is that predictor-outcome correlations are known only in those who have been selected, whereas selectors need to know how measures would predict in the entire pool of applicants. That problem of interpretation can be solved by calculating construct-level predictive validity, an estimate of true predictor-outcome correlation across the range of applicant abilities.MethodsConstruct-level predictive validities were calculated in six cohort studies of medical student selection and training (student entry, 1972 to 2009) for a range of predictors, including A-levels, General Certificates of Secondary Education (GCSEs)/O-levels, and aptitude tests (AH5 and UK Clinical Aptitude Test (UKCAT)). Outcomes included undergraduate basic medical science and finals assessments, as well as postgraduate measures of Membership of the Royal Colleges of Physicians of the United Kingdom (MRCP(UK)) performance and entry in the Specialist Register. Construct-level predictive validity was calculated with the method of Hunter, Schmidt and Le (2006), adapted to correct for right-censorship of examination results due to grade inflation.ResultsMeta-regression analyzed 57 separate predictor-outcome correlations (POCs) and construct-level predictive validities (CLPVs). Mean CLPVs are substantially higher (.450) than mean POCs (.171). Mean CLPVs for first-year examinations, were high for A-levels (.809; CI: .501 to .935), and lower for GCSEs/O-levels (.332; CI: .024 to .583) and UKCAT (mean = .245; CI: .207 to .276). A-levels had higher CLPVs for all undergraduate and postgraduate assessments than did GCSEs/O-levels and intellectual aptitude tests. CLPVs of educational attainment measures decline somewhat during training, but continue to predict postgraduate performance. Intellectual aptitude tests have lower CLPVs than A-levels or GCSEs/O-levels.ConclusionsEducational attainment has strong CLPVs for undergraduate and postgraduate performance, accounting for perhaps 65% of true variance in first year performance. Such CLPVs justify the use of educational attainment measure in selection, but also raise a key theoretical question concerning the remaining 35% of variance (and measurement error, range restriction and right-censorship have been taken into account). Just as in astrophysics, 'dark matter' and 'dark energy' are posited to balance various theoretical equations, so medical student selection must also have its 'dark variance', whose nature is not yet properly characterized, but explains a third of the variation in performance during training. Some variance probably relates to factors which are unpredictable at selection, such as illness or other life events, but some is probably also associated with factors such as personality, motivation or study skills.

Dataset Information

The value of the UK Clinical Aptitude Test in predicting pre-clinical performance: a prospective cohort study at Nottingham Medical School.

Background

Methods

Results

Conclusion

Publications

The value of the UK Clinical Aptitude Test in predicting pre-clinical performance: a prospective cohort study at Nottingham Medical School.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets