Dataset Information

Evaluating the validity of the selection measures used for the UK's foundation medical training programme: a national cohort study.

ABSTRACT:

Objectives

Currently relative performance at medical school (educational performance measure (EPM) decile), additional educational achievements and the score on a situational judgement test (SJT) are used to rank applicants to the UK Foundation Years postgraduate medical training programme. We sought to evaluate whether these three measures were predictive of subsequent successful completion of the programme, and thus were valid selection criteria.

Methods

Data were obtained from the UK Medical Education Database (UKMED) on 14 131 UK applicants to the foundation programme starting in 2013 and 2014. These data included training outcomes in the form of Annual Reviews of Competency Progression (ARCPs), which indicated whether the programme was successfully completed. The relationship between applicants' performance on the three selection measures to the odds of successful programme completion were modelled.

Results

On univariable analyses, all three measures were associated with the odds of successful completion of the programme. Converting the SJT score to deciles to compare the effect sizes suggested that one decile increase in the EPM increased the odds of completing the programme by approximately 15%, whereas the equivalent value was 8% for the SJT scores. On multivariable analyses (with all three measures included in the model), these effects were only independently and statistically significant for EPM decile (OR 1.14, 95%?CI 1.10 to 1.18, p<0.001) and SJT z-score decile (OR 1.05, 95% CI 1.01 to 1.09, p=0.02).

Conclusions

The EPM decile and SJT scores may be effective selection measures for the foundation programme. However, educational achievements does not add value to the other two measures when predicting programme completion. Thus, its usefulness in this context is less clear. Moreover, our findings suggest that the weighting for the EPM decile score, relative to SJT performance, should be increased.

SUBMITTER: Smith DT

PROVIDER: S-EPMC6082493 | biostudies-literature | 2018 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluating the validity of the selection measures used for the UK's foundation medical training programme: a national cohort study.

Smith Daniel T DT Tiffin Paul A PA

BMJ open 20180713 7

<h4>Objectives</h4>Currently relative performance at medical school (educational performance measure (EPM) decile), additional educational achievements and the score on a situational judgement test (SJT) are used to rank applicants to the UK Foundation Years postgraduate medical training programme. We sought to evaluate whether these three measures were predictive of subsequent successful completion of the programme, and thus were valid selection criteria.<h4>Methods</h4>Data were obtained from ...[more]

PMID: 30007931

Similar Datasets

Project description:ImportanceChemotherapy-induced peripheral neuropathy (CIPN) is a substantial adverse effect of anticancer treatments. As such, the assessment of CIPN remains critically important in both research and clinic settings.ObjectiveTo compare the validity of various patient-reported outcome measures (PROMs) with neurophysiological and sensory functional measures as the optimal method of CIPN assessment.Design, setting, and participantsThis cohort study evaluated participants treated with neurotoxic chemotherapy across 2 cohorts using a dual-study design. Participants commencing treatment were assessed prospectively at beginning of neurotoxic treatment, midtreatment, and at the end of treatment. Participants who completed treatment up to 5 years prior were assessed cross-sectionally and completed a single assessment time point. Participants were recruited from oncology centers in Australia from August 2015 to November 2022. Data analysis occurred from February to November 2023.ExposuresNeurotoxic cancer treatment including taxanes, platinums, vinca-alkaloids, proteasome inhibitors, and thalidomide.Main outcomes and measuresCIPN was assessed via PROMs (European Organization for Research and Treatment of Cancer Quality of Life Questionnaire [EORTC-CIPN20], Functional Assessment of Cancer Therapy/Gynecological Cancer Group Neurotoxicity Questionnaire (FACT/GOG-Ntx), and the patient-reported outcomes version of the Common Terminology Criteria for Adverse Events [PRO-CTCAE]), neurological and neurophysiological assessment (Total Neuropathy Score and sural and tibial compound nerve amplitudes), and sensory measures (Grating orientation, Von Frey monofilament, and 2-point discrimination tasks). Core measurement properties of CIPN outcome measures were evaluated. Convergent and known-groups validity was assessed cross-sectionally following treatment completion, and responsiveness was evaluated prospectively during treatment. Neurological, neurophysiological, and sensory outcome measure scores were compared between those who reported high and low levels of CIPN symptoms using linear regressions.ResultsA total of 1033 participants (median [IQR] age, 61 [50-59] years; 676 female [65.4%]) were recruited to this study, incorporating 1623 assessments. PROMs demonstrated best ability to accurately assess CIPN (convergent validity), especially the PRO-CTCAE composite score (r = 0.85; P < .001) and EORTC-CIPN20 (r = 0.79; P < .001). PROMS also demonstrated the best ability to discriminate between CIPN severity (known-groups validity) and to detect changes at onset of CIPN development (responsiveness), especially for EORTC-CIPN20 (d = 0.67; 95% CI, 0.52-0.83), FACT/GOG-Ntx (d = 0.65; 95% CI, 0.49-0.81) and the PRO-CTCAE (d = 0.83; 95% CI, 0.64-1.02). Other measures did not achieve threshold for convergent validity (α < 0.7). Neurophysiological and sensory measures did not demonstrate acceptable responsiveness. In regression models, neurological, neurophysiological, and sensory outcome measures were significantly impaired in participants who reported high levels of CIPN symptoms compared with those who reported low levels of CIPN symptoms.Conclusions and relevanceIn this cohort study of 1033 cancer patients, PROMs were the only measures to satisfy all 3 core measurement property criteria (convergent validity, known-groups validity, and responsiveness). These findings suggest that adoption of PROMs in clinical practice can equip clinicians with valuable information in assessing CIPN morbidity.

Project description:BackgroundApplying simulation-based education (SBE) into surgical curricula is challenging and exacerbated by the absence of guidance on implementation processes. Empirical studies evaluating implementation of SBE interventions focus primarily on outcomes. However, understanding the processes involved in organising, planning, and delivering SBE adds knowledge on how best to develop, implement, and sustain surgical SBE. This study used a reform of early years surgical training to explore the implementation of a new SBE programme in Scotland. It aimed to understand the processes that are involved in the relative success (or failure) when implementing surgical SBE interventions.MethodsThis qualitative case study, underpinned by social constructionism, used publicly available documents and the relevant surgical SBE literature to inform the research focus and contextualise data obtained from semi-structured interviews with core surgical trainees (n = 46), consultant surgeons (n = 25), and key leaders with roles in surgical training governance in Scotland (n = 7). Initial data coding and analysis were inductive. Secondary data analysis was then undertaken using Normalisation Process Theory (NPT). NPTs' four constructs (coherence, cognitive participation, collective action, reflexive monitoring) provided an explanatory framework for scrutinising how interventions are implemented, embedded, and integrated into practice, i.e. the "normalisation" process.ResultsDistributed leadership (individual SBE initiatives assigned to faculty but overall programme overseen by a single leader) and the quality improvement practise of iterative refinement were identified as key novel processes promoting successful normalisation of the new SBE programme. Other processes widely described in the literature were also identified: stakeholder collaboration, personal contacts/relational processes, effective communication, faculty development, effective leadership, and tight programme management. The study also identified that learners valued SBE activities in group- or team-based social environments over isolated deliberate practice.ConclusionsSBE is most effective when designed as a comprehensive programme aligned to the curriculum. Programmes incorporating both group-based and isolated SBE activities promote deliberate practice. Distributed leadership amongst faculty attracts wide engagement integral to SBE programme implementation, while iterative programme refinement through regular evaluation and action on feedback encourages integration into practice. The knowledge contributed by critically analysing SBE programme implementation processes can support development of much needed guidance in this area.

Dataset Information

Evaluating the validity of the selection measures used for the UK's foundation medical training programme: a national cohort study.

Objectives

Methods

Results

Conclusions

Publications

Evaluating the validity of the selection measures used for the UK's foundation medical training programme: a national cohort study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets