Dataset Information

Implementation of a multi-level evaluation strategy: a case study on a program for international medical graduates.

ABSTRACT: Evaluation of educational interventions is often focused on immediate and/or short-term metrics associated with knowledge and/or skills acquisition. We developed an educational intervention to support international medical graduates working in rural Victoria. We wanted an evaluation strategy that included participants' reactions and considered transfer of learning to the workplace and retention of learning. However, with participants in distributed locations and limited program resources, this was likely to prove challenging. Elsewhere, we have reported the outcomes of this evaluation. In this educational development report, we describe our evaluation strategy as a case study, its underpinning theoretical framework, the strategy, and its benefits and challenges. The strategy sought to address issues of program structure, process, and outcomes. We used a modified version of Kirkpatrick's model as a framework to map our evaluation of participants' experiences, acquisition of knowledge and skills, and their application in the workplace. The predominant benefit was that most of the evaluation instruments allowed for personalization of the program. The baseline instruments provided a broad view of participants' expectations, needs, and current perspective on their role. Immediate evaluation instruments allowed ongoing tailoring of the program to meet learning needs. Intermediate evaluations facilitated insight on the transfer of learning. The principal challenge related to the resource intensive nature of the evaluation strategy. A dedicated program administrator was required to manage data collection. Although resource-intensive, we recommend baseline, immediate, and intermediate data collection points, with multi-source feedback being especially illuminating. We believe our experiences may be valuable to faculty involved in program evaluations.

SUBMITTER: Nestel D

PROVIDER: S-EPMC3258549 | biostudies-literature | 2011

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Implementation of a multi-level evaluation strategy: a case study on a program for international medical graduates.

Nestel Debra D Regan Melanie M Vijayakumar Priyanga P Sunderji Irum I Haigh Cathy C Smith Cathy C Wright Alistair A

Journal of educational evaluation for health professions 20111217

Evaluation of educational interventions is often focused on immediate and/or short-term metrics associated with knowledge and/or skills acquisition. We developed an educational intervention to support international medical graduates working in rural Victoria. We wanted an evaluation strategy that included participants' reactions and considered transfer of learning to the workplace and retention of learning. However, with participants in distributed locations and limited program resources, this w ...[more]

PMID: 22259672

Similar Datasets

Project description:ContextMany health services and systems rely on the contribution of international medical graduates (IMGs) to the workforce. However, concern has grown around their regulation and professional practice. There is a need, in the absence of strong evidence and a robust theoretical base, for a deeper understanding of the efficacy of interventions used to support IMGs' transition to their host countries. This study seeks to explore and synthesise evidence relating to interventions developed for IMGs. It aims to provide educators and policy makers with an understanding of how interventions should be developed to support IMGs in their transition to the workplace, particularly looking to identify how and why they are effective.MethodsThe realist synthesis involved an initial systematic search of the literature for the period January 1990 to April 2015. Secondary searches were conducted throughout the review in order to inform and test the developing programme theory. The context, mechanism and outcome data were extracted from all sources meeting the inclusion criteria. Fourteen case studies were included to further aid theory refinement.ResultsSixty-two articles were identified, describing diverse interventions of varying intensity. A further 26 articles were identified through a secondary search. The findings illustrate that, alongside a developed programme, ongoing support and cultural awareness at organisational and training levels are crucial. Individual differences must also be taken into consideration. This will ensure that IMGs engage in transformative learning, increase their levels of self-efficacy and cultural health capital, and reduce feelings of stress and anxiety. These factors will have an impact on work, interactions and cultural adjustment.ConclusionsOrganisational, training and individual contexts all play a role in IMGs' adjustment during the transition process. Establishing ongoing support is critical. A list of recommendations for implementation is given.

Project description:With limited residency positions, the majority of international medical graduates living in Canada and other developed countries cannot work as physicians. The educational experience and career development of unmatched international medical graduates (those who are not matched to a residency position) residing in Canada have never been studied. Through an innovative collaboration of provincial international medical graduate organizations, we launched an online survey targeting the career development of unmatched international medical graduates, with 356 survey responses entering data analysis. Respondents reported that at the early career planning stage, close to a third had not had adequate knowledge of their career prospects in Canada. Although official resources are available, most respondents found that the information did not match well with reality. After arriving in Canada, educational resources for unmatched international medical graduates are scarce. The majority of them work in healthcare-related fields but reported significant difficulties finding these positions, and there were limited career training opportunities. Among respondents who were no longer pursuing residency positions and had moved on to alternate career paths, we found, unexpectedly, that 69% of them reported previous residency application experience did not contribute to their current occupation, and most were not satisfied with their current career status and continuing education opportunities. In conclusion, the unmatched international medical graduates could serve as a vital reservoir of skilled medical professionals to serve the community, especially during public health crises. Continuing education and career support of unmatched international medical graduates are crucial for their career development and should be an area of priority for policymakers. Career support, especially for alternative career paths, can be built on the current services that exist in most provinces in Canada. In addition, novel online and social media tools can be utilized to increase the outreach of these programs.

Dataset Information

Implementation of a multi-level evaluation strategy: a case study on a program for international medical graduates.

Publications

Implementation of a multi-level evaluation strategy: a case study on a program for international medical graduates.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets