Dataset Information

How We Evaluate Postgraduate Medical E-Learning: Systematic Review.

ABSTRACT:

Background

Electronic learning (e-learning) in postgraduate medical education has seen a rapid evolution; however, we tend to evaluate it only on its primary outcome or learning aim, whereas its effectiveness also depends on its instructional design. We believe it is important to have an overview of all the methods currently used to evaluate e-learning design so that the preferred method may be identified and the next steps needed to continue to evaluate postgraduate medical e-learning may be outlined.

Objective

This study aimed to identify and compare the outcomes and methods used to evaluate postgraduate medical e-learning.

Methods

We performed a systematic literature review using the Web of Science, PubMed, Education Resources Information Center, and Cumulative Index of Nursing and Allied Health Literature databases. Studies that used postgraduates as participants and evaluated any form of e-learning were included. Studies without any evaluation outcome (eg, just a description of e-learning) were excluded.

Results

The initial search identified 5973 articles, of which we used 418 for our analysis. The types of studies were trials, prospective cohorts, case reports, and reviews. The primary outcomes of the included studies were knowledge, skills, and attitude. A total of 12 instruments were used to evaluate a specific primary outcome, such as laparoscopic skills or stress related to training. The secondary outcomes mainly evaluated satisfaction, motivation, efficiency, and usefulness. We found 13 e-learning design methods across 19 studies (4% 19/418). The methods evaluated usability, motivational characteristics, and the use of learning styles or were based on instructional design theories, such as Gagne's instructional design, the Heidelberg inventory, Kern's curriculum development steps, and a scale based on the cognitive load theory. Finally, 2 instruments attempted to evaluate several aspects of a design, based on the experience of creating e-learning.

Conclusions

Evaluating the effect of e-learning design is complicated. Given the diversity of e-learning methods, there are many ways to carry out such an evaluation, and probably, many ways to do so correctly. However, the current literature shows us that we have yet to reach any form of consensus about which indicators to evaluate. There is a great need for an evaluation tool that is properly constructed, validated, and tested. This could be a more homogeneous way to compare the effects of e-learning and for the authors of e-learning to continue to improve their product.

SUBMITTER: de Leeuw R

PROVIDER: S-EPMC6473211 | biostudies-literature | 2019 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

How We Evaluate Postgraduate Medical E-Learning: Systematic Review.

de Leeuw Robert R de Soet Anneloes A van der Horst Sabine S Walsh Kieran K Westerman Michiel M Scheele Fedde F

JMIR medical education 20190405 1

<h4>Background</h4>Electronic learning (e-learning) in postgraduate medical education has seen a rapid evolution; however, we tend to evaluate it only on its primary outcome or learning aim, whereas its effectiveness also depends on its instructional design. We believe it is important to have an overview of all the methods currently used to evaluate e-learning design so that the preferred method may be identified and the next steps needed to continue to evaluate postgraduate medical e-learning m ...[more]

PMID: 30950805

Similar Datasets

Project description:BackgroundResidents need to be trained across the boundaries of their own specialty to prepare them for collaborative practice. Intraprofessional learning (i.e. between individuals of different disciplines within the same profession) has received little attention in the postgraduate medical education literature, in contrast to the extensive literature on interprofessional learning between individuals of different professions. To address this gap, we performed a scoping review to investigate what and how residents learn from workplace-related intraprofessional activities, and what factors influence learning.MethodsThe PRISMA guidelines were used to conduct a scoping review of empirical studies on intraprofessional workplace learning in postgraduate medical education published between 1 January 2000 to 16 April 2020 in Pubmed, Embase, PsycINFO, ERIC and Web of Science. This study applied 'best fit' framework-based synthesis to map the existing evidence, using the presage-process-product (3P) model developed by Tynjälä (2013).ResultsFour thousand three hundred thirty records were screened, and 37 articles were included. This review identified influencing (presage) factors that derived from the sociocultural environment, learner and learning context. Studies described that complexity of care can both facilitate and hinder learning. Furthermore, intraprofessional learning is threatened by professional stereotyping and negative perceptions, and awareness of learning opportunities and explicit reflection are critical in intraprofessional workplace learning. Studies described a range of informal and formal intraprofessional activities (process) under the headings of collaboration in clinical practice, rotations or placements, formal educational sessions and simulated workplace training. In general, learners responded well and their attitudes and perceptions improved, learners reported increased knowledge and skills and positive behavioural changes (product). Learning outcomes were reported in the domains of patient-centred care, collaborative attitudes and respect, mutual knowledge and understanding, collaborative decision making, communication, leadership, teamwork and reflexivity.ConclusionsThis review gives insight into the high learning potential of intraprofessional activities. Many of the included studies relied on self-reported perceptions of change, therefore, future research should focus on generating more robust evidence including objectively examined outcome measures. This review offers a comprehensive overview of the factors that influence intraprofessional workplace learning in postgraduate medical education. Finally, we provide recommendations for enhancing intraprofessional learning in clinical practice.

Project description:Objective: In order to evaluate the quality of postgraduate medical education in Germany, we examined how the learning of theoretical and practical competencies is conceptualized and how the learning process takes place in real terms. The training conditions, as perceived by medical residents, are compared with the learning objectives, as stated by the Federal Chamber of Physicians in its regulation on postgraduate education. The analysis is based on the data of the "KarMed" study (abbreviation of "career progression and career breaks among physicians during postgraduate education"), a multicentre cohort study of medical school graduates from seven universities who arre followed until they receive their licenses as specialist physicians. The study was conducted by the Institute of Primary Medical Care of the University Medical Center Hamburg-Eppendorf starting in 2008. Methodology: The KarMed study is based on annual, standardized surveys of the population described above. 48% responded in the initial survey (n=1012) and 85% or more in each of the following surveys. Descriptive statistics and logistical regressions were used for analysis. Gender-specific analyses were performed where possible in order to highlight the differences in the professional objectives and workloads between female and male physicians. Results: The study shows that both the practical and the theoretical components of postgraduate medical education in Germany are insufficient. There is a lack of a curriculum with precise learning objectives and descriptions of the corresponding educational settings. In fact, the act of learning is identical to daily clinical work. There is no structuring of the work process with regard to learning; for example, documentation procedures and feedback discussions are prescribed but largely omitted. Evidence-based medicine is not a systematic part of the training, nor is the evaluation of residents' progress. The summative final oral examination pretends that the necessary specialist competencies can be evaluated within 30 minutes. Many factors indicate that female doctors, especially those with children, have fewer learning opportunities than male doctors. Conclusion: The quality of postgraduate medical education in Germany has become inadequate, especially in an international comparison. The deficits are well known. The responsible institutions are called upon to implement sustainable reforms in the sense of postgraduate education that is structured according to educational principles and whose quality is assured.

Project description:BackgroundPoor communication skills can potentially compromise patient care. However, as communication skills training (CST) programs are not seen as a priority to many clinical departments, there is a discernible absence of a standardised, recommended framework for these programs to be built upon. This systematic scoping review (SSR) aims to gather prevailing data on existing CSTs to identify key factors in teaching and assessing communication skills in the postgraduate medical setting.MethodsIndependent searches across seven bibliographic databases (PubMed, PsycINFO, EMBASE, ERIC, CINAHL, Scopus and Google Scholar) were carried out. Krishna's Systematic Evidence-Based Approach (SEBA) was used to guide concurrent thematic and content analysis of the data. The themes and categories identified were compared and combined where possible in keeping with this approach and then compared with the tabulated summaries of the included articles.ResultsTwenty-five thousand eight hundred ninety-four abstracts were identified, and 151 articles were included and analysed. The Split Approach revealed similar categories and themes: curriculum design, teaching methods, curriculum content, assessment methods, integration into curriculum, and facilitators and barriers to CST. Amidst a wide variety of curricula designs, efforts to develop the requisite knowledge, skills and attitudes set out by the ACGME current teaching and assessment methods in CST maybe categorised into didactic and interactive methods and assessed along Kirkpatrick's Four Levels of Learning Evaluation.ConclusionsA major flaw in existing CSTs is a lack of curriculum structure, focus and standardisation. Based upon the findings and current design principles identified in this SSR in SEBA, we forward a stepwise approach to designing CST programs. These involve 1) defining goals and learning objectives, 2) identifying target population and ideal characteristics, 3) determining curriculum structure, 4) ensuring adequate resources and mitigating barriers, 5) determining curriculum content, and 6) assessing learners and adopting quality improvement processes.

Project description:BackgroundThe stressful nature of medical training and other work-related factors put postgraduate medical trainees at high risk of burnout and poor psychological wellbeing. This has negative implications for patient care and the effectiveness of the healthcare system. The structure of the healthcare system and postgraduate medical education in Australia is different to that of other countries. Whilst a significant body of research exists on burnout and wellbeing in trainees in the USA, evidence specific to Australian trainees is lacking. The aim of this review is to synthesise the current knowledge on the factors that impact burnout and psychological wellbeing in Australian postgraduate medical trainees.Methods/designA systematic review will be conducted across eight digital databases: Academic Search Complete, MEDLINE, Embase, Web of Science, PsychInfo, Scopus, CINAHL Plus and Informit Health Collection. Peer reviewed empirical studies and relevant grey literature published after 2000 that address an aspect of burnout or psychological wellbeing in Australian postgraduate medical trainees will be included. Two reviewers will independently review each article against the inclusion and exclusion criteria, with disagreements resolved via discussion and consensus. Data will be extracted using a standard form and quality will be assessed using the assessment tools available from the Joanna Briggs Institute. A thematic narrative synthesis of the studies will be presented, along with an assessment of current gaps in the literature and areas for future research.DiscussionThis review will be the first to integrate the evidence on burnout and psychological wellbeing specific to Australian postgraduate medical trainees. The findings will contribute to a better understanding of the factors that impact burnout and psychological wellbeing in this population and will lay the foundation for future research into appropriate strategic interventions.Systematic review registrationThis protocol has been registered in the International Prospective Register of Systematic Reviews (PROSPERO: CRD42020203195 ).

Project description:BackgroundMolding competent clinicians capable of applying ethics principles in their practice is a challenging task, compounded by wide variations in the teaching and assessment of ethics in the postgraduate setting. Despite these differences, ethics training programs should recognise that the transition from medical students to healthcare professionals entails a longitudinal process where ethics knowledge, skills and identity continue to build and deepen over time with clinical exposure. A systematic scoping review is proposed to analyse current postgraduate medical ethics training and assessment programs in peer-reviewed literature to guide the development of a local physician training curriculum.MethodsWith a constructivist perspective and relativist lens, this systematic scoping review on postgraduate medical ethics training and assessment will adopt the Systematic Evidence Based Approach (SEBA) to create a transparent and reproducible review.ResultsThe first search involving the teaching of ethics yielded 7669 abstracts with 573 full text articles evaluated and 66 articles included. The second search involving the assessment of ethics identified 9919 abstracts with 333 full text articles reviewed and 29 articles included. The themes identified from the two searches were the goals and objectives, content, pedagogy, enabling and limiting factors of teaching ethics and assessment modalities used. Despite inherent disparities in ethics training programs, they provide a platform for learners to apply knowledge, translating it to skill and eventually becoming part of the identity of the learner. Illustrating the longitudinal nature of ethics training, the spiral curriculum seamlessly integrates and fortifies prevailing ethical knowledge acquired in medical school with the layering of new specialty, clinical and research specific content in professional practice. Various assessment methods are employed with special mention of portfolios as a longitudinal assessment modality that showcase the impact of ethics training on the development of professional identity formation (PIF).ConclusionsOur systematic scoping review has elicited key learning points in the teaching and assessment of ethics in the postgraduate setting. However, more research needs to be done on establishing Entrustable Professional Activities (EPA)s in ethics, with further exploration of the use of portfolios and key factors influencing its design, implementation and assessment of PIF and micro-credentialling in ethics practice.

Project description:BACKGROUND:Postgraduate medical e-learning (PGMeL) is being progressively used and evaluated. Its impact continues to grow, yet there are barriers to its implementation. Although more attention is now being paid to quality evaluation models, little has been written about the successful implementation of PGMeL. This study aims to determine factors and define themes influencing the successful implementation of PGMeL. METHODS:We performed 10 semi-structured interviews with experienced e-learning creators, after which we carried out a thematic analysis to name and describe factors and themes. RESULTS:Although this was not the objective of the study, the participants stressed the importance of a definition of success. Associated with this definition were: reaching your target audience, achieving learning aims, satisfying your audience and maintaining continuity. Three themes were identified containing eleven factors that influence successful implementation. The themes were named and defined after the group that had the most influence on the factors. We named them creator-, organization- and learner-dependent factors. The creator dependent factors are: the learning aim, pedagogical strategies, content expertise, evaluation and the creators motivational path. The organization dependent factors are management support, recourse and culture. Finally, the learner dependent factors are technology, motivators/barriers and value. CONCLUSIONS:This study shows that implementing PGMeL has creator-, organization- and learner-dependent factors which should be taken into account during the creating of the PGMeL. Although creator- and learner-dependent factors are mentioned in other studies, the present study also stresses the importance of organization-dependent factors. Innovation implementation theories such as Rogers' diffusion of innovation or Kotter's eight steps of change management show a great overlap with these factors. Future studies can both evaluate the use of these innovation models in creating PGMeL and assess the effect of the organizational factors in greater depth.

Project description:BackgroundBlended learning, which combines face-to-face learning and e-learning, has grown rapidly to be commonly used in education. Nevertheless, the effectiveness of this learning approach has not been completely quantitatively synthesized and evaluated using knowledge outcomes in health education.ObjectiveThe aim of this study was to assess the effectiveness of blended learning compared to that of traditional learning in health education.MethodsWe performed a systematic review of blended learning in health education in MEDLINE from January 1990 to July 2019. We independently selected studies, extracted data, assessed risk of bias, and compared overall blended learning versus traditional learning, offline blended learning versus traditional learning, online blended learning versus traditional learning, digital blended learning versus traditional learning, computer-aided instruction blended learning versus traditional learning, and virtual patient blended learning versus traditional learning. All pooled analyses were based on random-effect models, and the I2 statistic was used to quantify heterogeneity across studies.ResultsA total of 56 studies (N=9943 participants) assessing several types of learning support in blended learning met our inclusion criteria; 3 studies investigated offline support, 7 studies investigated digital support, 34 studies investigated online support, 8 studies investigated computer-assisted instruction support, and 5 studies used virtual patient support for blended learning. The pooled analysis comparing all blended learning to traditional learning showed significantly better knowledge outcomes for blended learning (standardized mean difference 1.07, 95% CI 0.85 to 1.28, I2=94.3%). Similar results were observed for online (standardized mean difference 0.73, 95% CI 0.60 to 0.86, I2=94.9%), computer-assisted instruction (standardized mean difference 1.13, 95% CI 0.47 to 1.79, I2=78.0%), and virtual patient (standardized mean difference 0.62, 95% CI 0.18 to 1.06, I2=78.4%) learning support, but results for offline learning support (standardized mean difference 0.08, 95% CI -0.63 to 0.79, I2=87.9%) and digital learning support (standardized mean difference 0.04, 95% CI -0.45 to 0.52, I2=93.4%) were not significant.ConclusionsFrom this review, blended learning demonstrated consistently better effects on knowledge outcomes when compared with traditional learning in health education. Further studies are needed to confirm these results and to explore the utility of different design variants of blended learning.

Dataset Information

How We Evaluate Postgraduate Medical E-Learning: Systematic Review.

Background

Objective

Methods

Results

Conclusions

Publications

How We Evaluate Postgraduate Medical E-Learning: Systematic Review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets