Dataset Information

Choice of outcomes and measurement instruments in randomised trials on eLearning in medical education: a systematic mapping review protocol.

ABSTRACT: BACKGROUND:There will be a lack of 18 million healthcare workers by 2030. Multiplying the number of well-trained healthcare workers through innovative ways such as eLearning is highly recommended in solving this shortage. However, high heterogeneity of learning outcomes in eLearning systematic reviews reveals a lack of consistency and agreement on core learning outcomes in eLearning for medical education. In addition, there seems to be a lack of validity evidence for measurement instruments used in these trials. This undermines the credibility of these outcome measures and affects the ability to draw accurate and meaningful conclusions. The aim of this research is to address this issue by determining the choice of outcomes, measurement instruments and the prevalence of measurement instruments with validity evidence in randomised trials on eLearning for pre-registration medical education. METHODS:We will conduct a systematic mapping and review to identify the types of outcomes, the kinds of measurement instruments and the prevalence of validity evidence among measurement instruments in eLearning randomised controlled trials (RCTs) in pre-registration medical education. The search period will be from January 1990 until August 2017. We will consider studies on eLearning for health professionals' education. Two reviewers will extract and manage data independently from the included studies. Data will be analysed and synthesised according to the aim of the review. DISCUSSION:Appropriate choice of outcomes and measurement tools is essential for ensuring high-quality research in the field of eLearning and eHealth. The results of this study could have positive implications for other eHealth interventions, including (1) improving quality and credibility of eLearning research, (2) enhancing the quality of digital medical education and (3) informing researchers, academics and curriculum developers about the types of outcomes and validity evidence for measurement instruments used in eLearning studies. The protocol aspires to assist in the advancement of the eLearning research field as well as in the development of high-quality healthcare professionals' digital education. SYSTEMATIC REVIEW REGISTRATION:PROSPERO CRD42017068427.

SUBMITTER: Law GC

PROVIDER: S-EPMC5960094 | biostudies-literature | 2018 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Choice of outcomes and measurement instruments in randomised trials on eLearning in medical education: a systematic mapping review protocol.

Law Gloria C GC Apfelbacher Christian C Posadzki Pawel P PP Kemp Sandra S Tudor Car Lorainne L

Systematic reviews 20180517 1

<h4>Background</h4>There will be a lack of 18 million healthcare workers by 2030. Multiplying the number of well-trained healthcare workers through innovative ways such as eLearning is highly recommended in solving this shortage. However, high heterogeneity of learning outcomes in eLearning systematic reviews reveals a lack of consistency and agreement on core learning outcomes in eLearning for medical education. In addition, there seems to be a lack of validity evidence for measurement instrume ...[more]

PMID: 29776434

Similar Datasets

Project description:BackgroundExtended reality, which encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR), is increasingly used in medical education. Studies assessing the effectiveness of these new educational modalities should measure relevant outcomes using outcome measurement tools with validity evidence.ObjectiveOur aim is to determine the choice of outcomes, measurement instruments, and the use of measurement instruments with validity evidence in randomized controlled trials (RCTs) on the effectiveness of VR, AR, and MR in medical student education.MethodsWe conducted a systematic mapping review. We searched 7 major bibliographic databases from January 1990 to April 2020, and 2 reviewers screened the citations and extracted data independently from the included studies. We report our findings in line with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.ResultsOf the 126 retrieved RCTs, 115 (91.3%) were on VR and 11 (8.7%) were on AR. No RCT on MR in medical student education was found. Of the 115 studies on VR, 64 (55.6%) were on VR simulators, 30 (26.1%) on screen-based VR, 9 (7.8%) on VR patient simulations, and 12 (10.4%) on VR serious games. Most studies reported only a single outcome and immediate postintervention assessment data. Skills outcome was the most common outcome reported in studies on VR simulators (97%), VR patient simulations (100%), and AR (73%). Knowledge was the most common outcome reported in studies on screen-based VR (80%) and VR serious games (58%). Less common outcomes included participants' attitudes, satisfaction, cognitive or mental load, learning efficacy, engagement or self-efficacy beliefs, emotional state, competency developed, and patient outcomes. At least one form of validity evidence was found in approximately half of the studies on VR simulators (55%), VR patient simulations (56%), VR serious games (58%), and AR (55%) and in a quarter of the studies on screen-based VR (27%). Most studies used assessment methods that were implemented in a nondigital format, such as paper-based written exercises or in-person assessments where examiners observed performance (72%).ConclusionsRCTs on VR and AR in medical education report a restricted range of outcomes, mostly skills and knowledge. The studies largely report immediate postintervention outcome data and use assessment methods that are in a nondigital format. Future RCTs should include a broader set of outcomes, report on the validity evidence of the measurement instruments used, and explore the use of assessments that are implemented digitally.

Project description:IntroductionResearch in medical education has increased in volume over the past decades but concerns have been raised regarding the quality of trials conducted within this field. Randomised controlled trials (RCTs) involving educational interventions that are reported in biomedical journals have been criticised for their insufficient conceptual, theoretical framework. RCTs published in journals dedicated to medical education, on the other hand, have been questioned regarding their methodological rigour. The aim of this study is therefore to assess the quality of RCTs of educational interventions reported in 2012 and 2013 in journals dedicated to medical education compared to biomedical journals with respect to objective quality criteria.Methods and analysisRCTs published between 1 January 2012 and 31 December 2013 in English are included. The search strategy is developed with the help of experienced librarians to search online databases for key terms. All of the identified RCTs are screened based on their titles and abstracts individually by the authors and then compared in pairs to assess agreement. Data are extracted from the included RCTs by independently scoring each RCT using a data collection form. The data collection form consists of four steps. Step 1 includes confirmation of RCT eligibility; step 2 consists of the CONSORT checklist; step 3 consists of the Medical Education Research Study Quality Instrument framework; step 4 consists of a Medical Education Extension (MEdEx) to the CONSORT checklist. The MEdEx includes the following elements: Description of scientific background, explanation of rationale, quality of research questions and hypotheses, clarity in the description of the use of the intervention and control as well as interpretation of results.Ethics and disseminationThis review is the first to systematically examine the quality of RCTs conducted in medical education. We plan to disseminate the results through publications and presentation at relevant conferences. Ethical approval is not sought for this review.

Project description:BackgroundOver the last three decades, various instruments were developed and employed to assess medical professionalism, but their measurement properties have yet to be fully evaluated. This study aimed to systematically evaluate these instruments' measurement properties and the methodological quality of their related studies within a universally acceptable standardized framework and then provide corresponding recommendations.MethodsA systematic search of the electronic databases PubMed, Web of Science, and PsycINFO was conducted to collect studies published from 1990-2015. After screening titles, abstracts, and full texts for eligibility, the articles included in this study were classified according to their respective instrument's usage. A two-phase assessment was conducted: 1) methodological quality was assessed by following the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist; and 2) the quality of measurement properties was assessed according to Terwee's criteria. Results were integrated using best-evidence synthesis to look for recommendable instruments.ResultsAfter screening 2,959 records, 74 instruments from 80 existing studies were included. The overall methodological quality of these studies was unsatisfactory, with reasons including but not limited to unknown missing data, inadequate sample sizes, and vague hypotheses. Content validity, cross-cultural validity, and criterion validity were either unreported or negative ratings in most studies. Based on best-evidence synthesis, three instruments were recommended: Hisar's instrument for nursing students, Nurse Practitioners' Roles and Competencies Scale, and Perceived Faculty Competency Inventory.ConclusionAlthough instruments measuring medical professionalism are diverse, only a limited number of studies were methodologically sound. Future studies should give priority to systematically improving the performance of existing instruments and to longitudinal studies.

Project description:IntroductionFrailty is associated with reduced functional capacity, decreased resistance to stressors and is predictive of a range of adverse health outcomes, including dependency, hospitalisation and mortality. Early identification of frailty may prevent, reduce and postpone adverse health outcomes. However, there is a need for additional evidence to guide decision-making for the care of frail patients since frail persons are frequently excluded from studies, the differential impact of frailty is often not examined in clinical trials and few large-scale clinical trials examining frail cohorts have been conducted. Randomised control trials (RCTs) published to date have used a diverse range of definitions of frailty, as well as a variety of outcome measures. The objective of this systematic review is to comprehensively characterise the frail populations enrolled and the end points reported in frailty RCTs.Methods and analysisWe will identify all RCTs reporting on the outcome of interventions in adult (age ≥18 years) frail populations as defined by authors, in all settings of care. Databases will include MEDLINE, CINAHL, EMBASE, PsycInfo, Global Health, the Joanna Briggs database and Cochrane Library. Two reviewers will independently determine trial eligibility. For each included trial, we will conduct duplicate independent data extraction, inter-rater reliability, risk of bias assessment and evaluation of the quality of the evidence using the Grading of Recommendations, Assessment, Development and Evaluations approach.Ethics and disseminationThis systematic review will comprehensively identify RCTs including frail patients to identify how frailty is measured and which outcomes are reported. The results of this systematic review may inform clinicians caring for persons with frailty, facilitate conduct of future RCTs and inform future efforts to develop common data elements and core outcomes for frailty studies. Our findings will be disseminated through conference presentation and publication in peer-reviewed journals.Prospero registration numberCRD42017065233.

Project description:BACKGROUND:Physical resilience is the ability to optimize or recover motor function in the face of disease, injury, or aging-related decline. Greater knowledge of how some individuals regain or maintain function despite pathology may help identify protective factors and approaches that promote healthy aging. To date, a scoping review on physical resilience has not been conducted. The aims are to (1) identify measurement instruments for physical resilience, (2) synthesize and map the key concepts of physical resilience, and (3) identify gaps and make recommendations for future research. METHODS:A scoping review of Scopus, Web of Science, Cumulative Index to Nursing and Allied Health Literature, Medline Ovid, PsycINFO, and AgeLine databases will take place using the search strategy "resilience" AND (aging OR elderly OR older adult). The initial electronic search will be supplemented by hand searching the reference lists and review articles to identify any missing studies. Two parallel independent assessments of study eligibility will be conducted for the title, abstract, and full-text screens. To meet study inclusion criteria, the term "resilience" must be applied in relation to the physical health of older adults. Any disagreement will be resolved by consensus and a third reviewer consulted to make a decision if consensus is not achieved initially. Physical resilience information to be extracted are measurement instruments that describe the core domains of (1) body function or structure (signs or symptoms, etc.), (2) activity and participation (quality of life, etc.), and (3) societal impact. Tables and/or charts will map the data with distribution of studies by core domains. Finally, the amalgamation of results will be an iterative process whereby reviewers will refine the plan for presenting results after data extraction is completed so that all of the contents of the extraction may be included in the results. DISCUSSION:The information gleaned in this scoping review will be essential to understand how physical resilience is currently measured and identify gaps for further research.

Project description:BackgroundWhiplash is a common traumatic cervical injury which is most often a consequence of rear-end motor vehicle accidents. It has been estimated that up to 50% of the whiplash patients suffer from chronic symptoms, resulting in extensive individual and societal burden. Several measurement instruments are used for initial assessment of whiplash and evaluation of response to treatment. However, a comprehensive assessment of the performance of these measures is lacking. Furthermore, there is no consensus on the most relevant outcome domains and their corresponding measurement instruments of choice. This systematic review aims to identify, describe, and critically appraise the performance properties of health-related measurement instruments in whiplash population.MethodsThe following literature databases will be searched from their date of establishment: PubMed, Embase®, MEDLINE, CINAHL Complete, PsycINFO, and HAPI. All original articles evaluating the reliability, validity, responsiveness, and feasibility of health-related measurement instruments in whiplash will be included, without additional restriction on their intended use, source of data, and structure. Risk of bias will be assessed using the COSMIN Risk of Bias checklist. Findings of the studies will be judged against the criteria for good measurement properties, and results from all studies will be qualitatively summarized to generate an overall quality of findings. Overall quality of evidence will be determined using a modified GRADE approach, which will be used in conjunction with the overall quality of results for generation of recommendations. Two reviewers will perform all steps of the review independently. Discrepancies will be discussed between the reviewers, and in case of remaining disagreement, the senior reviewer will make the final decision.DiscussionThis systematic review will summarize the body of literature on health-related measurement instruments in whiplash, aiming to facilitate the selection of high-quality measurement instrument for researchers and physicians. Findings of this study will guide the ongoing efforts for development of a core outcome set.Systematic review registrationPROSPERO reference number CRD42018070901.

Project description:IntroductionThe prevention and relief of suffering are regarded as a goal at the end of life; therefore, suffering assessment at the end of life is essential. In this regard, we need instruments that allow us to evaluate this construct for gathering more evidence, as the assessment of suffering is increasingly used in research and the clinical setting. Many measures have been designed to assess this construct, and the selection of the most appropriate instrument is crucial. The aims of this systematic review are to (1) identify the measures assessing suffering in patients with advanced disease and their psychometric properties and (2) evaluate the methodological quality of studies on measurement properties.Methods and analysisThe protocol of this systematic review was developed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocols Guidelines. A systematic psychometric review of measures assessing suffering in patients with advanced disease and their psychometric properties will be carried out according to the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN). The search strategy will be performed following the Peer Review of Electronic Search Strategies. Searches will be conducted in Cumulative Index to Nursing and Allied Health Literature, Medline, PsycINFO, Cochrane Library, SciELO, Open Grey, Scopus, Web of Science and COSMIN database of systematic reviews, and it will be limited by time (1980-2018) and language (only literature in English and Spanish). Literature will be evaluated by two independent reviewers according to the COSMIN checklist, and measurement properties data of each study that meet the inclusion criteria will be scored independently by two researchers according to COSMIN quality ratings.Ethics and disseminationEthical approval is not necessary for systematic review protocols. The results will be disseminated by publication in a peer-reviewed journal and presented at a relevant conference.Prospero registration numberCRD42018106488.

Project description:BackgroundCoaching is emerging as a novel approach to guide medical students toward becoming competent, reflective physicians and master adaptive learners. However, no instruments currently exist to measure academic coaching at the undergraduate medical education level.ObjectiveTo describe the development and psychometric assessments of two instruments designed to assess academic coaching of medical students toward creating a robust measurement model of this educational paradigm.DesignObservational psychometric.ParticipantsAll medical students in the 2014 and 2015 cohorts and all their coaches were invited to complete the instruments being tested, which led to 662 medical student responses from 292 medical students and 468 coaching responses from 22 coaches being included in analyses. Medical student response rates were 75.7% from 2014 and 75.5% from 2015. Overall, the coaches' response rate was 71%.Main measuresTwo 31-item instruments were initially developed, one for medical students to assess their coach and one for faculty coaches to assess their students, both of which evaluated coaching based on definitions we formulated using existing literature. Each was administered to two cohorts of medical students and coaches in 2015 and 2016. An exploratory factor analysis using principal component analysis as the extraction method and Varimax with Kaiser normalization as the rotation method was conducted.Key resultsEighteen items reflecting four domains were retained with eigenvalues higher than 1.0 for medical students' assessment of coaching, which measured promoting self-monitoring, relationship building, promoting reflective behavior, and establishing foundational ground rules. Sixteen items reflecting two domains were retained for the faculty assessment of coaching with eigenvalues higher than 1.0: the Practice of Coaching and Relationship Formation.ConclusionsWe successfully developed and psychometrically validated surveys designed to measure key aspects of the coaching relationship, coaching processes, and reflective outcomes. The new validated instruments offer a robust measurement model for academic coaching.

Dataset Information

Choice of outcomes and measurement instruments in randomised trials on eLearning in medical education: a systematic mapping review protocol.

Publications

Choice of outcomes and measurement instruments in randomised trials on eLearning in medical education: a systematic mapping review protocol.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets