Dataset Information

Quality of randomised controlled trials in medical education reported between 2012 and 2013: a systematic review protocol.

ABSTRACT:

Introduction

Research in medical education has increased in volume over the past decades but concerns have been raised regarding the quality of trials conducted within this field. Randomised controlled trials (RCTs) involving educational interventions that are reported in biomedical journals have been criticised for their insufficient conceptual, theoretical framework. RCTs published in journals dedicated to medical education, on the other hand, have been questioned regarding their methodological rigour. The aim of this study is therefore to assess the quality of RCTs of educational interventions reported in 2012 and 2013 in journals dedicated to medical education compared to biomedical journals with respect to objective quality criteria.

Methods and analysis

RCTs published between 1 January 2012 and 31 December 2013 in English are included. The search strategy is developed with the help of experienced librarians to search online databases for key terms. All of the identified RCTs are screened based on their titles and abstracts individually by the authors and then compared in pairs to assess agreement. Data are extracted from the included RCTs by independently scoring each RCT using a data collection form. The data collection form consists of four steps. Step 1 includes confirmation of RCT eligibility; step 2 consists of the CONSORT checklist; step 3 consists of the Medical Education Research Study Quality Instrument framework; step 4 consists of a Medical Education Extension (MEdEx) to the CONSORT checklist. The MEdEx includes the following elements: Description of scientific background, explanation of rationale, quality of research questions and hypotheses, clarity in the description of the use of the intervention and control as well as interpretation of results.

Ethics and dissemination

This review is the first to systematically examine the quality of RCTs conducted in medical education. We plan to disseminate the results through publications and presentation at relevant conferences. Ethical approval is not sought for this review.

SUBMITTER: Tolsgaard MG

PROVIDER: S-EPMC4120313 | biostudies-literature | 2014 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Quality of randomised controlled trials in medical education reported between 2012 and 2013: a systematic review protocol.

Tolsgaard Martin G MG Ku Cheryl C Woods Nicole N NN Kulasegaram Kulamakan Mahan KM Brydges Ryan R Ringsted Charlotte C

BMJ open 20140730 7

<h4>Introduction</h4>Research in medical education has increased in volume over the past decades but concerns have been raised regarding the quality of trials conducted within this field. Randomised controlled trials (RCTs) involving educational interventions that are reported in biomedical journals have been criticised for their insufficient conceptual, theoretical framework. RCTs published in journals dedicated to medical education, on the other hand, have been questioned regarding their metho ...[more]

PMID: 25079932

Similar Datasets

Project description:BACKGROUND:There will be a lack of 18 million healthcare workers by 2030. Multiplying the number of well-trained healthcare workers through innovative ways such as eLearning is highly recommended in solving this shortage. However, high heterogeneity of learning outcomes in eLearning systematic reviews reveals a lack of consistency and agreement on core learning outcomes in eLearning for medical education. In addition, there seems to be a lack of validity evidence for measurement instruments used in these trials. This undermines the credibility of these outcome measures and affects the ability to draw accurate and meaningful conclusions. The aim of this research is to address this issue by determining the choice of outcomes, measurement instruments and the prevalence of measurement instruments with validity evidence in randomised trials on eLearning for pre-registration medical education. METHODS:We will conduct a systematic mapping and review to identify the types of outcomes, the kinds of measurement instruments and the prevalence of validity evidence among measurement instruments in eLearning randomised controlled trials (RCTs) in pre-registration medical education. The search period will be from January 1990 until August 2017. We will consider studies on eLearning for health professionals' education. Two reviewers will extract and manage data independently from the included studies. Data will be analysed and synthesised according to the aim of the review. DISCUSSION:Appropriate choice of outcomes and measurement tools is essential for ensuring high-quality research in the field of eLearning and eHealth. The results of this study could have positive implications for other eHealth interventions, including (1) improving quality and credibility of eLearning research, (2) enhancing the quality of digital medical education and (3) informing researchers, academics and curriculum developers about the types of outcomes and validity evidence for measurement instruments used in eLearning studies. The protocol aspires to assist in the advancement of the eLearning research field as well as in the development of high-quality healthcare professionals' digital education. SYSTEMATIC REVIEW REGISTRATION:PROSPERO CRD42017068427.

Project description:INTRODUCTION:We aim to compare the interpretation of health news items reported with or without spin. 'Spin' is defined as a misrepresentation of study results, regardless of motive (intentionally or unintentionally) that overemphasises the beneficial effects of the intervention and overstates safety compared with that shown by the results. METHODS AND ANALYSIS:We have planned a series of 16 randomised controlled trials (RCTs) to perform a prospective meta-analysis. We will select a sample of health news items reporting the results of four types of study designs, evaluating the effect of pharmacological treatment and containing the highest amount of spin in the headline and text. News items reporting four types of studies will be included: (1) preclinical studies; (2) phase I/II (non-randomised) trials; (3) RCTs and (4) observational studies. We will rewrite the selected news items and remove the spin. The original news and rewritten news will be appraised by four types of populations: (1) French-speaking patients; (2) French-speaking general public; (3) English-speaking patients and (4) English-speaking general public. Each RCT will explore the interpretation of news items reporting one of the four study designs by each type of population and will include a sample size of 300 participants. The primary outcome will be participants' interpretation of the benefit of treatment after reading the news items: (What do you think is the probability that treatment X would be beneficial to patients? (scale, 0 (very unlikely) to 10 (very likely)).This study will evaluate the impact of spin on the interpretation of health news reporting results of studies by patients and the general public. ETHICS AND DISSEMINATION:This study has obtained ethics approval from the Institutional Review Board of the Institut national de la santé et de la recherche médicale (INSERM) (registration no: IRB00003888). The description of all the steps and the results of this prospective meta-analysis will be available online and will be disseminated as a published article. On the completion of this study, the results will be sent to all participants. PROSPERO REGISTRATION NUMBER:CRD42017058941.

Project description:IntroductionDiabetes mellitus is the most common metabolic complication of pregnancy and its prevalence worldwide is rising. The number of randomised controlled trials (RCTs) being conducted in people with diabetes is also increasing. Many studies preferentially publish findings on clinical endpoints and do not report patient-reported outcomes (PROs). In studies that do include PROs, PRO reporting is often of poor quality.MethodsWe will conduct this systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Using a combination of medical subject headings and keywords (combined using Boolean operators), we will search web-based databases (PubMed, Cochrane and EMBASE) for RCTs published in English between 2013 and 2021. Two reviewers will review titles and abstracts. We will review the full texts of any relevant abstracts and extract the following data: date of publication or recruitment period, journal of publication, country of study, multicentre or single centre, population and number of participants, type of intervention, frequency of PRO assessment and type of PRO (or PRO measurement) used. We will also record if the PRO was a primary, secondary or exploratory outcome. We will exclude reviews, observational studies, unpublished data for example, conference abstracts and trial protocols. Any published RCT that includes data on a PRO as a primary or secondary outcome will then be compared against the Consolidated Standards of Reporting Trials-Patient-Reported Outcome extension checklist, a structured and approved framework for the publication of results of PROs.Ethics and disseminationEthical approval to conduct this study was obtained from the ethics committee at Galway University Hospitals on 24 March 2021 (CA 2592). We aim to publish our findings in a peer-reviewed journal and present our findings at national and international conferences.Systematic review registrationThis systematic review was registered prospectively with the International Prospective Register of Systematic Reviews (PROSPERO). Registration number CRD42021234917.

Project description:ContextPatient-reported outcomes (PRO) data from randomised controlled trials (RCTs) are increasingly used to inform patient-centred care as well as clinical and health policy decisions.ObjectiveThe main objective of this study was to investigate the methodological quality of PRO assessment in RCTs of prostate cancer (PCa) and to estimate the likely impact of these studies on clinical decision making.Evidence acquisitionA systematic literature search of studies was undertaken on main electronic databases to retrieve articles published between January 2004 and March 2012. RCTs were evaluated on a predetermined extraction form, including (1) basic trial demographics and clinical and PRO characteristics; (2) level of PRO reporting based on the recently published recommendations by the International Society for Quality of Life Research; and (3) bias, assessed using the Cochrane Risk of Bias tool. Studies were systematically analysed to evaluate their relevance for supporting clinical decision making.Evidence synthesisSixty-five RCTs enrolling a total of 22 071 patients were evaluated, with 31 (48%) in patients with nonmetastatic disease. When a PRO difference between treatments was found, it related in most cases to symptoms only (n=29, 58%). Although the extent of missing data was generally documented (72% of RCTs), few reported details on statistical handling of this data (18%) and reasons for dropout (35%). Improvements in key methodological aspects over time were found. Thirteen (20%) RCTs were judged as likely to be robust in informing clinical decision making. Higher-quality PRO studies were generally associated with those RCTs that had higher internal validity.ConclusionsIncluding PRO in RCTs of PCa patients is critical for better evaluating the treatment effectiveness of new therapeutic approaches. Marked improvements in PRO quality reporting over time were found, and it is estimated that at least one-fifth of PRO RCTs have provided sufficient details to allow health policy makers and physicians to make critical appraisals of results.Patient summaryIn this report, we have investigated the methodological quality of PCa trials that have included a PRO assessment. We conclude that including PRO is critical to better evaluating the treatment effectiveness of new therapeutic approaches from the patient's perspective. Also, at least one-fifth of PRO RCTs in PCa have provided sufficient details to allow health policy makers and physicians to make a critical appraisal of results.

Project description:BackgroundThe "real" effect size of a medical therapy is constant over time. In contrast, the effect size reported in randomised controlled trials (RCTs) may change over time because the sum of all kinds of bias influencing the reported effectiveness is not necessarily constant. As this would affect the validity of meta-analyses, we tested the hypothesis that the reported effect size decreases over time. Furthermore, we tested three hypotheses that would explain a possible change.MethodsBecause of well established outcome measures, the lipid-lowering drugs Pravastatin and Atorvastatin (serum low-density lipoprotein cholesterol, LDL-C) and the anti-glaucoma drugs Timolol and Latanoprost (intraocular pressure, IOP) were chosen for this investigation. Studies were identified by a standardized MEDLINE search. RCTs investigating the above identified medications administered as monotherapy, and in defined dosages, were included. Publication year, baseline (= pre-treatment value in the treatment group of interest) and post intervention means, number of patients and the assignment to experimental or control group were extracted for each study.ResultsA total of 625 citations were screened; 206 met the inclusion criteria. The reported effect size of Pravastatin (change of reported effect size in five years: -3.22% LDL-C, P < .0001), Timolol (-0.56 mmHg, P < .0001) and Latanoprost (-1.78 mmHg, P = .0074) decreased over time, while there was no significant change for Atorvastatin (+0.31% LDL-C, P = .8618). Multiple regression analysis showed that baseline values were the most important influencing factor; study size or treatment group did not play a significant role.ConclusionThe effectiveness of medical therapies reported in RCTs decreases over time in three of the four investigated pharmaceuticals, caused mainly by baseline differences. We call this phenomenon "fading of reported effectiveness". Under this condition the validity of a meta-analysis may be impaired. Therefore we propose to observe this phenomenon in future meta-analyses in order to guarantee a maximum of transparency.

Dataset Information

Quality of randomised controlled trials in medical education reported between 2012 and 2013: a systematic review protocol.

Introduction

Methods and analysis

Ethics and dissemination

Publications

Quality of randomised controlled trials in medical education reported between 2012 and 2013: a systematic review protocol.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets