Dataset Information

Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education Special Issue.

ABSTRACT: BACKGROUND: Deficiencies in medical education research quality are widely acknowledged. Content, internal structure, and criterion validity evidence support the use of the Medical Education Research Study Quality Instrument (MERSQI) to measure education research quality, but predictive validity evidence has not been explored. OBJECTIVE: To describe the quality of manuscripts submitted to the 2008 Journal of General Internal Medicine (JGIM) medical education issue and determine whether MERSQI scores predict editorial decisions. DESIGN AND PARTICIPANTS: Cross-sectional study of original, quantitative research studies submitted for publication. MEASUREMENTS: Study quality measured by MERSQI scores (possible range 5-18). RESULTS: Of 131 submitted manuscripts, 100 met inclusion criteria. The mean (SD) total MERSQI score was 9.6 (2.6), range 5-15.5. Most studies used single-group cross-sectional (54%) or pre-post designs (32%), were conducted at one institution (78%), and reported satisfaction or opinion outcomes (56%). Few (36%) reported validity evidence for evaluation instruments. A one-point increase in MERSQI score was associated with editorial decisions to send manuscripts for peer review versus reject without review (OR 1.31, 95%CI 1.07-1.61, p = 0.009) and to invite revisions after review versus reject after review (OR 1.29, 95%CI 1.05-1.58, p = 0.02). MERSQI scores predicted final acceptance versus rejection (OR 1.32; 95% CI 1.10-1.58, p = 0.003). The mean total MERSQI score of accepted manuscripts was significantly higher than rejected manuscripts (10.7 [2.5] versus 9.0 [2.4], p = 0.003). CONCLUSIONS: MERSQI scores predicted editorial decisions and identified areas of methodological strengths and weaknesses in submitted manuscripts. Researchers, reviewers, and editors might use this instrument as a measure of methodological quality.

SUBMITTER: Reed DA

PROVIDER: S-EPMC2517948 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Background In order to track progress towards the target of universal access to sexual and reproductive health care services of the post-2015 Sustainable Development Goals (SDGs), a measure (demand for family planning satisfied with modern contraceptive methods) and a benchmark (at least 75% by 2030 in all countries) have been recommended. Objectives The goal of this study was to assess the prospects of reaching the benchmark at the country level. Such information can facilitate strategic planning, including resource allocation at global and country levels. Design We selected 63 countries based on their status as least developed according to the United Nations or as a priority country in global family planning initiatives. Using United Nations estimates and projections of family planning indicators between 1970 and 2030, we calculated percent demand for family planning satisfied with modern contraceptive methods for each year and country. We then calculated the annual percentage point changes between 2014 and 2030 required to meet the benchmark. The required rates of change were compared to current projections as well as estimates between 1970 and 2010. Results To reach the benchmark on average across the 63 countries, demand satisfied with modern methods must increase by 2.2 percentage points per year between 2014 and 2030 – more than double current projections. Between 1970 and 2010, such rapid progress was observed in 24 study countries but typically spanning 5–10 years. At currently projected rates, only 9 of the 63 study countries will reach the benchmark. Meanwhile, the gap between projected and required changes is largest in the Central and West African regions, 0.9 and 3.0 percentage points per year, respectively. If the benchmark is achieved, 334 million women across the study countries will use a modern contraceptive method in 2030, compared to 226 million women in 2014. Conclusions In order to achieve the component of the SDGs calling for universal access to sexual and reproductive health services, substantial effort is needed to accelerate rates of progress by a factor of 2 in most study countries and by a factor of 3 in Central and West African countries.

Project description:BackgroundSeveral studies in the United Kingdom and Asia have suggested that medical students and residents have particular difficulty in diagnosing and managing patients with neurological problems. Little recent information is available for US trainees. We examined whether students and residents at a US university have difficulty in dealing with patients with neurological problems, identified the perceived sources of these difficulties and provide suggestions for the development of an effective educational experience in neurology.MethodsA questionnaire was administered to third and fourth year medical students at a US school of medicine and to residents of an internal medicine residency program affiliated with that school. Perceived difficulties with eight medical specialties, including neurology, were examined. Methods considered to be most useful for learning medicine were documented. Reasons why neurology is perceived as difficult and ways to improve neurological teaching were assessed.Results152 surveys were completed. Participation rates varied, with medical students having higher response rates (> 50%) than medical residents (27%-48%). Respondents felt that neurology was the medical specialty they had least knowledge in (p < 0.001) and was most difficult (p < 0.001). Trainees also felt they had the least confidence when dealing with patients with neurological complaints (p < 0.001). Residents felt more competent in neurology than students (p < 0.001). The paramount reasons for perceived difficulties with neurology were the complexity of neuroanatomy, limited patient exposure and insufficient teaching. Transition from pre-clinical to clinical medicine led to a doubling of "poor" ratings for neurological teaching. Over 80% of the respondents felt that neurology teaching could be improved through greater exposure to patients and more bedside tutorials.ConclusionsMedical students and residents at this US medical university found neurology difficult. Although this is consistent with prior reports from Europe and Asia, studies in other universities are needed to confirm generalizability of these findings. The optimal opportunity for improvement is during the transition from preclinical to clinical years. Enhanced integration of basic neurosciences and clinical neurology with emphasis on increased bedside tutorials and patient exposure should improve teaching. Studies are needed to quantify the effect of these interventions on confidence of trainees when dealing with patients presenting with neurological complaints.

Project description:BackgroundAlthough used widely, there is limited evidence of the BioMedical Admissions Test's (BMAT) predictive validity and incremental validity over prior educational attainment (PEA). We investigated BMAT's predictive and incremental validity for performance in two undergraduate medical schools, Imperial College School of Medicine (ICSM), UK, and Lee Kong Chian School of Medicine (LKCMedicine), Singapore. Our secondary goal was to compare the evidence collected with published evidence relating to comparable tools.MethodsThis was a retrospective cohort study of four ICSM (1188 students, entering 2010-2013) and three LKCMedicine cohorts (222 students, 2013-2015). We investigated associations between BMAT Section 1 ('Thinking Skills'), Section 2 ('Scientific Knowledge and Applications') and Section 3a ('Writing Task') scores, with written and clinical assessment performance across all programme years. Incremental validity was investigated over PEA (A-levels) in a subset of ICSM students.ResultsWhen BMAT sections were investigated independently, Section 2 scores predicted performance on all written assessments in both institutions with mainly small effect sizes (standardised coefficient ranges: ICSM: 0.08-0.19; LKCMedicine: 0.22-0.36). Section 1 scores predicted Years 5 and 6 written assessment performance at ICSM (0.09-0.14) but nothing at LKCMedicine. Section 3a scores only predicted Year 5 clinical assessment performance at ICSM with a coefficient <0.1. There were no positive associations with standardised coefficients >0.1 between BMAT performance and clinical assessment performance. Multivariable regressions confirmed that Section 2 scores were the most predictive. We found no clear evidence of incremental validity for any BMAT section scores over A-level grades.DiscussionSchools who wish to assess scientific knowledge independently of A-levels may find BMAT Section 2 useful. Comparison with previous studies indicates that, overall, BMAT seems less useful than comparable tools. Larger scale studies are needed. Broader questions regarding why institutions adopt certain admissions tests, including those with little evidence, need consideration.

Dataset Information

Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education Special Issue.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets