Dataset Information

Characterizing the literature on validity and assessment in medical education: a bibliometric study.

ABSTRACT: INTRODUCTION:Assessment in Medical Education fills many roles and is under constant scrutiny. Assessments must be of good quality, and supported by validity evidence. Given the high-stakes consequences of assessment, and the many audiences within medical education (e.?g., training level, specialty-specific), we set out to document the breadth, scope, and characteristics of the literature reporting on validation of assessments within medical education. METHOD:Searches in Medline (Ovid), Web of Science, ERIC, EMBASE (Ovid), and PsycINFO (Ovid) identified articles reporting on assessment of learners in medical education published since 1999. Included articles were coded for geographic origin, journal, journal category, targeted assessment, and authors. A map of collaborations between prolific authors was generated. RESULTS:A total of 2,863 articles were included. The majority of articles were from the United States, with Canada producing the most articles per medical school. Most articles were published in journals with medical categorizations (73.1% of articles), but Medical Education was the most represented journal (7.4% of articles). Articles reported on a variety of assessment tools and approaches, and 89 prolific authors were identified, with a total of 228 collaborative links. DISCUSSION:Literature reporting on validation of assessments in medical education is heterogeneous. Literature is produced by a broad array of authors and collaborative networks, reported to a broad audience, and is primarily generated in North American and European contexts. Our findings speak to the heterogeneity of the medical education literature on assessment validation, and suggest that this heterogeneity may stem, at least in part, from differences in constructs measured, assessment purposes, or conceptualizations of validity.

SUBMITTER: Young M

PROVIDER: S-EPMC6002290 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Characterizing the literature on validity and assessment in medical education: a bibliometric study.

Young Meredith M St-Onge Christina C Xiao Jing J Vachon Lachiver Elise E Torabi Nazi N

Perspectives on medical education 20180601 3

<h4>Introduction</h4>Assessment in Medical Education fills many roles and is under constant scrutiny. Assessments must be of good quality, and supported by validity evidence. Given the high-stakes consequences of assessment, and the many audiences within medical education (e. g., training level, specialty-specific), we set out to document the breadth, scope, and characteristics of the literature reporting on validation of assessments within medical education.<h4>Method</h4>Searches in Medline (O ...[more]

PMID: 29796976

Similar Datasets

Project description:PurposeThis bibliometric analysis maps the landscape of knowledge syntheses in medical education. It provides scholars with a roadmap for understanding where the field has been and where it might go in the future, thereby informing research and educational practice. In particular, this analysis details the venues in which knowledge syntheses are published, the types of syntheses conducted, citation rates they produce, and altmetric attention they garner.MethodIn 2020, the authors conducted a bibliometric analysis of knowledge syntheses published in 14 core medical education journals from 1999 to 2019. To characterize the studies, metadata were extracted from PubMed, Web of Science, Altmetrics Explorer, and Unpaywall.ResultsThe authors analyzed 963 knowledge syntheses representing 3.1% of the total articles published (n = 30,597). On average, 45.9 knowledge syntheses were published annually (SD = 35.85, median = 33), and there was an overall 2620% increase in the number of knowledge syntheses published from 1999 to 2019. The journals each published, on average, a total of 68.8 knowledge syntheses (SD = 67.2, median = 41) with Medical Education publishing the most (n = 189; 19%). Twenty-one types of knowledge synthesis were identified, the most prevalent being systematic reviews (n = 341; 35.4%) and scoping reviews (n = 88; 9.1%). Knowledge syntheses were cited an average of 53.80 times (SD = 107.12, median = 19) and received a mean Altmetric Attention Score of 14.12 (SD = 37.59, median = 6).ConclusionsThere has been considerable growth in knowledge syntheses in medical education over the past 20 years, contributing to medical education's evidence base. Beyond this increase in volume, researchers have introduced methodological diversity in these publications, and the community has taken to social media to share knowledge syntheses. Implications for the field, including the impact of synthesis types and their relationship to knowledge translation, are discussed.

Project description:BackgroundThe field of medical education remains poorly delineated such that there is no broad consensus of articles or journals that comprise 'the field'. This lack of consensus indicates a missed opportunity for researchers to generate insights about the field that could facilitate conducting bibliometric studies and other research designs (e.g., systematic reviews) and also enable individuals to identify themselves as 'medical education researchers'. Other fields have utilised bibliometric field delineation, which is the assigning of articles or journals to a certain field in an effort to define that field.ProcessIn this Research Approach, three bibliometric field delineation approaches-information retrieval, core journals, and journal co-citation-are introduced. For each approach, the authors describe attempts to apply it in medical education and identify related strengths and weaknesses. Based on co-citation, the authors propose the Medical Education Journal List 24 (MEJ-24), as a starting point for delineating medical education and invite the community to collaborate on improving and potentially expanding this list.PearlsAs a research approach, field delineation is complicated, and there is no clear best way to delineate the field of medical education. However, recent advances in information science provide potentially fruitful approaches to deal with the field's complexity. When considering these approaches, researchers should consider collaborating with bibliometricians. Bibliometric approaches rely on available metadata for articles and journals, which necessitates that researchers examine the metadata prior to analysis to understand its strengths and weaknesses, and to assess how this might affect data interpretation. While using bibliometric approaches for field delineation is valuable, it is important to remember that these techniques are only as good as the research team's interpretation of the data, which suggests that an expanded approach is needed to better delineate medical education, an approach that includes active discussion within the medical education community.

Project description:BackgroundThere exists a lack of knowledge regarding the quantity and quality of scientific yield in relation to individual cancer types. We aimed to measure the proportion, quality and relevance of oncology-related articles, and to relate this output to their associated disease burden. By incorporating the impact factor(IF) and Eigenfactor™(EF) into our analysis we also assessed the relationship between these indices and the output under study.MethodsAll publications in 2007 were retrieved for the 26 most common cancers. The top 20 journals ranked by IF and EF in general medicine and oncology, and the presence of each malignancy within these titles was analysed. Journals publishing most prolifically on each cancer were identified and their impact assessed.Principal findings63260 (PubMed) and 126845 (WoS) entries were generated, respectively. 26 neoplasms accounted for 25% of total output from the top medical publications. 5 cancers dominated the first quartile of output in the top oncology journals; breast, prostate, lung, and intestinal cancer, and leukaemia. Journals associated with these cancers were associated with much higher IFs and EFs than those journals associated with the other cancer types under study, although these measures were not equivalent across all sub-specialties. In addition, yield on each cancer was related to its disease burden as measured by its incidence and prevalence.ConclusionsOncology enjoys disproportionate representation in the more prestigious medical journals. 5 cancers dominate yield, although this attention is justified given their associated disease burden. The commonly used IF and the recently introduced EF do not correlate in the assessment of the preeminent oncology journals, nor at the level of individual malignancies; there is a need to delineate between proxy measures of quality and the relevance of output when assessing its merit. These results raise significant questions regarding the best method of assessment of research and scientific output in the field of oncology.

Dataset Information

Characterizing the literature on validity and assessment in medical education: a bibliometric study.

Publications

Characterizing the literature on validity and assessment in medical education: a bibliometric study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets