Dataset Information

Item pre-knowledge true prevalence in clinical anatomy - application of gated item response theory model.

ABSTRACT: BACKGROUND:Computer and paper examinations in our days are constructed from an item pool which is regularly updated. Given the way that exams are created, one of the major concerns is the security of the items that are being used in order to ensure a good estimation of abilities. The aim of this study is to measure the prevalence of item pre-knowledge in our medical school. METHODS:The Deterministic, Gated Item Response Theory Model (DGM) was applied to estimate the prevalence of students who have had item pre-knowledge from six multiple choice examinations of the Clinical Anatomy course at the Faculty of Medicine of University of Porto. Each examination consisted of 100 items with an average of 200 students and 20% repeated items per examination. The estimation of the sensitivity and specificity was based on a simulation study. The sensitivity and specificity estimates, and apparent prevalence were used to estimate true prevalence of cheating students in the examinations under study. RESULTS:The specificity in the DGM for different simulation scenarios was between 68 and 98%, while the sensitivity ranged from 60 to 91%. The apparent prevalence was between 0.0 and 3.4%, while the true prevalence ranged from 1.2 to 3.7%. CONCLUSIONS:The true prevalence was much lower compared to the students self-reported copying of responses from other students; however, it is important to keep monitoring the pre-knowledge prevalence in order to enforce measures in case an increase occurs.

SUBMITTER: Severo M

PROVIDER: S-EPMC6659220 | biostudies-literature | 2019 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Item pre-knowledge true prevalence in clinical anatomy - application of gated item response theory model.

Severo Milton M Silva-Pereira Fernanda F Ferreira Maria Amelia MA Monteiro Magda M Pereira Isabel I

BMC medical education 20190725 1

<h4>Background</h4>Computer and paper examinations in our days are constructed from an item pool which is regularly updated. Given the way that exams are created, one of the major concerns is the security of the items that are being used in order to ensure a good estimation of abilities. The aim of this study is to measure the prevalence of item pre-knowledge in our medical school.<h4>Methods</h4>The Deterministic, Gated Item Response Theory Model (DGM) was applied to estimate the prevalence of ...[more]

PMID: 31345201

Similar Datasets

Project description:PURPOSE:The association of chronic lymphocytic leukemia (CLL) with health-related quality of life (HRQoL) is rarely studied globally. This study evaluated the psychometric properties of the EORTC-Chronic Lymphocytic Leukaemia (CLL17 [phase III]) module, a newly developed assessment on CLL patients' HRQoL, among Chinese CLL patients. METHODS:The Chinese CLL17, comprised of three subscales (symptom burden [SB], physical condition [PC] and worries/fears [WF]), was provided by the developer team through EORTC. A cross-sectional online survey was conducted to collect data. The classical traditional theory (CTT) and the item response theory (IRT) were used to evaluate the psychometric properties of CLL17. Internal consistency reliability was determined by the Cronbach's alpha and item-total correlation. Dimensionality was verified through confirmatory factor analysis (CFA). Convergent validity was also assessed. The generalized partial credit model was used for the IRT. The difficulty, discrimination, item fit, and differential item functioning (DIF) were calculated to assess the instrument's psychometric properties. RESULTS:In all, 318 patients, aged between 26 and 82 years, completed the questionnaire. A good level of internal reliability was achieved (Cronbach's alpha = 0.92). The item-total correlation coefficient ranged from 0.46 to 0.72. There was a mid-to-high correlation between CLL17 and domains of EQ-5D and QLQ-C30. The IRT model showed a satisfactory homogeneity, item fit and good discrimination of items, except for item 4, 6 and 16 (< 1.0). low information provided by item 16 and 17. SB and PC provided more information with theta > 0, whereas WF provided more information with theta < 0. Item 17 perform inconsistently for respondents from different age groups (DIF). CONCLUSION:The EORTC-CLL17 Chinese version shows acceptable reliability and validity, making it a valuable instrument to evaluate the impact on the HRQoL of Chinese CLL patients.

Project description:Allostatic load is commonly operationalized using a sum-score of high-risk biomarkers. However, this method implies that biomarkers contribute equally to allostatic load, as each is given equal weight. Our goal in this methodological paper is to evaluate this, and complementarily, to identify biomarkers that are most informative and least informative for developing an allostatic load index. Item response theory models provide an alternate approach to calculating the allostatic load score, by treating individual biomarkers (e.g. “items”) as indicators of a latent allostatic load construct. Item response theory scores account for the data-driven discriminating power of each biomarker, and an individual’s pattern of biomarker responses. To demonstrate feasibility of this approach, we used data from the 2015–2016 National Health Examination and Nutrition Survey (NHANES; N = 3751), with twelve allostatic load biomarkers representing immune response, metabolic function and cardiovascular health. Item response theory models revealed that body-mass-index and C-reactive protein were the most informative biomarkers for allostatic load. Both higher allostatic load sum-score and allostatic load item response theory score were associated with lower socio-economic status (p = 0.008; p<0.001, respectively). Further, both formulations of allostatic load were positively associated with a nine-item depression screener (p<0.001 for both), but only the item response theory score was also positively associated with the impact of depressive symptoms on daily life (p = 0.045). Item response theory scores may be more finely tuned to tease out effects, compared to sum-scores, and also provide more flexibility when there are missing biomarker measurements. Supplemental R code for our approach are included. Highlights • Methodological paper to introduce item response theory for calculating allostatic load.• Biomarker data from NHANES 2015–2016 representative of United States adults.• Body-mass-index and C-reactive protein most informative for allostatic load.• Item response theory captures more variability in allostatic load compared to sum-scores.• Future work - item response theory can standardize allostatic load across datasets.

Dataset Information

Item pre-knowledge true prevalence in clinical anatomy - application of gated item response theory model.

Publications

Item pre-knowledge true prevalence in clinical anatomy - application of gated item response theory model.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets