Dataset Information

Evaluation of the Linear Composite Conjecture for Unidimensional IRT Scale for Multidimensional Responses.

ABSTRACT: The linear composite direction represents, theoretically, where the unidimensional scale would lie within a multidimensional latent space. Using compensatory multidimensional IRT, the linear composite can be derived from the structure of the items and the latent distribution. The purpose of this study was to evaluate the validity of the linear composite conjecture and examine how well a fitted unidimensional IRT model approximates the linear composite direction in a multidimensional latent space. Simulation experiment results overall show that the fitted unidimensional IRT model sufficiently approximates linear composite direction when correlation between bivariate latent variables is positive. When the correlation between bivariate latent variables is negative, instability occurs when the fitted unidimensional IRT model is used to approximate linear composite direction. A real data experiment was also conducted using 20 items from a multiple-choice mathematics test from American College Testing.

SUBMITTER: Strachan T

PROVIDER: S-EPMC9265490 | biostudies-literature | 2022 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluation of the Linear Composite Conjecture for Unidimensional IRT Scale for Multidimensional Responses.

Strachan Tyler T Cho Uk Hyun UH Ackerman Terry T Chen Shyh-Huei SH de la Torre Jimmy J Ip Edward H EH

Applied psychological measurement 20220615 5

The linear composite direction represents, theoretically, where the unidimensional scale would lie within a multidimensional latent space. Using compensatory multidimensional IRT, the linear composite can be derived from the structure of the items and the latent distribution. The purpose of this study was to evaluate the validity of the linear composite conjecture and examine how well a fitted unidimensional IRT model approximates the linear composite direction in a multidimensional latent space ...[more]

PMID: 35812816

Similar Datasets

Project description:Proper pain therapy requires adequate pain assessment. This study evaluated the reliability and validity of the Unesp-Botucatu horse acute pain scale (UHAPS), the Orthopedic Composite Pain Scale (CPS) and unidimensional scales in horses admitted for orthopedic and soft tissue surgery. Forty-two horses were assessed and videotaped before surgery, up to 4 hours postoperatively, up to 3 hours after analgesic treatment, and 24 hours postoperatively (168 video clips). After six evaluators viewing each edited video clip twice in random order at a 20-day interval, they chose whether analgesia would be indicated and applied the Simple Descriptive, Numeric and Visual Analog scales, CPS, and UHAPS. For all evaluators, intra-observer reliability of UHAPS and CPS ranged from 0.70 to 0.97. Reproducibility was variable among the evaluators and ranged from poor to very good for all scales. Principal component analysis showed a weak association among 50% and 62% of the UHAPS and CPS items, respectively. Criterion validity based on Spearman correlation among all scales was above 0.67. Internal consistency was minimally acceptable (0.51-0.64). Item-total correlation was acceptable (0.3-0.7) for 50% and 38% of UHAPS and CPS items, respectively. UHAPS and CPS were specific (90% and 79% respectively), but both were not sensitive (43 and 38%, respectively). Construct validity (responsiveness) was confirmed for all scales because pain scores increased after surgery. The cut-off point for rescue analgesia was ≥ 5 and ≥ 7 for the UHAPS and CPS, respectively. All scales presented adequate repeatability, criterion validity, and partial responsiveness. Both composite scales showed poor association among items, minimally acceptable internal consistency, and weak sensitivity, indicating that they are suboptimal instruments for assessing postoperative pain. Both composite scales require further refinement with the exclusion of redundant or needless items and reduction of their maximum score applied to each item or should be replaced by other tools.

Project description:We present an exploratory method for discovering likely misconceptions from multiple-choice concept test data, as well as preliminary evidence that this method recovers known misconceptions from real student responses. Our procedure is based on a Bayesian implementation of the Multidimensional Nominal Categories IRT model (MNCM) combined with standard factor-analytic rotation methods; by analyzing student responses at the level of individual distractors rather than at the level of entire questions, this approach is able to highlight multiple likely misconceptions for subsequent investigation without requiring any manual labeling of test content. We explore the performance of the Bayesian MNCM on synthetic data and find that it is able to recover multidimensional item parameters consistently at achievable sample sizes. These studies demonstrate the method's robustness to overfitting and ability to perform automatic dimensionality assessment and selection. The method also compares favorably to existing IRT software implementing marginal maximum likelihood estimation which we use as a validation benchmark. We then apply our method to approximately 10,000 students' responses to a research-designed concept test: the Force Concept Inventory. In addition to a broad first dimension strongly correlated with overall test score, we discover thirteen additional dimensions which load on smaller sets of distractors; we discuss two as examples, showing that these are consistent with already-known misconceptions in Newtonian mechanics. While work remains to validate our findings, our hope is that future applications of this method could aid in the refinement of existing concept inventories or the development of new ones, enable the discovery of previously-unknown student misconceptions across a variety of disciplines, and-by leveraging the method's ability to quantify the prevalence of particular misconceptions-provide opportunities for targeted instruction at both the individual and classroom level.

Project description:BACKGROUND:Many instruments to identify frail older people have been developed. One of the consequences is that the prevalence rates of frailty vary widely dependent on the instrument selected. The aims of this study were 1) to examine the concordances and differences between a unidimensional and multidimensional assessment of frailty, 2) to assess to what extent the characteristics of a 'frail sample' differ depending on the selected frailty measurement because 'being frail' is used in many studies as an inclusion criterion. METHOD:A cross-sectional study was conducted among 196 community-dwelling older adults (≥60 years), which were selected from the census records. Unidimensional frailty was operationalized according to the Fried Phenotype (FP) and multidimensional frailty was measured with the Comprehensive Frailty Assessment Instrument (CFAI). The concordances and differences were examined by prevalence, correlations, observed agreement and Kappa values. Differences between sample characteristics (e.g., age, physical activity, life satisfaction) were investigated with ANOVA and Kruskall-Wallis test. RESULTS:The mean age was 72.74 (SD 8.04) and 48.98% was male. According to the FP 23.59% was not-frail, 56.92% pre-frail and 19.49% frail. According to the CFAI, 44.33% was no-to-low frail, 37.63% was mild frail and 18.04% was high frail. The correlation between FP and the CFAI was r = 0.46 and the observed agreement was 52.85%. The Kappa value was κ = 0.35 (quadratic κ = 0.45). In total, 11.92% of the participants were frail according to both measurements, 7.77% was solely frail according to the FP and 6.21% was solely frail according to the CFAI. The 'frail sample respondents' according to the FP had higher levels of life satisfaction and net income, but performed less physical activities in comparison to high frail people according to the CFAI. CONCLUSION:The present study shows that the FP and CFAI partly measure the same 'frailty-construct', although differences were found for instance in the prevalence of frailty and the composition of the 'frail participants'. Since 'being frail' is an inclusion criterion in many studies, researchers must be aware that the choice of the frailty measurement has an impact on both the estimates of frailty prevalence and the characteristics of the selected sample.

Dataset Information

Evaluation of the Linear Composite Conjecture for Unidimensional IRT Scale for Multidimensional Responses.

Publications

Evaluation of the Linear Composite Conjecture for Unidimensional IRT Scale for Multidimensional Responses.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets