Browse
Submit Data
Databases
API
Help

Dataset Information

31 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Using multidimensional item response theory to evaluate how response styles impact measurement.

ABSTRACT: Multidimensional item response theory (MIRT) models for response style (e.g., Bolt, Lu, & Kim, 2014, Psychological Methods, 19, 528; Falk & Cai, 2016, Psychological Methods, 21, 328) provide flexibility in accommodating various response styles, but often present difficulty in isolating the effects of response style(s) from the intended substantive trait(s). In the presence of such measurement limitations, we consider several ways in which MIRT models are nevertheless useful in lending insight into how response styles may interfere with measurement for a given test instrument. Such a study can also inform whether alternative design considerations (e.g., anchoring vignettes, self-report items of heterogeneous content) that seek to control for response style effects may be helpful. We illustrate several aspects of an MIRT approach using real and simulated analyses.

SUBMITTER: Adams DJ

PROVIDER: S-EPMC6765459 | biostudies-literature | 2019 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Using multidimensional item response theory to evaluate how response styles impact measurement.

Adams Daniel J DJ Bolt Daniel M DM Deng Sien S Smith Stevens S SS Baker Timothy B TB

The British journal of mathematical and statistical psychology 20190328 3

Multidimensional item response theory (MIRT) models for response style (e.g., Bolt, Lu, & Kim, 2014, Psychological Methods, 19, 528; Falk & Cai, 2016, Psychological Methods, 21, 328) provide flexibility in accommodating various response styles, but often present difficulty in isolating the effects of response style(s) from the intended substantive trait(s). In the presence of such measurement limitations, we consider several ways in which MIRT models are nevertheless useful in lending insight in ...[more]

PMID: 30919943

Similar Datasets

Multidimensional Computerized Adaptive Testing Using Non-Compensatory Item Response Theory Models.

Project description:Current use of multidimensional computerized adaptive testing (MCAT) has been developed in conjunction with compensatory multidimensional item response theory (MIRT) models rather than with non-compensatory ones. In recognition of the usefulness of MCAT and the complications associated with non-compensatory data, this study aimed to develop MCAT algorithms using non-compensatory MIRT models and to evaluate their performance. For the purpose of the study, three item selection methods were adapted and compared, namely, the Fisher information method, the mutual information method, and the Kullback-Leibler information method. The results of a series of simulations showed that the Fisher information and mutual information methods performed similarly, and both outperformed the Kullback-Leibler information method. In addition, it was found that the more stringent the termination criterion and the higher the correlation between the latent traits, the higher the resulting measurement precision and test reliability. Test reliability was very similar across the dimensions, regardless of the correlation between the latent traits and termination criterion. On average, the difficulties of the administered items were found to be at a lower level than the examinees' abilities, which shed light on item bank construction for non-compensatory items.

| S-EPMC6696872 | biostudies-literature

Supervised Multidimensional Item Response Theory Modeling of Pediatric Iatrogenic Withdrawal Symptoms.

Project description:Item-level data from composite scales can be analyzed with pharmacometric item response theory (IRT) models to improve the quantification of disease severity compared with the use of total composite scores. However, regular IRT models assume unidimensionality, which is violated in the scale measuring iatrogenic withdrawal in children because some items are also affected by pain, undersedation, or delirium. Here, we compare regular IRT modelling of pediatric iatrogenic withdrawal symptom data with two new analysis approaches in which the latent variable is guided towards the condition of interest using numerical withdrawal severity scored by nurses as a "supervising variable:" supervised IRT (sIRT) and supervised multi-dimensional (smIRT) modelling. In this example, in which the items scores are affected by multiple conditions, regular IRT modeling is worse to quantify disease severity than the total composite score, whereas improved performance compared with the composite score is observed for the sIRT and smIRT models.

| S-EPMC6930857 | biostudies-literature

Associating Psychotic Symptoms with Altered Brain Anatomy in Psychotic Disorders Using Multidimensional Item Response Theory Models.

Project description:Reduced cortical thickness has been demonstrated in psychotic disorders, but its relationship to clinical symptoms has not been established. We aimed to identify the regions throughout neocortex where clinical psychosis manifestations correlate with cortical thickness. Rather than perform a traditional correlation analysis using total scores on psychiatric rating scales, we applied multidimensional item response theory to identify a profile of psychotic symptoms that was related to a region where cortical thickness was reduced. This analysis was performed using a large population of probands with psychotic disorders (N = 865), their family members (N = 678) and healthy volunteers (N = 347), from the 5-site Bipolar-Schizophrenia Network for Intermediate Phenotypes. Regional cortical thickness from structural magnetic resonance scans was measured using FreeSurfer; individual symptoms were rated using the Positive and Negative Syndrome Scale, Montgomery-Asberg Depression Rating Scale, and Young Mania Rating Scale. A cluster of cortical regions whose thickness was inversely related to severity of psychosis symptoms was identified. The regions turned out to be located contiguously in a large region of heteromodal association cortex including temporal, parietal and frontal lobe regions, suggesting a cluster of contiguous neocortical regions important to psychosis expression. When we tested the relationship between reduced cortical surface area and high psychotic symptoms we found no linked regions describing a related cortical set.

| S-EPMC7197072 | biostudies-literature

Parameter Recovery in Multidimensional Item Response Theory Models Under Complexity and Nonnormality.

Project description:Information about the psychometric properties of items can be highly useful in assessment development, for example, in item response theory (IRT) applications and computerized adaptive testing. Although literature on parameter recovery in unidimensional IRT abounds, less is known about parameter recovery in multidimensional IRT (MIRT), notably when tests exhibit complex structures or when latent traits are nonnormal. The current simulation study focuses on investigation of the effects of complex item structures and the shape of examinees' latent trait distributions on item parameter recovery in compensatory MIRT models for dichotomous items. Outcome variables included bias and root mean square error. Results indicated that when latent traits were skewed, item parameter recovery was generally adversely impacted. In addition, the presence of complexity contributed to decreases in the precision of parameter recovery, particularly for discrimination parameters along one dimension when at least one latent trait was generated as skewed.

| S-EPMC5978474 | biostudies-literature

A General Bayesian Multidimensional Item Response Theory Model for Small and Large Samples.

Project description:Although item response theory (IRT) models such as the bifactor, two-tier, and between-item-dimensionality IRT models have been devised to confirm complex dimensional structures in educational and psychological data, they can be challenging to use in practice. The reason is that these models are multidimensional IRT (MIRT) models and thus are highly parameterized, making them only suitable for data provided by large samples. Unfortunately, many educational and psychological studies are conducted on a small scale, leaving the researchers without the necessary MIRT models to confirm the hypothesized structures in their data. To address the lack of modeling options for these researchers, we present a general Bayesian MIRT model based on adaptive informative priors. Simulations demonstrated that our MIRT model could be used to confirm a two-tier structure (with two general and six specific dimensions), a bifactor structure (with one general and six specific dimensions), and a between-item six-dimensional structure in rating scale data representing sample sizes as small as 100. Although our goal was to provide a general MIRT model suitable for smaller samples, the simulations further revealed that our model was applicable to larger samples. We also analyzed real data from 121 individuals to illustrate that the findings of our simulations are relevant to real situations.

| S-EPMC7307486 | biostudies-literature

A Finite Mixture Item Response Theory Model for Continuous Measurement Outcomes.

Project description:A mixture extension of Samejima's continuous response model for continuous measurement outcomes and its estimation through a heuristic approach based on limited-information factor analysis is introduced. Using an empirical data set, it is shown that two groups of respondents that differ both qualitatively and quantitatively in their response behavior can be revealed. In addition to the real data application, the effectiveness of the heuristic estimation approach under real data analytic conditions was examined through a Monte Carlo simulation study. The results showed that the heuristic estimation approach provided reliable parameter estimates and the model successfully converged above 80% when the sample size was 250 and above 90% when the sample size was 500 or 1,000 for most conditions.

| S-EPMC7047264 | biostudies-literature

Does response shift impact interpretation of change even among scales developed using item response theory?

Project description:BACKGROUND:Response-shift effects impact the interpretation of change in quality-of-life (QOL) measures developed with classical test theory (CTT) methods. This study evaluated the impact of response shift on measures developed using Item Response Theory (IRT), as compared to CTT. METHODS:Chronically ill patients and caregivers (n?=?1481) participated in a web-based survey at baseline and 17?months later. Patients completed the IRT-based PROMIS-10; NeuroQOL Applied Cognition, Positive Affect & Well-Being short-forms; and the CTT-based Ryff Environmental Mastery subscale. Response-shift effects were evaluated using regression residual modeling and the QOL Appraisal Profile-v2. The sample was divided into positive and negative catalyst groups on the basis of marital, work, job-status, and comorbidity change. Regression models predicted residualized QOL change scores as a function of catalysts and appraisal changes. RESULTS:In this sample 859 (58%) reported a catalyst. No catalyst was associated with change in scales developed using IRT, but positive work change was associated with the CTT-based measure. Catalyst variables were associated with changes in appraisal, which in turn were related to all outcomes, particularly for global mental health after a positive work-change. CONCLUSIONS:Appraisal processes are relevant to interpreting IRT measures, particularly for global mental health in the face of life changes.

| S-EPMC6977794 | biostudies-literature

New resilience instrument for family caregivers in cancer: a multidimensional item response theory analysis.

Project description:ObjectiveResilience instruments specific to family caregivers (FCs) in cancer are limited. This study was designed to validate the 10-item Resilience Scale Specific to Cancer (RS-SC-10) in FCs using multidimensional item response theory (MIRT) analysis.Methods382 FCs were enrolled from Be Resilient to Cancer Program (BRCP) and administered with RS-SC-10 and 36-item Short Form Health Survey (SF-36). MIRT was performed to evaluate item parameters while Generalized Additive Model (GAM) and Latent Profile Analysis (LPA) were performed to test the non-linear relationship between resilience (RS-SC-10) and Quality of Life (QoL, SF-36).ResultsRS-SC-10 retained 10 items with high multidimensional discrimination, monotonous thresholds and its original two-factor structure (Generic and Shift-Persist). Four latent resilience subgroups were identified and a non-linear dose-response pattern between resilience and QoL was confirmed (per-SD increase OR = 1.62, 95% CI 1.16-2.13, p = 0.0019).ConclusionRS-SC-10 is a brief and suitable resilience instrument for FCs in cancer. The resilience screening of patients and FCs can be performed simultaneously in clinical practice.

| S-EPMC8600888 | biostudies-literature

Modeling Composite Assessment Data Using Item Response Theory.

Project description:Composite assessments aim to combine different aspects of a disease in a single score and are utilized in a variety of therapeutic areas. The data arising from these evaluations are inherently discrete with distinct statistical properties. This tutorial presents the framework of the item response theory (IRT) for the analysis of this data type in a pharmacometric context. The article considers both conceptual (terms and assumptions) and practical questions (modeling software, data requirements, and model building).

| S-EPMC5915608 | biostudies-literature

Robust Neural Automated Essay Scoring Using Item Response Theory

Project description:Automated essay scoring (AES) is the task of automatically assigning scores to essays as an alternative to human grading. Conventional AES methods typically rely on manually tuned features, which are laborious to effectively develop. To obviate the need for feature engineering, many deep neural network (DNN)-based AES models have been proposed and have achieved state-of-the-art accuracy. DNN-AES models require training on a large dataset of graded essays. However, assigned grades in such datasets are known to be strongly biased due to effects of rater bias when grading is conducted by assigning a few raters in a rater set to each essay. Performance of DNN models rapidly drops when such biased data are used for model training. In the fields of educational and psychological measurement, item response theory (IRT) models that can estimate essay scores while considering effects of rater characteristics have recently been proposed. This study therefore proposes a new DNN-AES framework that integrates IRT models to deal with rater bias within training data. To our knowledge, this is a first attempt at addressing rating bias effects in training data, which is a crucial but overlooked problem.

| S-EPMC7334153 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data