Browse
Submit Data
Databases
API
Help

Dataset Information

26 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

How Item Residual Heterogeneity Affects Tests for Differential Item Functioning.

ABSTRACT: Differential item functioning (DIF) occurs when people with the same proficiency have different probabilities of giving a certain response to an item. The present study focused on an assumption implicit in popular methods for DIF testing that has received little attention in published literature (item residual homogeneity). The assumption is explained, a strategy for detecting violations of it (i.e., item residual heterogeneity) is illustrated with empirical data, and simulations are carried out to evaluate the performance of binary logistic regression, two-group item response theory (IRT), and the Mantel-Haenszel (MH) test in the presence of item residual heterogeneity. Results indicated that heterogeneity inflated Type I error and attenuated power for logistic regression, and attenuated power and produced biased estimates of the latent focal group mean and standard deviation for two-group IRT. The MH test was robust to item residual heterogeneity, probably because it does not use the logistic function.

SUBMITTER: Woods CM

PROVIDER: S-EPMC5978544 | biostudies-literature | 2015 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

How Item Residual Heterogeneity Affects Tests for Differential Item Functioning.

Woods Carol M CM Harpole Jared J

Applied psychological measurement 20141211 4

Differential item functioning (DIF) occurs when people with the same proficiency have different probabilities of giving a certain response to an item. The present study focused on an assumption implicit in popular methods for DIF testing that has received little attention in published literature (item residual homogeneity). The assumption is explained, a strategy for detecting violations of it (i.e., item residual heterogeneity) is illustrated with empirical data, and simulations are carried out ...[more]

PMID: 29881006

Similar Datasets

Differential item functioning in the Unified Dyskinesia Rating Scale (UDysRS).

Project description:Test if differential item functioning due to gender, age, race/ethnicity, or education impacts Unified Dyskinesia Rating Scale scores.Testing rating scales for differential item functioning is a core validation step. If differential item functioning exists, interpretation of item scores must consider secondary influences on dyskinesia ratings.Using Unified Dyskinesia Rating Scale translation databases (N?=?3,132), we tested uniform and nonuniform differential item functioning. We required confirmation by two independent methods and considered differential item functioning pertinent if McFadden pseudo R2 magnitude statistics exceeded negligible ratings.No age, race/ethnicity, or education nonuniform differential item functioning was identified. Gender nonuniform differential item functioning occurred for 2 items, both with negligible magnitude. Gender, race, and education uniform differential item functioning was observed for multiple items, all with negligible magnitude.The Unified Dyskinesia Rating Scale items effectively capture dyskinesia severity without pertinent gender, age, race/ ethnicity, or education influence. © 2017 International Parkinson and Movement Disorder Society.

| S-EPMC5570621 | biostudies-literature

Detecting Differential Item Functioning Using Multiple-Group Cognitive Diagnosis Models.

Project description:This study proposes a multiple-group cognitive diagnosis model to account for the fact that students in different groups may use distinct attributes or use the same attributes but in different manners (e.g., conjunctive, disjunctive, and compensatory) to solve problems. Based on the proposed model, this study systematically investigates the performance of the likelihood ratio (LR) test and Wald test in detecting differential item functioning (DIF). A forward anchor item search procedure was also proposed to identify a set of anchor items with invariant item parameters across groups. Results showed that the LR and Wald tests with the forward anchor item search algorithm produced better calibrated Type I error rates than the ordinary LR and Wald tests, especially when items were of low quality. A set of real data were also analyzed to illustrate the use of these DIF detection procedures.

| S-EPMC7711248 | biostudies-literature

After Differential Item Functioning Is Detected: IRT Item Calibration and Scoring in the Presence of DIF.

Project description:Researchers are commonly interested in group comparisons such as comparisons of group means, called impact, or comparisons of individual scores across groups. A meaningful comparison can be made between the groups when there is no differential item functioning (DIF) or differential test functioning (DTF). During the past three decades, much progress has been made in detecting DIF and DTF. However, little research has been conducted on what researchers can do after such detection. This study presents and evaluates a confirmatory multigroup multidimensional item response model to obtain the purified item parameter estimates, person scores, and impact estimates on the primary dimension, controlling for the secondary dimension due to DIF. In addition, the item response model approach was compared with current practices of DIF treatment such as deleting and ignoring DIF items and using multigroup item response models through simulation studies. The authors suggested guidelines for DIF treatment based on the simulation study results.

| S-EPMC5978721 | biostudies-literature

Differential item functioning of the CAHPS® In-Center Hemodialysis Survey.

Project description:PURPOSE:End-stage renal disease patients' experience of care is an integral part of the assessment of the quality of the care provided at hemodialysis centers and is needed to promote patient choice, quality improvement, and accountability. The purpose of this study is to evaluate the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH-CAHPS®) survey and its equivalence in different age, gender, race, and education subgroups. METHODS:The ICH-CAHPS survey was administered to 1454 patients from 32 dialysis facilities. For the characteristics compared, the sample had 756 participants younger than 65 years old, 739 men, 516 Black, 567 White, and 970 with less than high school diploma. Three different patient experience constructs were studied including nephrologist's communication and caring, quality of care and operations, and providing information to patients. We used item response theory analysis to examine the possibility of differential item functioning (DIF) by patient age, gender, race, and education separately after controlling for the other DIF characteristics and additional confounding variables including survey mode, mental, and general health status as well as duration on dialysis. RESULTS:The three constructs studied were unidimensional and no major DIF was observed on the composites. Some non-equivalences were observed when confounders were not controlled for, suggesting that such covariates can be important factors in understanding the possibility of disparity in patients' experience. CONCLUSIONS:The ICH-CAHPS is a promising survey to elicit hemodialysis patients' experience that has good psychometric properties and provides a standardized tool for assessing age, gender, race, or education disparity.

| S-EPMC6810787 | biostudies-literature

Uncovering Differential Item Functioning effects using MIMIC and mediated MIMIC models.

Project description:The aim of this study was twofold: first, to examine the presence of bias across gender in a scholastic achievement test named the Academic Achievement Test (AAT) for the Science Track. Second, to understand the underlying mechanism that causes these bias effects by examining the effect of general cognitive ability as a mediator. The sample consisted of 1,300 Saudi high school students randomly selected from a larger pool of 173,133 participants to reduce the effects of excessive power. To examine both goals, the Multiple Indicators Multiple Causes (MIMIC) approach for detecting Differential Item Functioning (DIF) items was used. The results showed that 13 AAT items exhibited DIF effects for different gender groups. In most of these items, male participants were more likely to answer them correctly than their female counterparts. Next, the mediated MIMIC approach was applied to explore possible underlying mechanisms that explain these DIF effects. The results from this study showed that general cognitive ability (i.e., General Aptitude Test - GAT) seems to be a factor that could explain why an AAT item exhibits DIF across gender. It was found that GAT scores fully explain the DIF effect in two AAT items (full mediation). In most other cases, GAT helps account for only a proportion of the DIF effect (partial mediation). The results from this study will help experts improve the quality of their instruments by identifying DIF items and deciding how to revise them, considering the mediator's effect on participants' responses.

| S-EPMC10626479 | biostudies-literature

Examination of Gender-Related Differential Item Functioning Through Poly-BW Indices.

Project description:The existing differential item functioning (DIF) detection approaches relying on item difficulty or item discrimination are limited for understanding the associates of DIF items, and consequently, DIF items were conventionally either deleted or ignored. Given the importance of minimizing DIF items in test construction, teachers or testing practitioners need more information regarding possible associates of DIF items. Using an example of a teacher-made mathematics achievement test, this study aimed to examine how the Poly-BW indices (power, defenselessness, disturbance, and hint) contributed to the properties of gender-related DIF items. Data from a 34-item mathematics achievement test that involved 1,439 seventh-grade students from Taiwan (51.01% boys and 48.99% girls) showed that the differences of the defenselessness (mp) and power (cp) indices between men and women served as salient predictors of the DIF measures estimated by the Poly Simultaneous Item Bias Test (Poly-SIBTEST) procedure and with satisfactory accuracy of hit rates. Items with relatively large defenselessness for men were likely to present male-favoring DIFs, whereas items with relatively large power for men were likely to present female-favoring DIFs. The Poly-BW indices yielded directions for modifying items for teachers in practice.

| S-EPMC8914429 | biostudies-literature

Differential item functioning analysis of the Vanderbilt Expertise Test for cars.

Project description:The Vanderbilt Expertise Test for cars (VETcar) is a test of visual learning for contemporary car models. We used item response theory to assess the VETcar and in particular used differential item functioning (DIF) analysis to ask if the test functions the same way in laboratory versus online settings and for different groups based on age and gender. An exploratory factor analysis found evidence of multidimensionality in the VETcar, although a single dimension was deemed sufficient to capture the recognition ability measured by the test. We selected a unidimensional three-parameter logistic item response model to examine item characteristics and subject abilities. The VETcar had satisfactory internal consistency. A substantial number of items showed DIF at a medium effect size for test setting and for age group, whereas gender DIF was negligible. Because online subjects were on average older than those tested in the lab, we focused on the age groups to conduct a multigroup item response theory analysis. This revealed that most items on the test favored the younger group. DIF could be more the rule than the exception when measuring performance with familiar object categories, therefore posing a challenge for the measurement of either domain-general visual abilities or category-specific knowledge.

| S-EPMC4588597 | biostudies-literature

An Exploratory Strategy to Identify and Define Sources of Differential Item Functioning.

Project description:The sources of differential item functioning (DIF) items are usually identified through a qualitative content review by a panel of experts. However, the differential functioning for some DIF items might have been caused by reasons outside of the experts' experiences, leading to the sources for these DIF items possibly being misidentified. Quantitative methods can help to provide useful information, such as the DIF status and the number of sources of the DIF, which in turn help the item review and revision process to be more efficient and precise. However, the current quantitative methods assume all possible sources should be known in advance and collected to accompany the item response data, which is not always the case in reality. To this end, an exploratory strategy, combined with the MIMIC (multiple-indicator multiple-cause) method, that can be used to identify and name new sources of DIF is proposed in this study. The performance of this strategy was investigated through simulation. The results showed that when a set of DIF-free items can be correctly identified to define the main dimension, the proposed exploratory MIMIC method can accurately recover a number of possible sources of DIF and the items that belong to each. A real data analysis was also implemented to demonstrate how this strategy can be used in reality. The results and findings of this study are further discussed.

| S-EPMC7495792 | biostudies-literature

A Power Formula for the Mantel-Haenszel Test for Differential Item Functioning.

Project description:The asymptotic power of the Mantel-Haenszel (MH) test for the differential item function (DIF) is derived. The formula describes the behavior of the power when the number of items is large, so that the measured latent trait can be considered as the matching variable in the MH test. As shown in the derived formula, the power is related to the sample size, effect size of DIF, the item response function (IRF), and the distribution of the latent trait in the reference and the focal groups. The formula provides an approximation of the power of the MH test in practice and thus provides a guideline for DIF detection in practice. It also suggests analytical explanations of the behavior of the MH test as observed in many previous simulation studies. Based on the formula, this study shows how to conduct the sample size calculation. The power of MH test under some practical models such as the two-parameter logistic (2PL) and three-parameter logistic (3PL) item response theory (IRT) models is discussed.

| S-EPMC5978601 | biostudies-literature

Factorial invariance, differential item functioning, and norms of the Orgasm Rating Scale.

Project description:Background/Objective: Orgasm Rating Scale (ORS) assess the subjective orgasm experience in context of sexual relationship. It is composed of four dimensions attributed to the orgasm (Affective, Sensory, Intimacy, and Rewards). The purpose is to analyse the factorial invariance of the ORS across groups, to examine the metric equivalence across sex, and to present the standard scores. Method: A total of 1,472 Spanish adults (715 men and 757 women) were evaluated. They were distributed across age groups (18-34, 35-49 and 50 years old and older). Factorial invariance across different groups and the differential functioning of the items across sex were analyzed, internal consistency was examined, and the standard scores were developed. Results: The structure of the ORS showed strict measurement invariance across sex, relationship status, sexual orientation and education level. It also reached a scalar measurement invariance across age range and duration of the relationship. Some items showed a differential functioning between sexes. Conclusions: The Spanish version of the ORS is invariant across different groups at a factorial level, and it shows equivalence across sex in most of its items at a metric level. The standard scores allow a more accurate assessment of the subjective orgasm experience in context of sexual relationship.

| S-EPMC6300712 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data