Score-based tests of measurement invariance: use in practice.
Ontology highlight
ABSTRACT:
SUBMITTER: Wang T
PROVIDER: S-EPMC4038958 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectiveComparability of meaning of neuropsychological test results across ethnic, linguistic, and cultural groups is important for clinicians challenged with assessing increasing numbers of older ethnic minorities. We examined the dimensional structure of a neuropsychological test battery in linguistically and demographically diverse older adults.MethodThe Spanish and English Neuropsychological Assessment Scales (SENAS), developed to provide psychometrically sound measures of cognition for multiethnic and multilingual applications, was administered to a community dwelling sample of 760 Whites, 443 African Americans, 451 English-speaking Hispanics, and 882 Spanish-speaking Hispanics. Cognitive function spanned a broad range from normal to mildly impaired to demented. Multiple group confirmatory factor analysis was used to examine equivalence of the dimensional structure for the SENAS across the groups defined by language and ethnicity.ResultsCovariance among 16 SENAS tests was best explained by five cognitive dimensions corresponding to episodic memory, semantic memory/language, spatial ability, attention/working memory, and verbal fluency. Multiple Group confirmatory factor analysis supported a common dimensional structure in the diverse groups. Measures of episodic memory showed the most compelling evidence of measurement equivalence across groups. Measurement equivalence was observed for most but not all measures of semantic memory/language and spatial ability. Measures of attention/working memory defined a common dimension in the different groups, but results suggest that scores are not strictly comparable across groups.ConclusionsThese results support the applicability of the SENAS for use with multiethnic and bilingual older adults, and more broadly, provide evidence of similar dimensions of cognition in the groups represented in the study.
| S-EPMC3058248 | biostudies-literature
Project description:This study examined whether cutoffs in fit indices suggested for traditional formats with maximum likelihood estimators can be utilized to assess model fit and to test measurement invariance when a multiple group confirmatory factor analysis was employed for the Thurstonian item response theory (IRT) model. Regarding the performance of the evaluation criteria, detection of measurement non-invariance and Type I error rates were examined. The impact of measurement non-invariance on estimated scores in the Thurstonian IRT model was also examined through accuracy and efficiency in score estimation. The fit indices used for the evaluation of model fit performed well. Among six cutoffs for changes in model fit indices, only ΔCFI > .01 and ΔNCI > .02 detected metric non-invariance when the medium magnitude of non-invariance occurred and none of the cutoffs performed well to detect scalar non-invariance. Based on the generated sampling distributions of fit index differences, this study suggested ΔCFI > .001 and ΔNCI > .004 for scalar non-invariance and ΔCFI > .007 for metric non-invariance. Considering Type I error rate control and detection rates of measurement non-invariance, ΔCFI was recommended for measurement non-invariance tests for forced-choice format data. Challenges in measurement non-invariance tests in the Thurstonian IRT model were discussed along with the direction for future research to enhance the utility of forced-choice formats in test development for cross-cultural and international settings.
| S-EPMC7262996 | biostudies-literature
Project description:Factor analysis models are widely used in health research to summarize hard to measure predictor or outcome variable constructs. For example, in the ELEMENT study, factor models are used to summarize lead exposure biomarkers which are thought to indirectly measure prenatal exposure to lead. Classic latent factor models are fitted assuming that factor loadings are constant across all covariate levels (e.g., maternal age in ELEMENT); that is, measurement invariance (MI) is assumed. When the MI is not met, measurement bias is introduced. Traditionally, MI is examined by defining subgroups of the data based on covariates, fitting multi-group factor analysis, and testing differences in factor loadings across covariate groups. In this paper, we develop novel tests of measurement invariance by modeling the factor loadings as varying coeffcients, i.e., letting the factor loading vary across continuous covariate values instead of groups. These varying coeffcients are estimated using penalized splines, where spline coeffcients are penalized by treating them as random coeffcients. The test of MI is then carried out by conducting a likelihood ratio test for the null hypothesis that the variance of the random spline coeffcients equals zero. We use a Monte-Carlo EM algorithm for estimation, and obtain the likelihood using Monte-Carlo in tegration. Using simulations, we compare the Type I error and power of our testing approach and the multi-group testing method. We apply the proposed methods to to summarize data on prenatal biomarkers of lead exposure from the ELEMENT study and find violations of MI due to maternal age.
| S-EPMC6385881 | biostudies-literature
Project description:M-fluctuation tests are a recently proposed method for detecting differential item functioning in Rasch models. This article discusses a generalization of this method to two additional item response theory models: the two-parametric logistic model and the three-parametric logistic model with a common guessing parameter. The Type I error rate and the power of this method were evaluated by a variety of simulation studies. The results suggest that the new method allows the detection of various forms of differential item functioning in these models, which also includes differential discrimination and differential guessing effects. It is also robust against moderate violations of several assumptions made in the item parameter estimation.
| S-EPMC6425091 | biostudies-literature
Project description:Cross-national data production in social science research has increased dramatically in recent decades. Assessing the comparability of data is necessary before drawing substantive conclusions that are based on cross-national data. Researchers assessing data comparability typically use either quantitative methods such as multigroup confirmatory factor analysis or qualitative methods such as online probing. Because both methods have complementary strengths and weaknesses, this study applies both multigroup confirmatory factor analysis and online probing in a mixed-methods approach to assess the comparability of constructive patriotism and nationalism, two important concepts in the study of national identity. Previous measurement invariance tests failed to achieve scalar measurement invariance, which prohibits a cross-national comparison of latent means (Davidov 2009). The arrival of the 2013 ISSP Module on National Identity has encouraged a reassessment of both constructs and a push to understand why scalar invariance cannot be achieved. Using the example of constructive patriotism and nationalism, this study demonstrates how the combination of multigroup confirmatory factor analysis and online probing can uncover and explain issues related to cross-national comparability.
| S-EPMC5452432 | biostudies-literature
Project description:The Machiavellian IV [1] instrument, developed almost 50 years ago to measure trait Machiavellianism and still in wide use in personality research, uses item wording that is not gender-neutral, makes use of idiomatic expressions, and includes archaic references. In this two-sample study, exploratory factor analysis (EFA) was conducted on one sample to examine the structure of responses to the Mach IV. In an independent second sample the resulting EFA structure was analyzed using confirmatory factor analysis-based measurement equivalence/invariance (ME/I) tests in a control group with the original archaic items and a treatment group with eight items rewritten in a more modern vernacular. Specific model testing steps [2] and statistical tests [3] were applied in a bottom-up approach [4] to ME/I tests on these two versions of the Mach IV. The two versions were found to have equal form, equal factor loadings, but unequal indicator error variances. Subsequent item-by-item tests of error invariance resulted in substantial decrements to fit for three revised items suggesting that the error associated with these items was not equal across the two versions.
| S-EPMC6795441 | biostudies-literature
Project description:Data often have a nested, multilevel structure, for example when data are collected from children in classrooms. This kind of data complicate the evaluation of reliability and measurement invariance, because several properties can be evaluated at both the individual level and the cluster level, as well as across levels. For example, cross-level invariance implies equal factor loadings across levels, which is needed to give latent variables at the two levels a similar interpretation. Reliability at a specific level refers to the ratio of true score variance over total variance at that level. This paper aims to shine light on the relation between reliability, cross-level invariance, and strong factorial invariance across clusters in multilevel data. Specifically, we will illustrate how strong factorial invariance across clusters implies cross-level invariance and perfect reliability at the between level in multilevel factor models.
| S-EPMC5641393 | biostudies-literature
Project description:BackgroundThe Urbanicity Scale was developed based on the China Health and Nutrition Survey (CHNS) to measure the urbanization index of communities according to 12 components. The present study was designed to systematically investigate the factorial validity, reliability, and longitudinal measurement invariance (LMI) of the Urbanicity Scale.MethodsSix waves of CHNS data from 2000 to 2015 were adopted. The factor structure and reliability of the Urbanicity Scale for 301 communities were examined using Bayesian exploratory factor analysis. Metric and scalar LMIs were evaluated using both the conventional exact and a novel approximate LMI approach via Bayesian structural equation modeling across various timeframes.ResultsThe findings verified the one-factor structure for the Urbanicity Scale, with adequate reliability. LMI was established for the Urbanicity Scale only over a shorter timeframe from 2006 to 2009 but not over a longer timeframe from 2000 to 2015. Partial LMI was found in the factor loadings and item intercepts for the Urbanicity Scale over the 2004 to 2011 period.ConclusionInterpretation of the temporal change in urbanicity was supported only for a shorter (2006 to 2009) but not a longer timeframe (2000 to 2015). Adjustments addressing the partial non-invariance of the measurement parameters are needed for the analysis of temporal changes in urbanicity between 2004 and 2011.
| S-EPMC8431910 | biostudies-literature
Project description:On page 1 of his classic text, Millsap (2011) states, "Measurement invariance is built on the notion that a measuring device should function the same way across varied conditions, so long as those varied conditions are irrelevant [emphasis added] to the attribute being measured." By construction, measurement invariance techniques require not only detecting varied conditions but also ruling out that these conditions inform our understanding of measured domains (i.e., conditions that do not contain domain-relevant information). In fact, measurement invariance techniques possess great utility when theory and research inform their application to specific, varied conditions (e.g., cultural, ethnic, or racial background of test respondents) that, if not detected, introduce measurement biases, and, thus, depress measurement validity (e.g., academic achievement and intelligence). Yet, we see emerging bodies of work where scholars have "put the cart before the horse" when it comes to measurement invariance, and they apply these techniques to varied conditions that, in fact, may reflect domain-relevant information. These bodies of work highlight a larger problem in measurement that likely cuts across many areas of scholarship. In one such area, youth mental health, researchers commonly encounter a set of conditions that nullify the use of measurement invariance, namely discrepancies between survey reports completed by multiple informants, such as parents, teachers, and youth themselves (i.e., informant discrepancies). In this paper, we provide an overview of conceptual, methodological, and measurement factors that should prevent researchers from applying measurement invariance techniques to detect informant discrepancies. Along the way, we cite evidence from the last 15 years indicating that informant discrepancies reflect domain-relevant information. We also apply this evidence to recent uses of measurement invariance techniques in youth mental health. Based on prior evidence, we highlight the implications of applying these techniques to multi-informant data, when the informant discrepancies observed within these data might reflect domain-relevant information. We close by calling for a moratorium on applying measurement invariance techniques to detect informant discrepancies in youth mental health assessments. In doing so, we describe how the state of the science would need to fundamentally "flip" to justify applying these techniques to detect informant discrepancies in this area of work.
| S-EPMC9378825 | biostudies-literature
Project description:Background and objectivesSocial media use has been linked to socioemotional health; however, less is known regarding whether these associations are moderated by age. Additionally, as the use of social media in older adult populations is rapidly increasing, there is a greater need for the investigation of psychometric properties of social media usage scales before determining age differences in the impact of social media on socioemotional health outcomes.Research design and methodsUsing an online adult life-span sample (n = 592), the current cross-sectional study tested the measurement invariance of the general social media usage subscale of the Media and Technology Usage and Attitudes Scale across younger (aged 19-54) versus older (aged 55-81) adults and whether age moderated associations between social media use and socioemotional health (depressive symptoms, self-esteem, and envy).ResultsConfirmatory factor analyses revealed that posting-related and checking-related items were noninvariant across age groups. In multigroup structural equation models accounting for differential item functioning, higher social media use was associated with more depressive symptoms in younger adults, but not in older adults. While higher social media use was associated with higher envy in both age groups, this association was stronger in younger adults.Discussion and implicationsFindings suggest younger adults may be more susceptible to the detrimental effects of social media use on socioemotional health. Future directions regarding the measurement of social media use and the salience of social media use across the life span are discussed.
| S-EPMC8057130 | biostudies-literature