Dataset Information

On the Practical Consequences of Misfit in Mokken Scaling.

ABSTRACT: Mokken scale analysis is a popular method to evaluate the psychometric quality of clinical and personality questionnaires and their individual items. Although many empirical papers report on the extent to which sets of items form Mokken scales, there is less attention for the effect of violations of commonly used rules of thumb. In this study, the authors investigated the practical consequences of retaining or removing items with psychometric properties that do not comply with these rules of thumb. Using simulated data, they concluded that items with low scalability had some influence on the reliability of test scores, person ordering and selection, and criterion-related validity estimates. Removing the misfitting items from the scale had, in general, a small effect on the outcomes. Although important outcome variables were fairly robust against scale violations in some conditions, authors conclude that researchers should not rely exclusively on algorithms allowing automatic selection of items. In particular, content validity must be taken into account to build sensible psychometric instruments.

SUBMITTER: Crisan DR

PROVIDER: S-EPMC7383689 | biostudies-literature | 2020 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

On the Practical Consequences of Misfit in Mokken Scaling.

Crişan Daniela Ramona DR Tendeiro Jorge N JN Meijer Rob R RR

Applied psychological measurement 20200601 6

Mokken scale analysis is a popular method to evaluate the psychometric quality of clinical and personality questionnaires and their individual items. Although many empirical papers report on the extent to which sets of items form Mokken scales, there is less attention for the effect of violations of commonly used rules of thumb. In this study, the authors investigated the practical consequences of retaining or removing items with psychometric properties that do not comply with these rules of thu ...[more]

PMID: 32782419

Similar Datasets

Project description:Internet Gaming Disorder (IGD) was recently included as a condition for further study in the fifth and latest version of the Diagnostic and Statistical Manual of Mental Disorders. The present study investigated whether the IGD criteria comprise a unidimensional construct. Data stemmed from a sample of Norwegians aged 17.5 years in 2012 and 19.5 years in 2014 (N = 1258). The study used the Mokken scale analysis to investigate whether the score of the different items on the IGD scale measured a single latent variable and if the scale functions differently for males and females. Correlation analysis was conducted between the scores on the IGD scale (count) and the Gaming Addiction Scale for Adolescents (GASA, categorical), both assessed in 2014. Negative binomial regression analyses were applied in order to investigate how different predictors of mental health assessed in 2012 were associated with IGD assessed in 2014. The Mokken scale analysis showed that all item-coefficients of homogeneity exceeded 0.3 when the whole sample completed the scale and when females completed the scale, indicating that the items reflect a single latent variable. In both cases moderate (H > 0.40) unidimensionality was shown. The item measuring "tolerance" did not exceed 0.3 in the scale when completed by males, indicating that only eight out of nine items reflect a single latent variable when applied to males only. The eight-item scale containing males showed weak (H > 0.30) unidimensionality. The correlation analysis showed a positive correlation between the scores on the IGD scale and the GASA (r = 0.71, p < 0.01) when assessed simultaneously and a positive but lower correlation (r = 0.48, p < 0.01) when assessed longitudinally. Results from the negative binomial regression analysis showed that previous video-game addiction, being male, depression, aggression and loneliness were significant predictors of IGD. The associations were small for all independent variables except previous video game addiction and gender where the associations were large. Although the results from the correlation analysis and regression analysis showed predictive validity of the scale, the results from the Mokken analysis suggest that the IGD scale may not be applied as a unidimensional scale when the tolerance item is included.

Project description:BackgroundCOVID-19 has led to the adoption of unprecedented mitigation measures which could trigger many unintended consequences. These unintended consequences can be far-reaching and just as important as the intended ones. The World Health Organization identified the assessment of unintended consequences of COVID-19 mitigation measures as a top priority. Thus far, however, their systematic assessment has been neglected due to the inattention of researchers as well as the lack of training and practical tools.Main textOver six years our team has gained extensive experience conducting research on the unintended consequences of complex health interventions. Through a reflexive process, we developed insights that can be useful for researchers in this area. Our analysis is based on key literature and lessons learned reflexively in conducting multi-site and multi-method studies on unintended consequences. Here we present practical guidance for researchers wishing to assess the unintended consequences of COVID-19 mitigation measures. To ensure resource allocation, protocols should include research questions regarding unintended consequences at the outset. Social science theories and frameworks are available to help assess unintended consequences. To determine which changes are unintended, researchers must first understand the intervention theory. To facilitate data collection, researchers can begin by forecasting potential unintended consequences through literature reviews and discussions with stakeholders. Including desirable and neutral unintended consequences in the scope of study can help minimize the negative bias reported in the literature. Exploratory methods can be powerful tools to capture data on the unintended consequences that were unforeseen by researchers. We recommend researchers cast a wide net by inquiring about different aspects of the mitigation measures. Some unintended consequences may only be observable in subsequent years, so longitudinal approaches may be useful. An equity lens is necessary to assess how mitigation measures may unintentionally increase disparities. Finally, stakeholders can help validate the classification of consequences as intended or unintended.ConclusionStudying the unintended consequences of COVID-19 mitigation measures is not only possible but also necessary to assess their overall value. The practical guidance presented will help program planners and evaluators gain a more comprehensive understanding of unintended consequences to refine mitigation measures.

Dataset Information

On the Practical Consequences of Misfit in Mokken Scaling.

Publications

On the Practical Consequences of Misfit in Mokken Scaling.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets