On the Practical Consequences of Misfit in Mokken Scaling.
Ontology highlight
ABSTRACT:
SUBMITTER: Crisan DR
PROVIDER: S-EPMC7383689 | biostudies-literature | 2020 Sep
REPOSITORIES: biostudies-literature
Publications
On the Practical Consequences of Misfit in Mokken Scaling.
Crişan Daniela Ramona DR Tendeiro Jorge N JN Meijer Rob R RR
Applied psychological measurement 20200601 6
Mokken scale analysis is a popular method to evaluate the psychometric quality of clinical and personality questionnaires and their individual items. Although many empirical papers report on the extent to which sets of items form Mokken scales, there is less attention for the effect of violations of commonly used rules of thumb. In this study, the authors investigated the practical consequences of retaining or removing items with psychometric properties that do not comply with these rules of thu ...[more]
Similar Datasets
Project description:In this article, the practical consequences of violations of unidimensionality on selection decisions in the framework of unidimensional item response theory (IRT) models are investigated based on simulated data. The factors manipulated include the severity of violations, the proportion of misfitting items, and test length. The outcomes that were considered are the precision and accuracy of the estimated model parameters, the correlations of estimated ability ([Formula: see text]) and number-correct ([Formula: see text]) scores with the true ability ([Formula: see text]), the ranks of the examinees and the overlap between sets of examinees selected based on either [Formula: see text], [Formula: see text], or [Formula: see text] scores, and the bias in criterion-related validity estimates. Results show that the [Formula: see text] values were unbiased by violations of unidimensionality, but their precision decreased as multidimensionality and the proportion of misfitting items increased; the estimated item parameters were robust to violations of unidimensionality. The correlations between [Formula: see text], [Formula: see text], and [Formula: see text] scores, the agreement between the three selection criteria, and the accuracy of criterion-related validity estimates are all negatively affected, to some extent, by increasing levels of multidimensionality and the proportion of misfitting items. However, removing the misfitting items only improved the results in the case of severe multidimensionality and large proportion of misfitting items, and deteriorated them otherwise.
| S-EPMC5533251 | biostudies-literature
Project description:OBJECTIVES:In this study, we examined the consequences of ignoring violations of assumptions underlying the use of sum scores in assessing attention problems (AP) and if psychometrically more refined models improve predictions of relevant outcomes in adulthood. METHODS:Tracking Adolescents' Individual Lives data were used. AP symptom properties were examined using the AP scale of the Child Behavior Checklist at age 11. Consequences of model violations were evaluated in relation to psychopathology, educational attainment, financial status, and ability to form relationships in adulthood. RESULTS:Results showed that symptoms differed with respect to information and difficulty. Moreover, evidence of multidimensionality was found, with two groups of items measuring sluggish cognitive tempo and attention deficit hyperactivity disorder symptoms. Item response theory analyses indicated that a bifactor model fitted these data better than other competing models. In terms of accuracy of predicting functional outcomes, sum scores were robust against violations of assumptions in some situations. Nevertheless, AP scores derived from the bifactor model showed some superiority over sum scores. CONCLUSION:These findings show that more accurate predictions of later-life difficulties can be made if one uses a more suitable psychometric model to assess AP severity in children. This has important implications for research and clinical practice.
| S-EPMC7649959 | biostudies-literature
Project description:Testing item fit is an important step when calibrating and analyzing item response theory (IRT)-based tests, as model fit is a necessary prerequisite for drawing valid inferences from estimated parameters. In the literature, numerous item fit statistics exist, sometimes resulting in contradictory conclusions regarding which items should be excluded from the test. Recently, researchers argue to shift the focus from statistical item fit analyses to evaluating practical consequences of item misfit. This article introduces a method to quantify potential bias of relationship estimates (e.g., correlation coefficients) due to misfitting items. The potential deviation informs about whether item misfit is practically significant for outcomes of substantial analyses. The method is demonstrated using data from an educational test.
| S-EPMC5978672 | biostudies-literature
Project description:Internet Gaming Disorder (IGD) was recently included as a condition for further study in the fifth and latest version of the Diagnostic and Statistical Manual of Mental Disorders. The present study investigated whether the IGD criteria comprise a unidimensional construct. Data stemmed from a sample of Norwegians aged 17.5 years in 2012 and 19.5 years in 2014 (N = 1258). The study used the Mokken scale analysis to investigate whether the score of the different items on the IGD scale measured a single latent variable and if the scale functions differently for males and females. Correlation analysis was conducted between the scores on the IGD scale (count) and the Gaming Addiction Scale for Adolescents (GASA, categorical), both assessed in 2014. Negative binomial regression analyses were applied in order to investigate how different predictors of mental health assessed in 2012 were associated with IGD assessed in 2014. The Mokken scale analysis showed that all item-coefficients of homogeneity exceeded 0.3 when the whole sample completed the scale and when females completed the scale, indicating that the items reflect a single latent variable. In both cases moderate (H > 0.40) unidimensionality was shown. The item measuring "tolerance" did not exceed 0.3 in the scale when completed by males, indicating that only eight out of nine items reflect a single latent variable when applied to males only. The eight-item scale containing males showed weak (H > 0.30) unidimensionality. The correlation analysis showed a positive correlation between the scores on the IGD scale and the GASA (r = 0.71, p < 0.01) when assessed simultaneously and a positive but lower correlation (r = 0.48, p < 0.01) when assessed longitudinally. Results from the negative binomial regression analysis showed that previous video-game addiction, being male, depression, aggression and loneliness were significant predictors of IGD. The associations were small for all independent variables except previous video game addiction and gender where the associations were large. Although the results from the correlation analysis and regression analysis showed predictive validity of the scale, the results from the Mokken analysis suggest that the IGD scale may not be applied as a unidimensional scale when the tolerance item is included.
| S-EPMC6497737 | biostudies-literature
Project description:In the context of item response theory (IRT), linking the scales of two measurement points is a prerequisite to examine a change in competence over time. In educational large-scale assessments, non-identical test forms sharing a number of anchor-items are frequently scaled and linked using two- or three-parametric item response models. However, if item pools are limited and/or sample sizes are small to medium, the sparser Rasch model is a suitable alternative regarding the precision of parameter estimation. As the Rasch model implies stricter assumptions about the response process, a violation of these assumptions may manifest as model misfit in form of item discrimination parameters empirically deviating from their fixed value of one. The present simulation study investigated the performance of four IRT linking methods-fixed parameter calibration, mean/mean linking, weighted mean/mean linking, and concurrent calibration-applied to Rasch-scaled data with a small item pool. Moreover, the number of anchor items required in the absence/presence of moderate model misfit was investigated in small to medium sample sizes. Effects on the link outcome were operationalized as bias, relative bias, and root mean square error of the estimated sample mean and variance of the latent variable. In the light of this limited context, concurrent calibration had substantial convergence issues, while the other methods resulted in an overall satisfying and similar parameter recovery-even in the presence of moderate model misfit. Our findings suggest that in case of model misfit, the share of anchor items should exceed 20% as is currently proposed in the literature. Future studies should further investigate the effects of anchor item composition regarding unbalanced model misfit.
| S-EPMC8289883 | biostudies-literature
Project description:Purpose: Sample size in Mokken scales is mostly studied on simulated data, reflected in the lack of consideration of sample size in most Mokken scaling studies. Recently, [Straat, J. H., van der Ark, L. A., & Sijtsma, K. (2014). Minimum sample size requirements for Mokken scale analysis. Educational and Psychological Measurement, 74, 809-822] provided minimum sample size requirements for Mokken scale analysis based on simulation. Our study uses real data from the Warwick-Edinburgh Mental Well-Being Scale (N = 8463) to assess whether these hold. Methods: We use per element accuracy to evaluate the impact of sample size, with scaling coefficients and confidence intervals around scale, item and item pair scalability coefficients. Results: Per element accuracy, scalability coefficients, and confidence intervals around scalability coefficients are sensitive to sample size. The results from Straat et al. were not replicated; depending on the main goal of the research, sample sizes ranging from > 250 to > 1000 are needed. Conclusions: Using our pragmatic approach, some practical recommendations are made regarding sample sizes for studies of Mokken scaling.
| S-EPMC8114397 | biostudies-literature
Project description:BACKGROUND: Information processing in the brain requires large amounts of metabolic energy, the spatial distribution of which is highly heterogeneous, reflecting the complex activity patterns in the mammalian brain. RESULTS: In this study, it was found, based on empirical data, that despite this heterogeneity, the volume-specific cerebral glucose metabolic rate of many different brain structures scales with brain volume with almost the same exponent: around -0.15. The exception is white matter, the metabolism of which seems to scale with a standard specific exponent of -1/4. The scaling exponents for the total oxygen and glucose consumptions in the brain in relation to its volume are identical, at 0.86 +/- 0.03, which is significantly larger than the exponents 3/4 and 2/3 that have been suggested for whole body basal metabolism on body mass. CONCLUSION: These findings show explicitly that in mammals: (i) volume-specific scaling exponents of the cerebral energy expenditure in different brain parts are approximately constant (except brain stem structures), and (ii) the total cerebral metabolic exponent against brain volume is greater than the much-cited Kleiber's 3/4 exponent. The neurophysiological factors that might account for the regional uniformity of the exponents and for the excessive scaling of the total brain metabolism are discussed, along with the relationship between brain metabolic scaling and computation.
| S-EPMC1884139 | biostudies-other
Project description:Understanding the consequences of elevated CO2 (eCO2; 800 ppm) on terrestrial ecosystems is a central theme in global change biology, but relatively little is known about how altered plant C and N metabolism influences higher levels of biological organization. Here, we investigate the consequences of C and N interactions by genetically modifying the N-assimilation pathway in Arabidopsis and initiating growth chamber and mesocosm competition studies at current CO2 (cCO2; 400 ppm) and eCO2 over multiple generations. Using a suite of ecological, physiological, and molecular genomic tools, we show that a single-gene mutant of a key enzyme (nia2) elicited a highly orchestrated buffering response starting with a fivefold increase in the expression of a gene paralog (nia1) and a 63% increase in the expression of gene network module enriched for N-assimilation genes. The genetic perturbation reduced amino acids, protein, and TCA-cycle intermediate concentrations in the nia2 mutant compared to the wild-type, while eCO2 mainly increased carbohydrate concentrations. The mutant had reduced net photosynthetic rates due to a 27% decrease in carboxylation capacity and an 18% decrease in electron transport rates. The expression of these buffering mechanisms resulted in a penalty that negatively correlated with fitness and population dynamics yet showed only minor alterations in our estimates of population function, including total per unit area biomass, ground cover, and leaf area index. This study provides insight into the consequences of buffering mechanisms that occur post-genetic perturbations in the N pathway and the associated outcomes these buffering systems have on plant populations relative to eCO2.
| S-EPMC4541989 | biostudies-literature
Project description:Limited health literacy is known to impact on medication adherence, hospital readmission and potentially poorer health outcomes. The literature on the health literacy of those with musculoskeletal conditions suggests greater functional limitations and increased pain levels. There are a number of measures of health literacy. One that specifically relates to musculoskeletal complaints is the Literacy in Musculoskeletal Problems (LiMP) questionnaire. The LiMP contains 9 multiple choice items that cover anatomy, musculoskeletal conditions and the diagnosis of musculoskeletal complaints. The aim of the study was to evaluate the dimensionality and internal structure of the LiMP in patients attending for osteopathy care at a student-led clinic, as a potential measure of musculoskeletal health literacy.Three hundred and sixty-one (n =?361) new patients attending the Victoria University Osteopathy Clinic completed the LiMP and a demographic and health information questionnaire prior to their initial consultation. Mokken scale analysis, a nonparametric item response theory approach, was used to evaluate the dimensionality and structure of the LiMP in this population, to ascertain whether the questionnaire was measuring a single latent construct - musculoskeletal health literacy. McDonald's omega and Cronbach's alpha were calculated as the reliability estimations. The relationship between the LiMP and a single item screen of health literacy was also undertaken.The 9 items on the LiMP did not form a Mokken scale and the reliability estimations were below an acceptable level (alpha and omega <0.45). LiMP items 5 and 8 were more likely to be answered correctly by those with higher health literacy (p <?0.05), however the effect sizes were small (<0.20).Calculation of a total score for the LiMP, as advocated by the original authors, is not supported based on data in the present study. Further research is required to explore the relationship of the LiMP items to demographic and clinical data, and to other broader measures of health literacy. Further research may also develop a health literacy measure that is specific to patients seeking manual therapy care for musculoskeletal complaints.
| S-EPMC5740964 | biostudies-literature
Project description:BackgroundCOVID-19 has led to the adoption of unprecedented mitigation measures which could trigger many unintended consequences. These unintended consequences can be far-reaching and just as important as the intended ones. The World Health Organization identified the assessment of unintended consequences of COVID-19 mitigation measures as a top priority. Thus far, however, their systematic assessment has been neglected due to the inattention of researchers as well as the lack of training and practical tools.Main textOver six years our team has gained extensive experience conducting research on the unintended consequences of complex health interventions. Through a reflexive process, we developed insights that can be useful for researchers in this area. Our analysis is based on key literature and lessons learned reflexively in conducting multi-site and multi-method studies on unintended consequences. Here we present practical guidance for researchers wishing to assess the unintended consequences of COVID-19 mitigation measures. To ensure resource allocation, protocols should include research questions regarding unintended consequences at the outset. Social science theories and frameworks are available to help assess unintended consequences. To determine which changes are unintended, researchers must first understand the intervention theory. To facilitate data collection, researchers can begin by forecasting potential unintended consequences through literature reviews and discussions with stakeholders. Including desirable and neutral unintended consequences in the scope of study can help minimize the negative bias reported in the literature. Exploratory methods can be powerful tools to capture data on the unintended consequences that were unforeseen by researchers. We recommend researchers cast a wide net by inquiring about different aspects of the mitigation measures. Some unintended consequences may only be observable in subsequent years, so longitudinal approaches may be useful. An equity lens is necessary to assess how mitigation measures may unintentionally increase disparities. Finally, stakeholders can help validate the classification of consequences as intended or unintended.ConclusionStudying the unintended consequences of COVID-19 mitigation measures is not only possible but also necessary to assess their overall value. The practical guidance presented will help program planners and evaluators gain a more comprehensive understanding of unintended consequences to refine mitigation measures.
| S-EPMC7873511 | biostudies-literature