Dataset Information

Performance of Fit Indices in Choosing Correct Cognitive Diagnostic Models and Q-Matrices.

ABSTRACT: In applications of cognitive diagnostic models (CDMs), practitioners usually face the difficulty of choosing appropriate CDMs and building accurate Q-matrices. However, functions of model-fit indices that are supposed to inform model and Q-matrix choices are not well understood. This study examines the performance of several promising model-fit indices in selecting model and Q-matrix under different sample size conditions. Relative performance between Akaike information criterion and Bayesian information criterion in model and Q-matrix selection appears to depend on the complexity of data generating models, Q-matrices, and sample sizes. Among the absolute fit indices, MX2 is least sensitive to sample size under correct model and Q-matrix specifications, and performs the best in power. Sample size is found to be the most influential factor on model-fit index values. Consequences of selecting inaccurate model and Q-matrix in classification accuracy of attribute mastery are also evaluated.

SUBMITTER: Lei PW

PROVIDER: S-EPMC5978496 | biostudies-literature | 2016 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Performance of Fit Indices in Choosing Correct Cognitive Diagnostic Models and Q-Matrices.

Lei Pui-Wa PW Li Hongli H

Applied psychological measurement 20160728 6

In applications of cognitive diagnostic models (CDMs), practitioners usually face the difficulty of choosing appropriate CDMs and building accurate Q-matrices. However, functions of model-fit indices that are supposed to inform model and Q-matrix choices are not well understood. This study examines the performance of several promising model-fit indices in selecting model and Q-matrix under different sample size conditions. Relative performance between Akaike information criterion and Bayesian in ...[more]

PMID: 29881062

Similar Datasets

Project description:BackgroundSelf-explanation without feedback has been shown to improve medical students' diagnostic reasoning. While feedback is generally seen as beneficial for learning, available evidence of the value of its combination with self-explanation is conflicting. This study investigated the effect on medical students' diagnostic performance of adding immediate or delayed content-feedback to self-explanation while solving cases.MethodsNinety-four 3rd-year students from a Canadian medical school were randomly assigned to three experimental conditions (immediate-feedback, delayed-feedback, control). In the learning phase, all students solved four clinical cases by giving i) the most likely diagnosis, ii) two main arguments supporting this diagnosis, and iii) two plausible alternative diagnoses, while using self-explanation. The immediate-feedback group was given the correct diagnosis after each case; delayed-feedback group received the correct diagnoses only after the four cases; control group received no feedback. One week later, all students solved four near-transfer (i.e., same final diagnosis as the learning cases but different scenarios) and four far-transfer cases (i.e., different final diagnosis from the learning cases and different scenarios) by answering the same three questions. Students' diagnostic accuracy (score for the response to the first question only) and diagnostic performance (combined score of responses to the three questions) scores were assessed in each phase. Four one-way ANOVAs were performed on each of the two scores for near and far-transfer cases.ResultsThere was a significant effect of experimental condition on diagnostic accuracy on near-transfer cases (p < .05). The immediate-feedback and delayed-feedback groups performed equally well, both better than control (respectively, mean = 90.73, standard deviation =10.69; mean = 89.92, standard deviation = 13.85; mean = 82.03, standard deviation = 17.66). The experimental conditions did not significantly differ on far-transfer cases.ConclusionsProviding feedback to students in the form of the correct diagnosis after using self-explanation with clinical cases is potentially beneficial to improve their diagnostic accuracy but this effect is limited to similar cases. Further studies should explore how more elaborated feedback combined with self-explanation may impact students' diagnostic performance on different cases.

Project description:The Occupational Depression Inventory (ODI) was recently developed to assess depressive symptoms that individuals specifically attribute to their work. Research on the criterion validity of the instrument is still in its infancy. In this study, we examined whether the ODI predicted performance on, and appreciation of, a cognitively challenging test. In light of the link established between clinical depression and neuropsychological impairment, and considering that individuals with depressive symptoms are more likely to feel helpless under challenging circumstances, we hypothesized that occupational depression would be associated with poorer cognitive performance and a darkened appreciation of the task undertaken. We relied on a sample of 1,359 educational staff members. We used an abridged version of Raven's Advanced Progressive Matrices (RAPMs) as a cognitively challenging task and measure of cognitive performance. RAPMs assess so-called eductive ability (meaning-making and problem-solving abilities) through items of various degrees of difficulty. Transient mood was assessed with a three-item measure before RAPMs were administered. Task appreciation was assessed with a single-item measure after the completion of RAPMs. We found occupational depression to be negatively linked to cognitive performance. A two-step cluster analysis, in which ODI and RAPMs scores were used as classifiers, revealed two profiles of respondents. The first profile was characterized by relatively low levels of depressive symptoms and high levels of cognitive performance; the second profile, by relatively high levels of depressive symptoms and low levels of cognitive performance. The two profiles differed strongly from one another, as indexed by Cohen's ds of 2.492 regarding depressive symptoms and 1.263 regarding cognitive performance. As anticipated, occupational depression predicted a darkened appreciation of the test. The association remained statistically significant, and largely unchanged, controlling for pretest mood and test performance. The highest levels of depressive symptoms were observed among individuals evaluating the task as "frustrating" and "discouraging." Our study suggests that occupational depression predicts poorer cognitive performance and a negativized experience of cognitive challenge. Such features may be part of a self-sustaining loop fostering the maintenance of depressive symptoms. The extent to which the ODI predicts performance in the work context needs to be investigated.

Dataset Information

Performance of Fit Indices in Choosing Correct Cognitive Diagnostic Models and Q-Matrices.

Publications

Performance of Fit Indices in Choosing Correct Cognitive Diagnostic Models and Q-Matrices.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets