Multilevel Modeling of Cognitive Diagnostic Assessment: The Multilevel DINA Example.
Ontology highlight
ABSTRACT:
SUBMITTER: Wang WC
PROVIDER: S-EPMC6297912 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The within-item characteristic dependency (WICD) means that dependencies exist among different types of item characteristics/parameters within an item. The potential WICD has been ignored by current modeling approaches and estimation algorithms for the deterministic inputs noisy "and" gate (DINA) model. To explicitly model WICD, this study proposed a modified Bayesian DINA modeling approach where a bivariate normal distribution was employed as a joint prior distribution for correlated item parameters. Simulation results indicated that the model parameters were well recovered and that explicitly modeling WICD improved model parameter estimation accuracy, precision, and efficiency. In addition, when potential item blocks existed, the proposed modeling approach still demonstrated good performance and high robustness. Furthermore, the fraction subtraction data were analyzed to illustrate the application and advantage of the proposed modeling approach.
| S-EPMC6376533 | biostudies-literature
Project description:Multi-level teaching has been proven to be more effective than a one-size-fits-all learning approach. This study aimed to develop and implement a multi-level remedial teaching scheme in various high school classes containing students of a wide range of learning levels and to determine its effect of their learning. The deterministic inputs noisy and gate model of cognitive diagnosis theory was used to classify students at multiple levels according to their knowledge and desired learning outcomes. A total of 680 senior high school students from central provinces in China participated in the initial cognitive diagnostic test, and 1,615 high school sophomores from seven high schools in China participated in a formal cognitive diagnosis test. Thirty-six high school students from Southwestern China participated in the think-aloud protocols, and 258 seniors from three high schools in southwest China participated in the remedial teaching experiment. Through an analysis of students' think-aloud protocols, cognitive errors of students at all levels were determined, and multi-level remedial teaching programs were designed to address these common cognitive errors. The remedial teaching programs were then implemented in three schools and compared with a control group. The results indicated that the students in the experimental group showed a more significant improvement. In this study, the steps of designing multi-level remedial teaching include assessment, classification, and preparing a teaching scheme, which are feasible and can have remarkable teaching effects. This process can be used for reference by teachers of various subjects.
| S-EPMC8984267 | biostudies-literature
Project description:Purpose Diagnosis and classification of primary progressive aphasia (PPA) requires confirmation of specific speech and language symptoms, highlighting the important role of speech-language pathologists in the evaluation process. The purpose of this case report is to inform speech-language pathologists regarding current practices for diagnostic assessment in PPA, describing standard approaches as well as complementary, state-of-the-art procedures that may improve diagnostic precision. Method We describe the diagnostic evaluation of a 49-year-old woman with complaints of progressive word-finding difficulty. She completed standard neurological, neuropsychological, and speech-language evaluations, as well as magnetic resonance and positron emission tomography imaging of her brain. In addition, a history of developmental speech, language, and learning abilities was obtained, as well as genetic testing and assessment of cerebrospinal fluid biomarkers. We discuss the evaluation results in the context of the most current research related to PPA diagnosis. Conclusion Detailed behavioral assessment, thorough intake of symptom history and neurodevelopmental differences, multimodal neuroimaging, and comprehensive examination of genes and biomarkers are of paramount importance for detecting and characterizing PPA, with ramifications for early behavioral and/or pharmacological intervention. Supplemental Material https://doi.org/10.23641/asha.12771113.
| S-EPMC8740567 | biostudies-literature
Project description:BackgroundThe Montreal Cognitive Assessment (MoCA) is one of the most widely-used cognitive screening instruments and has been translated into several different languages and dialects. Although the original validation study suggested to use a cutoff of ≤26, subsequent studies have shown that lower cutoff values may yield fewer false-positive indications of cognitive impairment. The aim of this study was to summarize the diagnostic accuracy and mean difference of the MoCA when comparing cognitively unimpaired (CU) older adults to those with amnestic mild cognitive impairment (aMCI).MethodsPubMed and EMBASE databases were searched from inception to 22 February 2022. Meta-analyses for area under the curve (AUC) and standardized mean difference (SMD) values were performed.ResultsFifty-five observational studies that included 17,343 CU and 8,413 aMCI subjects were selected for inclusion. Thirty-nine studies were used in the AUC analysis while 44 were used in the SMD analysis. The overall AUC value was 0.84 (95% CI: 0.81, 0.87) indicating good diagnostic accuracy and a large effect size was noted for the SMD analysis (Hedge's g = 1.49, 95% CI: 1.33, 1.64). Both analyses had high levels of between-study heterogeneity. The median cutoff score for identifying aMCI was <24.Discussion and conclusionThe MoCA has good diagnostic accuracy for detecting aMCI across several different languages. The findings of this meta-analysis also support the use of 24 as the optimal cutoff when the MoCA is used to screen for suspected cognitive impairment.
| S-EPMC10896827 | biostudies-literature
Project description:BACKGROUND:Methods developed to estimate intervention effects in external target populations assume that all important effect measure modifiers have been identified and appropriately modeled. Propensity score-based diagnostics can be used to assess the plausibility of these assumptions for weighting methods. METHODS:We demonstrate the use of these diagnostics when assessing the transportability of treatment effects from the standard of care for metastatic colorectal cancer control arm in a phase III trial (HORIZON III) to a target population of 1,942 Medicare beneficiaries age 65+ years. RESULTS:In an unadjusted comparison, control arm participants had lower mortality compared with target population patients treated with the standard of care therapy (trial vs. target hazard ratio [HR] = 0.72, 95% confidence interval [CI], 0.58, 0.89). Applying inverse odds of sampling weights attenuated the trial versus target HR (weighted HR = 0.96, 95% CI = 0.73, 1.26). However, whether unadjusted or weighted, hazards did not appear proportional. At 6 months of follow-up, mortality was lower in the weighted trial population than the target population (weighted trial vs. target risk difference [RD] = -0.07, 95% CI = -0.13, -0.01), but not at 12 months (weighted RD = 0.00, 95% CI = -0.09, 0.09). CONCLUSION:These diagnostics suggest that direct transport of treatment effects from HORIZON III to the Medicare population is not valid. However, the proposed sampling model might allow valid transport of the treatment effects on longer-term mortality from HORIZON III to the Medicare population treated in clinical practice. See video abstract at, http://links.lww.com/EDE/B435.
| S-EPMC6269648 | biostudies-literature
Project description:The Asymptotic Classification Theory of Cognitive Diagnosis (ACTCD) developed by Chiu, Douglas, and Li proved that for educational test data conforming to the Deterministic Input Noisy Output "AND" gate (DINA) model, the probability that hierarchical agglomerative cluster analysis (HACA) assigns examinees to their true proficiency classes approaches 1 as the number of test items increases. This article proves that the ACTCD also covers test data conforming to the Deterministic Input Noisy Output "OR" gate (DINO) model. It also demonstrates that an extension to the statistical framework of the ACTCD, originally developed for test data conforming to the Reduced Reparameterized Unified Model or the General Diagnostic Model (a) is valid also for both the DINA model and the DINO model and (b) substantially increases the accuracy of HACA in classifying examinees when the test data conform to either of these two models.
| S-EPMC5978609 | biostudies-literature
Project description:Given advantages in reperfusion therapy leading to mild stroke, less apparent cognitive deficits can be overseen in a routine neurological examination. Despite the widespread use of the Montreal Cognitive Assessment (MoCA), age- and education-specific cutoffs for the detection of post-stroke cognitive impairment (PSCI) are not established, hampering its valid application in stroke. We aimed to establish age- and education-specific MoCA cutoffs to better discriminate patients with and without acute PSCI. Patients with acute ischemic stroke underwent the MoCA and a detailed neuropsychological assessment. PSCI was defined as a performance < - 1.5 SD in ≥ 2 cognitive domains. As secondary data analysis, the discriminant abilities of the MoCAraw-score (not adding + 1 as correction for ≤ 12 years of education, YoE) cutoffs were automatically derived based on Youden Index and evaluated by receiver operating characteristic analyses across age- (< 55, 55-70, > 70 years old) and education-specific (≤ 12 and > 12 YoE) groups. 351 stroke patients (67.4 ± 14.1 years old; 13.1 ± 2.8 YoE) underwent the neuropsychological assessment 2.7 ± 2.0 days post-stroke. The original MoCA cutoff < 26 falsely classified 26.2% of examined patients, with poor sensitivity in younger adults (34.8% in patients < 55 years > 12 YoE) and poor specificity in older adults (55.0%, in > 70 years ≤ 12 YoE). By maximizing both sensitivity and specificity, the optimal MoCAraw cutoffs were: (i) < 28 in patients aged < 55 with > 12 YoE (sensitivity = 69.6%, specificity = 77.8%); (ii) < 22 and < 25 in patients > 70 years with ≤ 12 and > 12 YoE (sensitivity = 61.6%, specificity = 90.0%; sensitivity = 63.3%, specificity = 84.0%, respectively). In other groups the optimal MoCAraw cutoff was < 26. Age and education level should be considered when interpreting MoCA-scores. Though new age- and education-specific cutoffs demonstrated higher discriminant ability for PSCI, their performance in young stroke and adults with higher education level was low due to ceiling effects and MoCA subtests structure, and cautious interpretation in these patients is warranted.Trial registration: ClinicalTrials.gov Identifier: NCT05653141.
| S-EPMC11362592 | biostudies-literature
Project description:Affect regulation models state that affect both motivates and reinforces alcohol use. We aimed to examine whether affect levels and rates of change differed across drinking versus nondrinking days in a manner consistent with affect regulation models. Four hundred four regularly drinking adults, aged 18-70 years, completed ecological momentary assessments over 3 weeks. Participants provided positive affect (PA; enthusiastic, excited, happy) and negative affect (NA; distressed, sad) reports during all prompts; alcohol consumption reports were also provided. Multilevel spline models revealed that on drinking days, PA was higher and NA was lower both before and after drinking compared to matched times on nondrinking days. PA and NA were also higher and lower, respectively, both before and after drinking, when heavy drinking days were compared to moderate drinking days. Examination of affect rates of change revealed that (a) accelerating increases in PA and accelerating decreases in NA preceded drinking initiation, (b) PA increases and NA decreases were seen up to 2 hr after drinking initiation, and (c) pre- and postdrinking PA increases were larger on heavy versus moderate drinking days, whereas only postdrinking NA decreases were larger on heavy drinking days. Results supported affect regulation models while adding nuance, showing accelerating changes in predrinking affect on drinking days and pre- and postdrinking differences in affect levels and rates of change across days of varying drinking intensity. Beyond theory, our results suggest that accelerating changes in affect may provide a clue to future commencement of heavy drinking, which may aid momentary intervention development. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
| S-EPMC7148183 | biostudies-literature
Project description:The primary purpose of this study is to provide an overview of multilevel modeling for Psychosomatic Medicine readers and contributors. The article begins with a general introduction to multilevel modeling. Multilevel regression modeling at two levels is emphasized because of its prevalence in psychosomatic medicine research. Simulated data sets based on some core ideas from the Familias Unidas effectiveness study are used to illustrate key concepts including communication of model specification, parameter interpretation, sample size and power, and missing data. Input and key output files from Mplus and SAS are provided. A cluster randomized trial with repeated measures (i.e., three-level regression model) is then briefly presented with simulated data based on some core ideas from a cognitive-behavioral stress management intervention in prostate cancer.
| S-EPMC3498540 | biostudies-literature
Project description:As a new generation of measurement theory, cognitive diagnosis theory shows significant potential and advantages in educational evaluation in that it combines a cognitive process and a measurement method. The application of the theory not only reveals the potential characteristics of learners in cognitive processing, but also provides targeted remedies and strategic guidance for individuals. Given the difficulties of traditional assessment models in providing an insightful and fine-grained account for individualized and procedural learning, providing personalized learning strategies for learners of Chinese as a second language has been a new goal of teaching and measurement in Chinese listening. This study constructs a cognitive diagnosis model of Chinese listening comprehension for Chinese-as-a-second-language learners through theoretical exploration, model hypotheses, repeated verification, and model modification. The results show that the Q-matrix (Q3) constructed by the experts within modification has the highest fitting degree with the empirical data. The parameter recovery rate, the accuracy of the tested attribute or mode, and the relative fitting index obtained from the simulation study are consistent with the information extracted from the empirical data. The diagnostic reliability and effectiveness of generalized deterministic inputs, noise "and" gate (G-DINA) are higher than those of DINA, deterministic inputs, noisy "or" gate (DINO), and reduced reparametrized unified model (RRUM). In the estimation of the item and subject parameters, the G-DINA model shows good convergence, and the average classification accuracy rate based on attribute level is 0.861.
| S-EPMC8452943 | biostudies-literature