Dataset Information

Theorems and Methods of a Complete Q Matrix With Attribute Hierarchies Under Restricted Q-Matrix Design.

ABSTRACT: The design of test Q matrix can directly influence the classification accuracy of a cognitive diagnostic assessment. In this paper, we focus on Q matrix design when attribute hierarchies are known prior to test development. A complete Q matrix design is proposed and theorems are presented to demonstrate that it is a necessary and sufficient condition to guarantee the identifiability of ideal response patterns. A simulation study is also conducted to detect the effects of the proposed design on a family of conjunctive diagnostic models. The results revealed that the proposed Q matrix design is the key condition for guaranteeing classification accuracy. When only one type of item pattern in R matrix is missing from the associated test Q matrix, the related attribute-wise agreement rate will decrease dramatically. When the entire R matrix is missing, both the pattern-wise and attribute-wise agreement rates will decrease sharply. This indicates that the proposed procedures for complete Q matrix design with attribute hierarchies can serve as guidelines for test blueprint development prior to item writing in a cognitive diagnostic assessment.

SUBMITTER: Cai Y

PROVIDER: S-EPMC6092632 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Theorems and Methods of a Complete Q Matrix With Attribute Hierarchies Under Restricted Q-Matrix Design.

Cai Yan Y Tu Dongbo D Ding Shuliang S

Frontiers in psychology 20180808

The design of test Q matrix can directly influence the classification accuracy of a cognitive diagnostic assessment. In this paper, we focus on Q matrix design when attribute hierarchies are known prior to test development. A complete Q matrix design is proposed and theorems are presented to demonstrate that it is a necessary and sufficient condition to guarantee the identifiability of ideal response patterns. A simulation study is also conducted to detect the effects of the proposed design on a ...[more]

PMID: 30135672

Similar Datasets

Project description:In a 2-alternative forced-choice (2AFC) discrimination task, observers choose which of two stimuli has the higher value. The psychometric function for this task gives the probability of a correct response for a given stimulus difference, ?x. This paper proves four theorems about the psychometric function. Assuming the observer applies a transducer and adds noise, Theorem 1 derives a convenient general expression for the psychometric function. Discrimination data are often fitted with a Weibull function. Theorem 2 proves that the Weibull "slope" parameter, ?, can be approximated by ?(Noise) x ?(Transducer), where ?(Noise) is the ? of the Weibull function that fits best to the cumulative noise distribution, and ?(Transducer) depends on the transducer. We derive general expressions for ?(Noise) and ?(Transducer), from which we derive expressions for specific cases. One case that follows naturally from our general analysis is Pelli's finding that, when d' ? (?x)(b), ? ? ?(Noise) x b. We also consider two limiting cases. Theorem 3 proves that, as sensitivity improves, 2AFC performance will usually approach that for a linear transducer, whatever the actual transducer; we show that this does not apply at signal levels where the transducer gradient is zero, which explains why it does not apply to contrast detection. Theorem 4 proves that, when the exponent of a power-function transducer approaches zero, 2AFC performance approaches that of a logarithmic transducer. We show that the power-function exponents of 0.4-0.5 fitted to suprathreshold contrast discrimination data are close enough to zero for the fitted psychometric function to be practically indistinguishable from that of a log transducer. Finally, Weibull ? reflects the shape of the noise distribution, and we used our results to assess the recent claim that internal noise has higher kurtosis than a Gaussian. Our analysis of ? for contrast discrimination suggests that, if internal noise is stimulus-independent, it has lower kurtosis than a Gaussian.

Project description:BACKGROUND: Experimental designs constitute a vital component of all Stated Choice (aka discrete choice experiment) studies. However, there exists limited empirical evaluation of the statistical benefits of Stated Choice (SC) experimental designs that employ non-zero prior estimates in constructing non-orthogonal constrained designs. This paper statistically compares the performance of contrasting SC experimental designs. In so doing, the effect of respondent literacy on patterns of Attribute non-Attendance (ANA) across fractional factorial orthogonal and efficient designs is also evaluated. The study uses a 'real' SC design to model consumer choice of primary health care providers in rural north India. A total of 623 respondents were sampled across four villages in Uttar Pradesh, India. METHODS: Comparison of orthogonal and efficient SC experimental designs is based on several measures. Appropriate comparison of each design's respective efficiency measure is made using D-error results. Standardised Akaike Information Criteria are compared between designs and across recall periods. Comparisons control for stated and inferred ANA. Coefficient and standard error estimates are also compared. RESULTS: The added complexity of the efficient SC design, theorised elsewhere, is reflected in higher estimated amounts of ANA among illiterate respondents. However, controlling for ANA using stated and inferred methods consistently shows that the efficient design performs statistically better. Modelling SC data from the orthogonal and efficient design shows that model-fit of the efficient design outperform the orthogonal design when using a 14-day recall period. The performance of the orthogonal design, with respect to standardised AIC model-fit, is better when longer recall periods of 30-days, 6-months and 12-months are used. CONCLUSIONS: The effect of the efficient design's cognitive demand is apparent among literate and illiterate respondents, although, more pronounced among illiterate respondents. This study empirically confirms that relaxing the orthogonality constraint of SC experimental designs increases the information collected in choice tasks, subject to the accuracy of the non-zero priors in the design and the correct specification of a 'real' SC recall period.

Dataset Information

Theorems and Methods of a Complete Q Matrix With Attribute Hierarchies Under Restricted Q-Matrix Design.

Publications

Theorems and Methods of a Complete Q Matrix With Attribute Hierarchies Under Restricted Q-Matrix Design.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure