Dataset Information

Clarifying the Effect of Test Speededness.

ABSTRACT: In the context of high-stakes tests, test takers who do not have enough time to complete a test rush toward the end and may engage in speeded behavior when tests do not penalize guessing. Using mathematical derivations and simulations, previous research showed that random guessing responses should attenuate interitem correlations, and therefore, decrease estimates of reliability. Meanwhile, other researchers showed that random guessing could in fact inflate reliability estimates using real data. We provide analytical derivations on how speededness could affect correlations between two dichotomous items in multiple ways, depending on the manifestation and prevalence of test speededness. Furthermore, we provide two simulation studies that evaluate the magnitude of impact of test speededness on interitem correlations and Cronbach's alpha. We found that the impact of test speededness can vary between item pairs and that it depends on the manifestation of test speededness and item level characteristics. Furthermore, speeded responses will, in general, attenuate or not affect reliability estimates, depending on the prevalence of such responses and conceptual interpretation of speeded responses. Implications of the findings are discussed.

SUBMITTER: Hong MR

PROVIDER: S-EPMC6745631 | biostudies-literature | 2019 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Clarifying the Effect of Test Speededness.

Hong Maxwell R MR Cheng Ying Y

Applied psychological measurement 20181219 8

In the context of high-stakes tests, test takers who do not have enough time to complete a test rush toward the end and may engage in speeded behavior when tests do not penalize guessing. Using mathematical derivations and simulations, previous research showed that random guessing responses should attenuate interitem correlations, and therefore, decrease estimates of reliability. Meanwhile, other researchers showed that random guessing could in fact inflate reliability estimates using real data. ...[more]

PMID: 31551639

Similar Datasets

Project description:BackgroundConsensus guidelines have recommended that decision aids include a process for helping patients clarify their values. We sought to examine the theoretical and empirical evidence related to the use of values clarification methods in patient decision aids.MethodsBuilding on the International Patient Decision Aid Standards (IPDAS) Collaboration's 2005 review of values clarification methods in decision aids, we convened a multi-disciplinary expert group to examine key definitions, decision-making process theories, and empirical evidence about the effects of values clarification methods in decision aids. To summarize the current state of theory and evidence about the role of values clarification methods in decision aids, we undertook a process of evidence review and summary.ResultsValues clarification methods (VCMs) are best defined as methods to help patients think about the desirability of options or attributes of options within a specific decision context, in order to identify which option he/she prefers. Several decision making process theories were identified that can inform the design of values clarification methods, but no single "best" practice for how such methods should be constructed was determined. Our evidence review found that existing VCMs were used for a variety of different decisions, rarely referenced underlying theory for their design, but generally were well described in regard to their development process. Listing the pros and cons of a decision was the most common method used. The 13 trials that compared decision support with or without VCMs reached mixed results: some found that VCMs improved some decision-making processes, while others found no effect.ConclusionsValues clarification methods may improve decision-making processes and potentially more distal outcomes. However, the small number of evaluations of VCMs and, where evaluations exist, the heterogeneity in outcome measures makes it difficult to determine their overall effectiveness or the specific characteristics that increase effectiveness.

Dataset Information

Clarifying the Effect of Test Speededness.

Publications

Clarifying the Effect of Test Speededness.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets