Dataset Information

Development of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.

ABSTRACT: Traditional "paper-and-pencil" testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on 4 neurocognitive tests, and compared the concurrent, convergent, discriminant, and predictive validity of these scores with simple averaging of standardized speed and accuracy-summed scores. Concurrent validity was measured by the scores' abilities to distinguish men from women and their correlations with age; convergent and discriminant validity were measured by correlations with other scores inside and outside of their neurocognitive domains; predictive validity was measured by correlations with brain volume in regions associated with the specific neurocognitive abilities. Results provide support for the ability of itemwise efficiency scoring to detect signals as strong as those detected by standard efficiency scoring methods. We find no evidence of superior validity of the itemwise scores over traditional scores, but point out several advantages of the former. The itemwise efficiency scoring method shows promise as an alternative to standard efficiency scoring methods, with overall moderate support from tests of 4 different types of validity. This method allows the use of existing item analysis methods and provides the convenient ability to adjust the overall emphasis of accuracy versus speed in the efficiency score, thus adjusting the scoring to the real-world demands the test is aiming to fulfill. (PsycINFO Database Record

SUBMITTER: Moore TM

PROVIDER: S-EPMC4981571 | biostudies-literature | 2016 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.

Moore Tyler M TM Reise Steven P SP Roalf David R DR Satterthwaite Theodore D TD Davatzikos Christos C Bilker Warren B WB Port Allison M AM Jackson Chad T CT Ruparel Kosha K Savitt Adam P AP Baron Robert B RB Gur Raquel E RE Gur Ruben C RC

Psychological assessment 20160211 12

Traditional "paper-and-pencil" testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on 4 neurocognitive tests, and compared the concurrent, convergent, discriminan ...[more]

PMID: 26866796

Similar Datasets

Project description:Background: An association between gaming disorder (GD) and the symptoms of common mental disorders is unraveled yet. In this preregistered study, we quantitatively synthesized reliability, convergent and discriminant validity of GD scales to examine association between GD and other constructs. Methods: Five representative GD instruments (GAS-7, AICA, IGDT-10, Lemmens IGD-9, and IGDS9-SF) were chosen based on recommendations by the previous systematic review study to conduct correlation meta-analyses and reliability generalization. A systematic literature search was conducted through Pubmed, Proquest, Embase, and Google Scholar to identify studies that reported information on either reliability or correlation with related variables. 2,124 studies were full-text assessed as of October 2020, and 184 were quantitatively synthesized. Conventional Hedges two-level meta-analytic method was utilized. Results: The result of reliability generalization reported a mean coefficient alpha of 0.86 (95% CI = 0.85-0.87) and a mean test-retest estimate of 0.86 (95% CI = 0.81-0.89). Estimated effect sizes of correlation between GD and the variables were as follows: 0.33 with depression (k = 45; number of effect sizes), 0.29 with anxiety (k = 37), 0.30 with aggression (k = 19), -0.22 with quality of life (k = 18), 0.29 with loneliness (k = 18), 0.56 with internet addiction (k = 20), and 0.40 with game playtime (k = 53), respectively. The result of moderator analyses, funnel and forest plots, and publication bias analyses were also presented. Discussion and Conclusion: All five GD instruments have good internal consistency and test-retest reliability. Relatively few studies reported the test-retest reliability. The result of correlation meta-analysis revealed that GD scores were only moderately associated with game playtime. Common psychological problems such as depression and anxiety were found to have a slightly smaller association with GD than the gaming behavior. GD scores were strongly correlated with internet addiction. Further studies should adopt a rigorous methodological procedure to unravel the bidirectional relationship between GD and other psychopathologies. Limitations: The current study did not include gray literature. The representativeness of the five tools included in the current study could be questioned. High heterogeneity is another limitation of the study. Systematic Review Registration: [https://www.crd.york.ac.uk/PROSPERO/], identifier [CRD42020219781].

Dataset Information

Development of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.

Publications

Development of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets