Dataset Information

Development of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.

ABSTRACT: Traditional "paper-and-pencil" testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on 4 neurocognitive tests, and compared the concurrent, convergent, discriminant, and predictive validity of these scores with simple averaging of standardized speed and accuracy-summed scores. Concurrent validity was measured by the scores' abilities to distinguish men from women and their correlations with age; convergent and discriminant validity were measured by correlations with other scores inside and outside of their neurocognitive domains; predictive validity was measured by correlations with brain volume in regions associated with the specific neurocognitive abilities. Results provide support for the ability of itemwise efficiency scoring to detect signals as strong as those detected by standard efficiency scoring methods. We find no evidence of superior validity of the itemwise scores over traditional scores, but point out several advantages of the former. The itemwise efficiency scoring method shows promise as an alternative to standard efficiency scoring methods, with overall moderate support from tests of 4 different types of validity. This method allows the use of existing item analysis methods and provides the convenient ability to adjust the overall emphasis of accuracy versus speed in the efficiency score, thus adjusting the scoring to the real-world demands the test is aiming to fulfill. (PsycINFO Database Record

SUBMITTER: Moore TM

PROVIDER: S-EPMC4981571 | biostudies-literature | 2016 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.

Moore Tyler M TM Reise Steven P SP Roalf David R DR Satterthwaite Theodore D TD Davatzikos Christos C Bilker Warren B WB Port Allison M AM Jackson Chad T CT Ruparel Kosha K Savitt Adam P AP Baron Robert B RB Gur Raquel E RE Gur Ruben C RC

Psychological assessment 20160211 12

Traditional "paper-and-pencil" testing is imprecise in measuring speed and hence limited in assessing performance efficiency, but computerized testing permits precision in measuring itemwise response time. We present a method of scoring performance efficiency (combining information from accuracy and speed) at the item level. Using a community sample of 9,498 youths age 8-21, we calculated item-level efficiency scores on 4 neurocognitive tests, and compared the concurrent, convergent, discriminan ...[more]

PMID: 26866796

Similar Datasets

Project description:Background: An association between gaming disorder (GD) and the symptoms of common mental disorders is unraveled yet. In this preregistered study, we quantitatively synthesized reliability, convergent and discriminant validity of GD scales to examine association between GD and other constructs. Methods: Five representative GD instruments (GAS-7, AICA, IGDT-10, Lemmens IGD-9, and IGDS9-SF) were chosen based on recommendations by the previous systematic review study to conduct correlation meta-analyses and reliability generalization. A systematic literature search was conducted through Pubmed, Proquest, Embase, and Google Scholar to identify studies that reported information on either reliability or correlation with related variables. 2,124 studies were full-text assessed as of October 2020, and 184 were quantitatively synthesized. Conventional Hedges two-level meta-analytic method was utilized. Results: The result of reliability generalization reported a mean coefficient alpha of 0.86 (95% CI = 0.85-0.87) and a mean test-retest estimate of 0.86 (95% CI = 0.81-0.89). Estimated effect sizes of correlation between GD and the variables were as follows: 0.33 with depression (k = 45; number of effect sizes), 0.29 with anxiety (k = 37), 0.30 with aggression (k = 19), -0.22 with quality of life (k = 18), 0.29 with loneliness (k = 18), 0.56 with internet addiction (k = 20), and 0.40 with game playtime (k = 53), respectively. The result of moderator analyses, funnel and forest plots, and publication bias analyses were also presented. Discussion and Conclusion: All five GD instruments have good internal consistency and test-retest reliability. Relatively few studies reported the test-retest reliability. The result of correlation meta-analysis revealed that GD scores were only moderately associated with game playtime. Common psychological problems such as depression and anxiety were found to have a slightly smaller association with GD than the gaming behavior. GD scores were strongly correlated with internet addiction. Further studies should adopt a rigorous methodological procedure to unravel the bidirectional relationship between GD and other psychopathologies. Limitations: The current study did not include gray literature. The representativeness of the five tools included in the current study could be questioned. High heterogeneity is another limitation of the study. Systematic Review Registration: [https://www.crd.york.ac.uk/PROSPERO/], identifier [CRD42020219781].

Project description:BackgroundThe Minimal Eating Observation Form - Version II (MEOF-II) is a brief and easy to use screening tool for eating difficulties, that is psychometrically robust. The aim of this study was to explore convergent (measuring similar constructs) and discriminant (measuring somewhat different constructs) validity of the MEOF-II to other validated dysphagia specific, activity and participation related instruments.MethodsIn this cross-sectional study, participants (n = 100, mean age 72, n = 42 women), diagnosed with either chronic pulmonary disease, Parkinson´s disease, Multiple Sclerosis, or stroke were recruited from rehabilitation centres. Patient-reported outcomes and clinical-rated assessments, capturing eating ability in general and swallowing in specific, included: The Dysphagia Handicap Index (DHI), the 4-question test (4QT), the Minimal Eating Observation Form - II, the Volume - Viscosity Swallow Test (V-VST), Flexible Endoscopic Evaluation of Swallowing (FEES) documented according to the Penetration-Aspiration Scale (PAS). Type of oral intake was documented using the Functional Oral Intake Scale (FOIS). Activities in daily living was assessed with Barthel index (BI). Spearman's correlation coefficient was used to analyze associations. The MEOF-II total score was hypothesised to have moderate correlations (r ≥ 0.3) with the other assessments, besides with PAS and FOIS (weak correlations, r < 0.3).ResultsIn total 78 participants had any type of eating difficulties (MEOF-II), 69 reported dysphagia (4QT), 62 had dysphagia according to V-VST, 29 showed evidence of penetration/aspiration (PAS), and 31 participants had decreased oral intake ability (FOIS). The MEOF-II total score had moderate correlations with DHI, BI, 4QT, V-VST volume, and weak correlations with V-VST dysphagia and viscosity, PAS, and FOIS. Comparing a prior hypothesised correlation strengths against empirical findings showed that 83% of the hypothesised correlations were correct.ConclusionsThe MEOF-II is a holistic and objective screening tool that can indicate the need for further assessment and corresponds well with the persons' subjective experiences. MEOF-II does not specifically assess the risk for penetration/aspiration.

Dataset Information

Development of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.

Publications

Development of an itemwise efficiency scoring method: Concurrent, convergent, discriminant, and neuroimaging-based predictive validity assessed in a large community sample.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets