Dataset Information

Construction of a computerized adaptive test (CAT-CCNB) for efficient neurocognitive and clinical psychopathology assessment.

ABSTRACT:

Background

Traditional paper-and-pencil neurocognitive evaluations and semi-structured mental health interviews can take hours to administer and score. Computerized assessment has decreased that burden substantially, and contemporary psychometric tools such as item response theory and computerized adaptive testing (CAT) allow even further abbreviation.

New method

The goal of this paper was to describe the application of CAT and related methods to the Penn Computerized Neurocognitive Battery (CNB) and a well-validated clinical assessment in order to increase efficiency in assessment and relevant domain coverage. To calibrate item banks for CAT, N = 5053 participants (63% female; mean age 45 years, range 18-80) were collected from across the United States via crowdsourcing, providing item parameters that were then linked to larger item banks and used in individual test construction. Tests not amenable to CAT were abbreviated using complementary short-form methods.

Results

The final "CAT-CCNB" battery comprised 21 cognitive tests (compared to 14 in the original) and five adaptive clinical scales (compared to 16 in the original).

Comparison with existing methods

This new battery, derived with contemporary psychometric approaches, provides further improvements over existing assessments that use collections of fixed-length tests developed for stand-alone administration. The CAT-CCNB provides an improved version of the CNB that shows promise as a maximally efficient tool for neuropsychiatric assessment.

Conclusions

We anticipate CAT-CCNB will help satisfy the clear need for broad yet efficient measurement of cognitive and clinical domains, facilitating implementation of large-scale, "big science" approaches to data collection, and potential widespread clinical implementation.

SUBMITTER: Moore TM

PROVIDER: S-EPMC9892357 | biostudies-literature | 2023 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Construction of a computerized adaptive test (CAT-CCNB) for efficient neurocognitive and clinical psychopathology assessment.

Moore Tyler M TM Di Sandro Akira A Scott J Cobb JC Lopez Katherine C KC Ruparel Kosha K Njokweni Lucky J LJ Santra Satrajit S Conway David S DS Port Allison M AM D'Errico Lisa L Rush Sage S Wolf Daniel H DH Calkins Monica E ME Gur Raquel E RE Gur Ruben C RC

Journal of neuroscience methods 20230116

<h4>Background</h4>Traditional paper-and-pencil neurocognitive evaluations and semi-structured mental health interviews can take hours to administer and score. Computerized assessment has decreased that burden substantially, and contemporary psychometric tools such as item response theory and computerized adaptive testing (CAT) allow even further abbreviation.<h4>New method</h4>The goal of this paper was to describe the application of CAT and related methods to the Penn Computerized Neurocogniti ...[more]

PMID: 36657647

Similar Datasets

Project description:CONTEXT Unlike other areas of medicine, psychiatry is almost entirely dependent on patient report to assess the presence and severity of disease; therefore, it is particularly crucial that we find both more accurate and efficient means of obtaining that report. OBJECTIVE To develop a computerized adaptive test (CAT) for depression, called the Computerized Adaptive Test-Depression Inventory (CAT-DI), that decreases patient and clinician burden and increases measurement precision. DESIGN Case-control study. SETTING A psychiatric clinic and community mental health center. PARTICIPANTS A total of 1614 individuals with and without minor and major depression were recruited for study. MAIN OUTCOME MEASURES The focus of this study was the development of the CAT-DI. The 24-item Hamilton Rating Scale for Depression, Patient Health Questionnaire 9, and the Center for Epidemiologic Studies Depression Scale were used to study the convergent validity of the new measure, and the Structured Clinical Interview for DSM-IV was used to obtain diagnostic classifications of minor and major depressive disorder. RESULTS A mean of 12 items per study participant was required to achieve a 0.3 SE in the depression severity estimate and maintain a correlation of r = 0.95 with the total 389-item test score. Using empirically derived thresholds based on a mixture of normal distributions, we found a sensitivity of 0.92 and a specificity of 0.88 for the classification of major depressive disorder in a sample consisting of depressed patients and healthy controls. Correlations on the order of r = 0.8 were found with the other clinician and self-rating scale scores. The CAT-DI provided excellent discrimination throughout the entire depressive severity continuum (minor and major depression), whereas the traditional scales did so primarily at the extremes (eg, major depression). CONCLUSIONS Traditional measurement fixes the number of items administered and allows measurement uncertainty to vary. In contrast, a CAT fixes measurement uncertainty and allows the number of items to vary. The result is a significant reduction in the number of items needed to measure depression and increased precision of measurement.

Project description:PurposeEvaluate efficiency, precision, and validity of RetCAT, which comprises ten diabetic retinopathy (DR) quality of life (QoL) computerized adaptive tests (CATs).MethodsIn this cross-sectional clinical study, 183 English and/or Mandarin-speaking participants with DR (mean age ± standard deviation [SD] 56.4 ± 11.9 years; 38% proliferative DR [worse eye]) were recruited from retinal clinics in Singapore. Participants answered the RetCAT tests (Symptoms, Activity Limitation, Mobility, Emotional, Health Concerns, Social, Convenience, Economic, Driving, and Lighting), which were capped at seven items each, and other questionnaires, and underwent eye tests. Our primary evaluation focused on RetCAT efficiency (i.e. standard error of measurement [SEM] ± SD achieved and time needed to complete each CAT). Secondary evaluations included an assessment of RetCAT's test precision and validity.ResultsMean SEM across all RetCAT tests was 0.351, ranging from 0.272 ± 0.130 for Economic to 0.484 ± 0.130 for Emotional. Four tests (Mobility, Social, Convenience, and Driving) had a high level of measurement error. The median time to take each RetCAT test was 1.79 minutes, ranging from 1.12 (IQR [interquartile range] 1.63) for Driving to 3.28 (IQR 2.52) for Activity Limitation. Test precision was highest for participants at the most impaired end of the spectrum. Most RetCAT tests displayed expected correlations with other scales (convergent/divergent validity) and were sensitive to DR and/or vision impairment severity levels (criterion validity).ConclusionsRetCAT can provide efficient, precise, and valid measurement of DR-related QoL impact. Future application of RetCAT will employ a stopping rule based on SE rather than number of items to ensure that all tests can detect meaningful differences in person abilities. Responsiveness of RetCAT to treatment interventions must also be determined.Translational relevanceRetCAT may be useful for measuring the patient-centered impact of DR severity and disease progression and evaluating the effectiveness of new therapies.

Dataset Information

Construction of a computerized adaptive test (CAT-CCNB) for efficient neurocognitive and clinical psychopathology assessment.

Background

New method

Results

Comparison with existing methods

Conclusions

Publications

Construction of a computerized adaptive test (CAT-CCNB) for efficient neurocognitive and clinical psychopathology assessment.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets