Dataset Information

Development and public release of the Penn Reading Assessment Computerized Adaptive Test (PRA-CAT) for premorbid IQ.

ABSTRACT: An important component of neuropsychological testing is assessment of premorbid intelligence to estimate a patient's ability independent of neurological impairment. A common test of premorbid IQ-namely, the Reading section of the Wide Range Achievement Test (WRAT)-has been shown to have high measurement error in the high ability range, is unnecessarily long (55 items), and is proprietary. We describe the development of an alternative, nonproprietary, computerized adaptive test for premorbid IQ, the Penn Reading Assessment (PRA-CAT). PRA-CAT items were calibrated using a 1-parameter item response theory model in a large community sample (N = 9,498), Ages 8 to 21, and the resulting parameters were used to simulate computerized adaptive testing sessions. Simulations demonstrated that the PRA-CAT achieves low measurement error (0.25; equivalent to Cronbach's alpha = .94) and acceptable measurement error (0.40; Cronbach's alpha = .84) after only 18 and 6 items, respectively (on average). Correlation of WRAT and PRA-CAT scores with numerous clinical, cognitive, demographic, and neuroimaging criteria suggests that validity of PRA-CAT score interpretation is comparable (and sometimes superior) with the WRAT. The fully functioning PRA-CAT for public use (including item parameter estimates reported here) has been built using the open-source program Concerto, and can be installed by anyone on a local computer or on the "cloud." Given the length and proprietary nature of the WRAT, the PRA-CAT shows promise as a potential alternative (and with minimal or no cost). Further validation in the context of neurological injury is needed. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

SUBMITTER: Yudien MA

PROVIDER: S-EPMC6706308 | biostudies-literature | 2019 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and public release of the Penn Reading Assessment Computerized Adaptive Test (PRA-CAT) for premorbid IQ.

Yudien Mikhal A MA Moore Tyler M TM Port Allison M AM Ruparel Kosha K Gur Raquel E RE Gur Ruben C RC

Psychological assessment 20190613 9

An important component of neuropsychological testing is assessment of premorbid intelligence to estimate a patient's ability independent of neurological impairment. A common test of premorbid IQ-namely, the Reading section of the Wide Range Achievement Test (WRAT)-has been shown to have high measurement error in the high ability range, is unnecessarily long (55 items), and is proprietary. We describe the development of an alternative, nonproprietary, computerized adaptive test for premorbid IQ, ...[more]

PMID: 31192630

Similar Datasets

Project description:CONTEXT Unlike other areas of medicine, psychiatry is almost entirely dependent on patient report to assess the presence and severity of disease; therefore, it is particularly crucial that we find both more accurate and efficient means of obtaining that report. OBJECTIVE To develop a computerized adaptive test (CAT) for depression, called the Computerized Adaptive Test-Depression Inventory (CAT-DI), that decreases patient and clinician burden and increases measurement precision. DESIGN Case-control study. SETTING A psychiatric clinic and community mental health center. PARTICIPANTS A total of 1614 individuals with and without minor and major depression were recruited for study. MAIN OUTCOME MEASURES The focus of this study was the development of the CAT-DI. The 24-item Hamilton Rating Scale for Depression, Patient Health Questionnaire 9, and the Center for Epidemiologic Studies Depression Scale were used to study the convergent validity of the new measure, and the Structured Clinical Interview for DSM-IV was used to obtain diagnostic classifications of minor and major depressive disorder. RESULTS A mean of 12 items per study participant was required to achieve a 0.3 SE in the depression severity estimate and maintain a correlation of r = 0.95 with the total 389-item test score. Using empirically derived thresholds based on a mixture of normal distributions, we found a sensitivity of 0.92 and a specificity of 0.88 for the classification of major depressive disorder in a sample consisting of depressed patients and healthy controls. Correlations on the order of r = 0.8 were found with the other clinician and self-rating scale scores. The CAT-DI provided excellent discrimination throughout the entire depressive severity continuum (minor and major depression), whereas the traditional scales did so primarily at the extremes (eg, major depression). CONCLUSIONS Traditional measurement fixes the number of items administered and allows measurement uncertainty to vary. In contrast, a CAT fixes measurement uncertainty and allows the number of items to vary. The result is a significant reduction in the number of items needed to measure depression and increased precision of measurement.

Project description:In clinical neuropsychology the cognitive abilities of neurological patients are commonly estimated using well-established paper-based tests. Typically, scores on some tests remain relatively well preserved, whilst others exhibit a significant and disproportionate decline. Scores on those tests that measure preserved cognitive functions (so-called 'hold' tests) may be used to estimate premorbid abilities, including scores in non-hold tests that would have been expected prior to the onset of cognitive impairment. Many hold tests entail word reading, with each word being graded as correctly or incorrectly pronounced. Inevitably, such tests are likely to contain words that provide little or no diagnostic power (i.e., can be eliminated without negatively affecting prediction accuracy). In this paper, a genetic algorithm is developed and demonstrated, using n = 92 neurologically healthy participants, to identify optimal word subsets from the National Adult Reading Test that minimize the mean error in predicting the most widely used clinical measure of IQ and cognitive ability, the Wechsler Adult Intelligence Scale Fourth Edition IQ. In addition to requiring only 17-20 of the original 50 words (suggesting that this test could be revised to be up to 66% shorter) and minimizing mean prediction error, the algorithm increases the proportion of the variance in the predicted variable explained in comparison to using all words (from r2 = 0.46 to r2 = 0.61). In a clinical setting this would improve estimates of premorbid cognitive function and, if an abbreviated revision to this test were to be adopted, reduce the arduousness of the test for patients. The proposed method is evaluated with jackknifing and leave one out cross validation. The general approach may be used to optimize the relationship between any two psychological tests by finding the question subset in one test that minimizes the prediction error in a second test by training the genetic algorithm using data collected from participants upon whom both tests have been administered. This approach may also be used to develop new predictive tests, since it provides a method to identify an optimal subset of a set of candidate questions (for which empirical data have been collected) that maximizes prediction accuracy and the proportion of variance in the predicted variable that can be explained.

Project description:Background and aimsThe focus of this paper is on the improvement of substance use disorder (SUD) screening and measurement. Using a multi-dimensional item response theory model, the bifactor model, we provide a psychometric harmonization between SUD, depression, anxiety, trauma, social isolation, functional impairment and risk-taking behavior symptom domains, providing a more balanced view of SUD. The aims are to (1) develop the item-bank, (2) calibrate the item-bank using a bifactor model that includes a primary dimension and symptom-specific subdomains, (3) administer using computerized adaptive testing (CAT) and (4) validate the CAT-SUD in Spanish and English in the United States and Spain.DesignItem bank construction, item calibration phase, CAT-SUD validation phase.SettingPrimary care, community clinics, emergency departments and patient-to-patient referrals in Spain (Barcelona and Madrid) and the United States (Boston and Los Angeles).Participants/casesCalibration phase: the CAT-SUD was developed via simulation from complete item responses in 513 participants. Validation phase: 297 participants received the Composite International Diagnostic Interview (CIDI) and the CAT-SUD.MeasurementsA total of 252 items from five subdomains: (1) SUD, (2) psychological disorders, (3) risky behavior, (4) functional impairment and (5) social support. CAT-SUD scale scores and CIDI SUD diagnosis.FindingsCalibration: the bifactor model provided excellent fit to the multi-dimensional item bank; 168 items had high loadings (> 0.4 with the majority > 0.6) on the primary SUD dimension. Using an average of 11 items (four to 26), which represents a 94% reduction in respondent burden (average administration time of approximately 2 minutes), we found a correlation of 0.91 with the 168-item scale (precision of 5 points on a 100-point scale).Validationstrong agreement was found between the primary CAT-SUD dimension estimate and the results of a structured clinical interview. There was a 20-fold increase in the likelihood of a CIDI SUD diagnosis across the range of the CAT-SUD (AUC = 0.85).ConclusionsWe have developed a new approach for the screening and measurement of SUD and related severity based on multi-dimensional item response theory. The bifactor model harmonized information from mental health, trauma, social support and traditional SUD items to provide a more complete characterization of SUD. The CAT-SUD is highly predictive of a current SUD diagnosis based on a structured clinical interview, and may be predictive of the development of SUD in the future.

Dataset Information

Development and public release of the Penn Reading Assessment Computerized Adaptive Test (PRA-CAT) for premorbid IQ.

Publications

Development and public release of the Penn Reading Assessment Computerized Adaptive Test (PRA-CAT) for premorbid IQ.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets