Dataset Information

Self-administered Web-Based Tests of Executive Functioning and Perceptual Speed: Measurement Development Study With a Large Probability-Based Survey Panel.

ABSTRACT:

Background

Cognitive testing in large population surveys is frequently used to describe cognitive aging and determine the incidence rates, risk factors, and long-term trajectories of the development of cognitive impairment. As these surveys are increasingly administered on internet-based platforms, web-based and self-administered cognitive testing calls for close investigation.

Objective

Web-based, self-administered versions of 2 age-sensitive cognitive tests, the Stop and Go Switching Task for executive functioning and the Figure Identification test for perceptual speed, were developed and administered to adult participants in the Understanding America Study. We examined differences in cognitive test scores across internet device types and the extent to which the scores were associated with self-reported distractions in everyday environments in which the participants took the tests. In addition, national norms were provided for the US population.

Methods

Data were collected from a probability-based internet panel representative of the US adult population-the Understanding America Study. Participants with access to both a keyboard- and mouse-based device and a touch screen-based device were asked to complete the cognitive tests twice in a randomized order across device types, whereas participants with access to only 1 type of device were asked to complete the tests twice on the same device. At the end of each test, the participants answered questions about interruptions and potential distractions that occurred during the test.

Results

Of the 7410 (Stop and Go) and 7216 (Figure Identification) participants who completed the device ownership survey, 6129 (82.71% for Stop and Go) and 6717 (93.08% for Figure Identification) participants completed the first session and correctly responded to at least 70% of the trials. On average, the standardized differences across device types were small, with the absolute value of Cohen d ranging from 0.05 (for the switch score in Stop and Go and the Figure Identification score) to 0.13 (for the nonswitch score in Stop and Go). Poorer cognitive performance was moderately associated with older age (the absolute value of r ranged from 0.32 to 0.61), and this relationship was comparable across device types (the absolute value of Cohen q ranged from 0.01 to 0.17). Approximately 12.72% (779/6123 for Stop and Go) and 12.32% (828/6721 for Figure Identification) of participants were interrupted during the test. Interruptions predicted poorer cognitive performance (P<.01 for all scores). Specific distractions (eg, watching television and listening to music) were inconsistently related to cognitive performance. National norms, calculated as weighted average scores using sampling weights, suggested poorer cognitive performance as age increased.

Conclusions

Cognitive scores assessed by self-administered web-based tests were sensitive to age differences in cognitive performance and were comparable across the keyboard- and touch screen-based internet devices. Distraction in everyday environments, especially when interrupted during the test, may result in a nontrivial bias in cognitive testing.

SUBMITTER: Liu Y

PROVIDER: S-EPMC9127643 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Background: Cerebral neuroplasticity is compromised due to substance abuse. There is damage to neuronal areas that are involved in memory and executive functioning. Treatments with worse outcomes are often associated with cognitive deficits that have resulted from substance dependence. However, there is evidence that cognitive training can lead to improvements in cognitive functions and can be useful when treating addictions. This systematic review aims to synthesize evidence on the effectiveness of cognitive training in memory, executive functioning, and processing speed in individuals with substance use disorder (SUD). Methods: The Joanna Briggs Institute's PICO strategy was used to develop this systematic literature review. Four databases were searched (PubMed, the Cochrane Library, Web of Science, and PsycINFO) to identify controlled randomized clinical studies and quasi-experimental studies, in English, Portuguese, and Spanish, from 1985 to 2019. The literature found was examined by two independent reviewers, who assessed the quality of studies that met the inclusion criteria. The Cochrane risk-of-bias tool for the randomized controlled trials and the ROBINS-I tool for non-randomized studies were used to assess the risk of bias. In data extraction, the Cochrane Handbook for Systematic Reviews was considered. Results: From a total of 470 studies, 319 were selected for analysis after the elimination of duplicates. According to the inclusion criteria defined, 26 studies were eligible and evaluated. An evaluation was performed considering the participant characteristics, countries, substance type, study and intervention details, and key findings. Of the 26 selected studies, 14 considered only alcoholics, six included participants with various SUD (alcohol and other substances), three exclusively looked into methamphetamine-consuming users and another three into opioid/methadone users. Moreover, 18 studies found some kind of cognitive improvement, with two of these reporting only marginally significant effects. One study found improvements only in measures similar to the training tasks, and two others had ambiguous results. Conclusions: The included studies revealed the benefits of cognitive training with regard to improving cognitive functions in individuals with SUD. Memory was the most scrutinized cognitive function in this type of intervention, and it is also one of the areas most affected by substance use. Systematic Review Registration: [PROSPERO], identifier [CRD42020161039].

Project description:Probability matching is a classic theory of decision making that was first developed in models of cognition. Posterior probability matching, a variant in which observers match their response probabilities to the posterior probability of each response being correct, is being used increasingly often in models of perception. However, little is known about whether posterior probability matching is consistent with the vast literature on vision and hearing that has developed within signal detection theory. Here we test posterior probability matching models using two tools from detection theory. First, we examine the models' performance in a two-pass experiment, where each block of trials is presented twice, and we measure the proportion of times that the model gives the same response twice to repeated stimuli. We show that at low performance levels, posterior probability matching models give highly inconsistent responses across repeated presentations of identical trials. We find that practised human observers are more consistent across repeated trials than these models predict, and we find some evidence that less practised observers more consistent as well. Second, we compare the performance of posterior probability matching models on a discrimination task to the performance of a theoretical ideal observer that achieves the best possible performance. We find that posterior probability matching is very inefficient at low-to-moderate performance levels, and that human observers can be more efficient than is ever possible according to posterior probability matching models. These findings support classic signal detection models, and rule out a broad class of posterior probability matching models for expert performance on perceptual tasks that range in complexity from contrast discrimination to symmetry detection. However, our findings leave open the possibility that inexperienced observers may show posterior probability matching behaviour, and our methods provide new tools for testing for such a strategy.