Dataset Information

The validity and reliability of a digital Ruff Figural Fluency Test (RFFT).

ABSTRACT:

Background

The Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validity and reliability of this digital RFFT.

Methods

We randomly allocated participants to the digital or paper-and-pencil RFFT. After the first test, the other test was performed immediately (cross-over). Participants were invited for a second digital RFFT 1 week later. For the digital RFFT, an (automatic) algorithm and two independent raters (criterion standard) assessed the number of unique designs (UD) and perseverative errors (PE). These raters also assessed the paper-and-pencil RFFT. We used Intraclass correlation coefficients (ICC), sensitivity, specificity, %-agreement, Kappa, and Bland-Altman plots.

Results

We included 94 participants (mean (SD) age 39.9 (14.8), 73.4% follow-up). Mean (SD) UD and median (IQR) PE of the digital RFFT were 84.2 (26.0) and 4 (2-7.3), respectively. Agreement between manual and automatic scoring of the digital RFFT was high for UD (ICC = 0.99, 95% CI 0.98, 0.99, sensitivity = 0.98; specificity = 0.96) and PE (ICC = 0.99, 95% CI 0.98, 0.99; sensitivity = 0.90, specificity = 1.00), indicating excellent criterion validity. Small but significant differences in UD were found between the automatic and manual scoring (mean difference: - 1.12, 95% CI - 1.92, - 0.33). Digital and paper-and-pencil RFFT had moderate agreement for UD (ICC = 0.73, 95% CI 0.34, 0.87) and poor agreement for PE (ICC = 0.47, 95% CI 0.30, 0.62). Participants had fewer UD on the digital than paper-and-pencil RFFT (mean difference: - 7.09, 95% CI - 11.80, - 2.38). The number of UD on the digital RFFT was associated with higher education (Spearman's r = 0.43, p < 0.001), and younger age (Pearson's r = - 0.36, p < 0.001), showing its ability to discriminate between different age categories and levels of education. Test-retest reliability was moderate (ICC = 0.74, 95% CI 0.61, 0.83).

Conclusions

The automatic scoring of the digital RFFT has good criterion and convergent validity. There was low agreement between the digital RFFT and paper-and-pencil RFFT and moderate test-retest reliability, which can be explained by learning effects. The digital RFFT is a valid and reliable instrument to measure executive cognitive function among the general population and is a feasible alternative to the paper-and-pencil RFFT in large-scale studies. However, its scores cannot be used interchangeably with the paper-and-pencil RFFT scores.

SUBMITTER: Vrijsen J

PROVIDER: S-EPMC8080381 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe Ruff Figural Fluency Test (RFFT; a pencil and paper test) and the CogState (a computerized cognitive test battery) are well-validated and suitable tests to evaluate cognitive functioning in large observational studies at the population level. The LifeLines Cohort Study includes the RFFT as baseline measurement and incorporated the CogState as replacement for the RFFT at follow-up. It is unknown how these two tests relate to each other. Therefore, the aim of this study is to examine the correlation between the RFFT and the CogState and the impact of demographic characteristics on this association.MethodsA subcohort of the LifeLines Cohort Study, a large population based cohort study, participated in this study. Correlations between the RFFT and six subtasks of the CogState were examined. Subgroup analyses were performed to investigate the influence of age, education, and gender on the results. With sensitivity analyses we investigated the influence of computer experience and (physical) impairments.ResultsA total of 509 participants (mean age (SD): 53 years (14.6); range 18-87 years) participated in this study. All correlations between the RFFT and the CogState were statistically significant (except for the correlation between the RFFT error ratio and the CogState One Back Task), ranging from -0.39 to 0.28. Stratifying the analyses for age, education, and gender did not substantially affect our conclusions. Sensitivity analyses showed no substantial influence of level of computer experience or (physical) impairments.ConclusionsCorrelations found in the present study were only weak to moderate, indicating that cognitive functioning measured by the RFFT does not measure the same components of cognitive functioning as six subtasks of the CogState. Computerized testing such as the CogState may be very well suited for large cohort studies to assess cognitive functioning in the general population and to identify cognitive changes as early as possible, as it is a less time- and labor intensive tool.

Project description:ObjectiveTo evaluate the convergent validity of the six-minute stepper test (6MST) with the variables used in the diagnosis of sarcopenia (appendicular muscle mass, handgrip strength and six-meter gait speed test), as well as to evaluate test-retest reliability and safety when applied to hospitalized elderly patients. Finally, we aimed to compare the performance in the 6MST between hospitalized elderly patients and healthy elderly from the community.Materials and methodsObservational and cross-sectional study. Elderly patients admitted to a private hospital and healthy elderly from the community were recruited. On the first day, the patients included underwent the following assessments: anthropometric, handgrip strength (HGS), six-meter gait speed test (6GST) and 6MST. On the second day, before breakfast, patients underwent body composition assessment. The healthy elderly were evaluated on a single day and performed only anthropometric assessment and 6MST.Results30 hospitalized patients (age 71.0±7.9 years) and 15 healthy elderly (age 68.1±5.8 years) were included. There was a high correlation of 6MST with 6GST (r = 0.78; p<0.001), moderate correlation with HGS (r = 0.59; p<0.001) and low correlation with appendicular muscle mass (r = 0.45; p = 0.01). There was no statistical difference between the first and second 6MST performed by hospitalized elderly (196.2±91.0 cycles vs. 191.3±103.7 cycles; p = 0.66), in addition to an excellent agreement between these measures (ICC = 0.90; 95% IC 0.78-0.95). Only one adverse event (3.3%) occurred in the sample.Conclusion6MST showed convergent validity with the functional variables used in the diagnosis of sarcopenia. In addition, excellent test-retest reliability was observed, which indicates the need for a single assessment in hospitalized elderly patients. The prevalence of adverse events during the application of the test is low, without resulting in clinical symptoms; therefore, the test is considered safe for this population. In addition, hospitalized elderly patients perform worse in the 6MST compared to healthy elderly from the community.