Dataset Information

Test-retest Reliability of the qReading Method in Normally Sighted Young Adults.

ABSTRACT:

Significance

We recently developed a novel Bayesian adaptive method, qReading, to measure reading function. The qReading method has both the efficiency and excellent test-retest reliability in normally sighted young adults to make it an excellent candidate for future studies of its value in diagnosis and longitudinal evaluation of treatment and/or rehabilitation outcomes.

Purpose

A novel Bayesian adaptive method, qReading, was recently developed to measure reading function. Here we performed a systematic assessment of the test-retest reliability of the qReading method.

Methods

The variability of five repeated measurements of the reading curve was examined in two settings: within session and between sessions. For the within-session design, we considered two subpopulations: naive observers and experienced observers. All observers were normally sighted young adults. For each set of data, in addition to examining the intrinsic precision of the qReading method (the half width of the credible interval of the posterior distribution of the estimated performance), we computed four metrics to assess repeatability: standard deviation, Bland-Altman coefficient of repeatability, correlation coefficient, and Fractional Rank Precision.

Results

Extrinsic factors such as observer, time interval between repeated measures, and observer experience all contribute to the variation across measurements. Nevertheless, the four metrics consistently show that the variability across five repeated measurements is small for each set of data. This is true even without taking learning effects into account (standard deviations, ≤0.092 log10 units; Bland-Altman coefficient of repeatability, ≤0.15 (log10)2 units; correlation coefficient, ≥0.91; and Fractional Rank Precision, ≥0.81).

Conclusions

The qReading method has excellent test-retest reliability in normally sighted young adults.

SUBMITTER: Shepard TG

PROVIDER: S-EPMC8405568 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Functional connectivity (FC) has become a leading method for resting-state functional magnetic resonance imaging (rs-fMRI) analysis. However, the majority of the previous studies utilized pairwise, temporal synchronization-based FC. Recently, high-order FC (HOFC) methods were proposed with the idea of computing "correlation of correlations" to capture high-level, more complex associations among the brain regions. There are two types of HOFC. The first type is topographical profile similarity-based HOFC (tHOFC) and its variant, associated HOFC (aHOFC), for capturing different levels of HOFC. Instead of measuring the similarity of the original rs-fMRI signals with the traditional FC (low-order FC, or LOFC), tHOFC measures the similarity of LOFC profiles (i.e., a set of LOFC values between a region and all other regions) between each pair of brain regions. The second type is dynamics-based HOFC (dHOFC) which defines the quadruple relationship among every four brain regions by first calculating two pairwise dynamic LOFC "time series" and then measuring their temporal synchronization (i.e., temporal correlation of the LOFC fluctuations, not the BOLD fluctuations). Applications have shown the superiority of HOFC in both disease biomarker detection and individualized diagnosis than LOFC. However, no study has been carried out for the assessment of test-retest reliability of different HOFC metrics. In this paper, we systematically evaluate the reliability of the two types of HOFC methods using test-retest rs-fMRI data from 25 (12 females, age 24.48 ± 2.55 years) young healthy adults with seven repeated scans (with interval = 3-8 days). We found that all HOFC metrics have satisfactory reliability, specifically (1) fair-to-good for tHOFC and aHOFC, and (2) fair-to-moderate for dHOFC with relatively strong connectivity strength. We further give an in-depth analysis of the biological meanings of each HOFC metric and highlight their differences compared to the LOFC from the aspects of cross-level information exchanges, within-/between-network connectivity, and modulatory connectivity. In addition, how the dynamic analysis parameter (i.e., sliding window length) affects dHOFC reliability is also investigated. Our study reveals unique functional associations characterized by the HOFC metrics. Guidance and recommendations for future applications and clinical research using HOFC are provided. This study has made a further step toward unveiling more complex human brain connectome.

Project description:BackgroundNeurocognitive testing is an important concussion evaluation tool, but for neurocognitive tests to be useful, their psychometric properties must be well established. Test-retest reliability of computerized neurocognitive tests can influence their clinical utility. The reliability for a commonly used computerized neurocognitive test, CNS Vital Signs, is not well established. The purpose of this study was to examine test-retest reliability and reliable change indices for CNS Vital Signs in a healthy, physically active college population.HypothesisCNS Vital Signs yields acceptable test-retest reliability, with greater reliability between the second and third test administration compared with between the first and second administration.Study designCohort study.Level of evidenceLevel 3.MethodsForty healthy, active volunteers (16 men, 24 women; mean age, 21.05 ± 2.17 years) reported to a clinical laboratory for 3 sessions, 1 week apart. At each session, participants were administered CNS Vital Signs. Outcomes included standard scores for the following CNS Vital Signs domains: verbal memory, visual memory, psychomotor speed, cognitive flexibility, complex attention, processing speed, reaction time, executive functioning, and reasoning.ResultsParticipants performed significantly better on the second session and/or third session than they did on the first testing session on 6 of 9 neurocognitive domains. Pearson r test-retest correlations between sessions ranged from 0.11 to 0.87. Intraclass correlation coefficients ranged from 0.10 to 0.86.ConclusionClinicians should consider using reliable change indices to account for practice effects, identify meaningful score changes due to pathology, and inform clinical decisions.Clinical relevanceThis study highlights the importance of clinicians understanding the psychometric properties of computerized neurocognitive tests when using them in the management of sport-related concussion. If CNS Vital Signs is administered twice within a small time frame (such as 1 week), athletes should be expected to improve between the first and second administration.