Dataset Information

On the generalizability of resting-state fMRI machine learning classifiers.

ABSTRACT: Machine learning classifiers have become increasingly popular tools to generate single-subject inferences from fMRI data. With this transition from the traditional group level difference investigations to single-subject inference, the application of machine learning methods can be seen as a considerable step forward. Existing studies, however, have given scarce or no information on the generalizability to other subject samples, limiting the use of such published classifiers in other research projects. We conducted a simulation study using publicly available resting-state fMRI data from the 1000 Functional Connectomes and COBRE projects to examine the generalizability of classifiers based on regional homogeneity of resting-state time series. While classification accuracies of up to 0.8 (using sex as the target variable) could be achieved on test datasets drawn from the same study as the training dataset, the generalizability of classifiers to different study samples proved to be limited albeit above chance. This shows that on the one hand a certain amount of generalizability can robustly be expected, but on the other hand this generalizability should not be overestimated. Indeed, this study substantiates the need to include data from several sites in a study investigating machine learning classifiers with the aim of generalizability.

SUBMITTER: Huf W

PROVIDER: S-EPMC4114329 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

On the generalizability of resting-state fMRI machine learning classifiers.

Huf Wolfgang W Kalcher Klaudius K Boubela Roland N RN Rath Georg G Vecsei Andreas A Filzmoser Peter P Moser Ewald E

Frontiers in human neuroscience 20140729

Machine learning classifiers have become increasingly popular tools to generate single-subject inferences from fMRI data. With this transition from the traditional group level difference investigations to single-subject inference, the application of machine learning methods can be seen as a considerable step forward. Existing studies, however, have given scarce or no information on the generalizability to other subject samples, limiting the use of such published classifiers in other research pro ...[more]

PMID: 25120443

Similar Datasets

Project description:BACKGROUND:Traumatized earthquake survivors may develop poor memory function. Resting-state functional magnetic resonance imaging (rs-fMRI) and machine learning techniques may one day aid the clinical assessment of individual psychiatric patients. This study aims to use machine learning with Rs-fMRI from the perspectives of neurophysiology and neuroimaging to explore the association between it and the individual memory function of trauma survivors. METHODS:Rs-fMRI data was acquired for eighty-nine survivors (male (33%), average age (SD):45.18(6.31) years) of Wenchuan earthquakes in 2008 each of whom was screened by experienced psychiatrists based on the clinician-administered post-traumatic stress disorder (PTSD) scale (CAPS), and their memory function scores were determined by the Wechsler Memory Scale-IV (WMS-IV). We explored which memory function scores were significantly associated with CAPS scores. Using simple multiple kernel learning (MKL), Rs-fMRI was used to predict the memory function scores that were associated with CAPS scores. A support vector machine (SVM) was also used to make classifications in trauma survivors with or without PTSD. RESULTS:Spatial addition (SA), which is defined by spatial working memory function, was negatively correlated with the total CAPS score (r?=?-?0.22, P?=?0.04). The use of simple MKL allowed quantitative association of SA scores with statistically significant accuracy (correlation?=?0.28, P?=?0.03; mean squared error?=?8.36; P?=?0.04). The left middle frontal gyrus and the left precuneus contributed the largest proportion to the simple MKL association frame. The SVM could not make a quantitative classification of diagnosis with statistically significant accuracy. LIMITATIONS:The use of the cross-sectional study design after exposure to an earthquake and the leave-one-out cross-validation (LOOCV) increases the risk of overfitting. CONCLUSION:Spontaneous brain activity of the left middle frontal gyrus and the left precuneus acquired by rs-fMRI may be a brain mechanism of visual working memory that is related to PTSD symptoms. Machine learning may be a useful tool in the identification of brain mechanisms of memory impairment in trauma survivors.

Dataset Information

On the generalizability of resting-state fMRI machine learning classifiers.

Publications

On the generalizability of resting-state fMRI machine learning classifiers.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets