Dataset Information

When does combining markers improve classification performance and what are implications for practice?

ABSTRACT: When an existing standard marker does not have sufficient classification accuracy on its own, new markers are sought with the goal of yielding a combination with better performance. The primary criterion for selecting new markers is that they have good performance on their own and preferably be uncorrelated with the standard. Most often linear combinations are considered. In this paper, we investigate the increment in performance that is possible by combining a novel continuous marker with a moderately performing standard continuous marker under a variety of biologically motivated models for their joint distribution. We find that an uncorrelated continuous marker with moderate performance on its own usually yields only minimally improved performance. We identify other settings that lead to large improvements, including a novel marker that has very poor performance on its own but is highly correlated with the standard and a novel marker with poor to moderate performance that is highly correlated with the standard but only in one class category. These results suggest changing current strategies for identifying markers to be included in panels for possible combination. Using simulated and real datasets, we examine the merits of a broadened strategy that selects panels of markers as candidates on the basis of their joint performance with existing markers, compared with the standard strategy that selects markers on the basis of their marginal performance. We find that a broadened strategy can be fruitful but necessitates using studies with large numbers of subjects.

SUBMITTER: Bansal A

PROVIDER: S-EPMC3893148 | biostudies-literature | 2013 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

When does combining markers improve classification performance and what are implications for practice?

Bansal Aasthaa A Pepe Margaret Sullivan MS

Statistics in medicine 20130124 11

When an existing standard marker does not have sufficient classification accuracy on its own, new markers are sought with the goal of yielding a combination with better performance. The primary criterion for selecting new markers is that they have good performance on their own and preferably be uncorrelated with the standard. Most often linear combinations are considered. In this paper, we investigate the increment in performance that is possible by combining a novel continuous marker with a mod ...[more]

PMID: 23348801

Similar Datasets

Project description:Riemannian geometry-based classification (RGBC) gained popularity in the field of brain-computer interfaces (BCIs) lately, due to its ability to deal with non-stationarities arising in electroencephalography (EEG) data. Domain adaptation, however, is most often performed on sample covariance matrices (SCMs) obtained from EEG data, and thus might not fully account for components affecting covariance estimation itself, such as regional trends. Detrended cross-correlation analysis (DCCA) can be utilized to estimate the covariance structure of such signals, yet it is computationally expensive in its original form. A recently proposed online implementation of DCCA, however, allows for its fast computation and thus makes it possible to employ DCCA in real-time applications. In this study we propose to replace the SCM with the DCCA matrix as input to RGBC and assess its effect on offline and online BCI performance. First we evaluated the proposed decoding pipeline offline on previously recorded EEG data from 18 individuals performing left and right hand motor imagery (MI), and benchmarked it against vanilla RGBC and popular MI-detection approaches. Subsequently, we recruited eight participants (with previous BCI experience) who operated an MI-based BCI (MI-BCI) online using the DCCA-enhanced Riemannian decoder. Finally, we tested the proposed method on a public, multi-class MI-BCI dataset. During offline evaluations the DCCA-based decoder consistently and significantly outperformed the other approaches. Online evaluation confirmed that the DCCA matrix could be computed in real-time even for 22-channel EEG, as well as subjects could control the MI-BCI with high command delivery (normalized Cohen's κ: 0.7409 ± 0.1515) and sample-wise MI detection (normalized Cohen's κ: 0.5200 ± 0.1610). Post-hoc analysis indicated characteristic connectivity patterns under both MI conditions, with stronger connectivity in the hemisphere contralateral to the MI task. Additionally, fractal scaling exponent of neural activity was found increased in the contralateral compared to the ipsilateral motor cortices (C4 and C3 for left and right MI, respectively) in both classes. Combining DCCA with Riemannian geometry-based decoding yields a robust and effective decoder, that not only improves upon the SCM-based approach but can also provide relevant information on the neurophysiological processes behind MI.

Dataset Information

When does combining markers improve classification performance and what are implications for practice?

Publications

When does combining markers improve classification performance and what are implications for practice?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets