Browse
Submit Data
Databases
API
Help

Dataset Information

19 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Better-than-chance classification for signal detection.

ABSTRACT: The estimated accuracy of a classifier is a random quantity with variability. A common practice in supervised machine learning, is thus to test if the estimated accuracy is significantly better than chance level. This method of signal detection is particularly popular in neuroimaging and genetics. We provide evidence that using a classifier's accuracy as a test statistic can be an underpowered strategy for finding differences between populations, compared to a bona fide statistical test. It is also computationally more demanding than a statistical test. Via simulation, we compare test statistics that are based on classification accuracy, to others based on multivariate test statistics. We find that the probability of detecting differences between two distributions is lower for accuracy-based statistics. We examine several candidate causes for the low power of accuracy-tests. These causes include: the discrete nature of the accuracy-test statistic, the type of signal accuracy-tests are designed to detect, their inefficient use of the data, and their suboptimal regularization. When the purpose of the analysis is the evaluation of a particular classifier, not signal detection, we suggest several improvements to increase power. In particular, to replace V-fold cross-validation with the Leave-One-Out Bootstrap.

SUBMITTER: Rosenblatt JD

PROVIDER: S-EPMC8036001 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Using Signal Detection Theory to Better Understand Cognitive Fatigue.

Project description:When we are fatigued, we feel that our performance is worse than when we are fresh. Yet, for over 100 years, researchers have been unable to identify an objective, behavioral measure that covaries with the subjective experience of fatigue. Previous work suggests that the metrics of signal detection theory (SDT)-response bias (criterion) and perceptual certainty (d')-may change as a function of fatigue, but no work has yet been done to examine whether these metrics covary with fatigue. Here, we investigated cognitive fatigue using SDT. We induced fatigue through repetitive performance of the n-back working memory task, while functional magnetic resonance imaging (fMRI) data was acquired. We also assessed cognitive fatigue at intervals throughout. This enabled us to assess not only whether criterion and d' covary with cognitive fatigue but also whether similar patterns of brain activation underlie cognitive fatigue and SDT measures. Our results show that both criterion and d' were correlated with changes in cognitive fatigue: as fatigue increased, subjects became more conservative in their response bias and their perceptual certainty declined. Furthermore, activation in the striatum of the basal ganglia was also related to cognitive fatigue, criterion, and d'. These results suggest that SDT measures represent an objective measure of cognitive fatigue. Additionally, the overlap and difference in the fMRI results between cognitive fatigue and SDT measures indicate that these measures are related while also separate. In sum, we show the relevance of SDT measures in the understanding of fatigue, thus providing researchers with a new set of tools with which to better understand the nature and consequences of cognitive fatigue.

| S-EPMC7844088 | biostudies-literature

Second-chance signal transduction explains cooperative flagellar switching.

Project description:The reversal of flagellar motion (switching) results from the interaction between a switch complex of the flagellar rotor and a torque-generating stationary unit, or stator (motor unit). To explain the steeply cooperative ligand-induced switching, present models propose allosteric interactions between subunits of the rotor, but do not address the possibility of a reaction that stimulates a bidirectional motor unit to reverse direction of torque. During flagellar motion, the binding of a ligand-bound switch complex at the dwell site could excite a motor unit. The probability that another switch complex of the rotor, moving according to steady-state rotation, will reach the same dwell site before that motor unit returns to ground state will be determined by the independent decay rate of the excited-state motor unit. Here, we derive an analytical expression for the energy coupling between a switch complex and a motor unit of the stator complex of a flagellum, and demonstrate that this model accounts for the cooperative switching response without the need for allosteric interactions. The analytical result can be reproduced by simulation when (1) the motion of the rotor delivers a subsequent ligand-bound switch to the excited motor unit, thereby providing the excited motor unit with a second chance to remain excited, and (2) the outputs from multiple independent motor units are constrained to a single all-or-none event. In this proposed model, a motor unit and switch complex represent the components of a mathematically defined signal transduction mechanism in which energy coupling is driven by steady-state and is regulated by stochastic ligand binding. Mathematical derivation of the model shows the analytical function to be a general form of the Hill equation (Hill AV (1910) The possible effects of the aggregation of the molecules of haemoglobin on its dissociation curves. J Physiol 40: iv-vii).

| S-EPMC3402542 | biostudies-literature

Electroencephalogram Signal Classification for Automated Epileptic Seizure Detection Using Genetic Algorithm.

Project description:Epilepsy causes when the repeated seizure occurs in the brain. Electroencephalogram (EEG) test provides valuable information about the brain functions and can be useful to detect brain disorder, especially for epilepsy. In this study, application for an automated seizure detection model has been introduced successfully.The EEG signals are decomposed into sub-bands by discrete wavelet transform using db2 (daubechies) wavelet. The eight statistical features, the four gray level co-occurrence matrix and Renyi entropy estimation with four different degrees of order, are extracted from the raw EEG and its sub-bands. Genetic algorithm (GA) is used to select eight relevant features from the 16 dimension features. The model has been trained and tested using support vector machine (SVM) classifier successfully for EEG signals. The performance of the SVM classifier is evaluated for two different databases.The study has been experimented through two different analyses and achieved satisfactory performance for automated seizure detection using relevant features as the input to the SVM classifier.Relevant features using GA give better accuracy performance for seizure detection.

| S-EPMC5523521 | biostudies-other

Better-than-chance prediction of cooperative behaviour from first and second impressions.

Project description:Could cooperation among strangers be facilitated by adaptations that use sparse information to accurately predict cooperative behaviour? We hypothesise that predictions are influenced by beliefs, descriptions, appearance and behavioural history available for first and second impressions. We also hypothesise that predictions improve when more information is available. We conducted a two-part study. First, we recorded thin-slice videos of university students just before their choices in a repeated Prisoner's Dilemma with matched partners. Second, a worldwide sample of raters evaluated each player using videos, photos, only gender labels or neither images nor labels. Raters guessed players' first-round Prisoner's Dilemma choices and then their second-round choices after reviewing first-round behavioural histories. Our design allows us to investigate incremental effects of gender, appearance and behavioural history gleaned during first and second impressions. Predictions become more accurate and better-than-chance when gender, appearance or behavioural history is added. However, these effects are not incrementally cumulative. Predictions from treatments showing player appearance were no more accurate than those from treatments revealing gender labels and predictions from videos were no more accurate than those from photos. These results demonstrate how people accurately predict cooperation under sparse information conditions, helping explain why conditional cooperation is common among strangers.

| S-EPMC10955359 | biostudies-literature

caArray_louis-00379: Gene Expression-based Classification of Malignant Gliomas Correlates Better with Survival than Histological Classification

Project description:Microarray analysis was used to determine the expression of 12,000 genes in a set of 50 gliomas, 28 glioblastomas and 22 anaplastic oligodendrogliomas. Supervised learning approaches were used to build a two-class prediction model based on a subset of 14 glioblastomas and 7 anaplastic oligodendrogliomas with classic histology. A 20-feature k-nearest neighbor model correctly classified 18 of the 21 classic cases in leave-one-out cross-validation when compared with pathological diagnoses. This model was then used to predict the classification of clinically common, histologically nonclassic samples. When tumors were classified according to pathology, the survival of patients with nonclassic glioblastoma and nonclassic anaplastic oligodendroglioma was not significantly different (P = 0.19). However, class distinctions according to the model were significantly associated with survival outcome (P = 0.05). This class prediction model was capable of classifying high-grade, nonclassic glial tumors objectively and reproducibly. Moreover, the model provided a more accurate predictor of prognosis in these nonclassic lesions than did pathological classification. These data suggest that class prediction models, based on defined molecular profiles, classify diagnostically challenging malignant gliomas in a manner that better correlates with clinical outcome than does standard pathology.

2016-05-28 | GSE82009 | GEO

Novel immunotherapy combinations in neoadjuvant non-small cell lung cancer (NSCLC): a better chance at cure?

Project description: Not available

| S-EPMC11002516 | biostudies-literature

caArray_louis-00379: Gene Expression-based Classification of Malignant Gliomas Correlates Better with Survival than Histological Classification

2016-05-28 | E-GEOD-82009 | biostudies-arrayexpress

Plants with double genomes might have had a better chance to survive the Cretaceous-Tertiary extinction event.

Project description:Most flowering plants have been shown to be ancient polyploids that have undergone one or more whole genome duplications early in their evolution. Furthermore, many different plant lineages seem to have experienced an additional, more recent genome duplication. Starting from paralogous genes lying in duplicated segments or identified in large expressed sequence tag collections, we dated these youngest duplication events through penalized likelihood phylogenetic tree inference. We show that a majority of these independent genome duplications are clustered in time and seem to coincide with the Cretaceous-Tertiary (KT) boundary. The KT extinction event is the most recent mass extinction caused by one or more catastrophic events such as a massive asteroid impact and/or increased volcanic activity. These events are believed to have generated global wildfires and dust clouds that cut off sunlight during long periods of time resulting in the extinction of approximately 60% of plant species, as well as a majority of animals, including dinosaurs. Recent studies suggest that polyploid species can have a higher adaptability and increased tolerance to different environmental conditions. We propose that polyploidization may have contributed to the survival and propagation of several plant lineages during or following the KT extinction event. Due to advantages such as altered gene expression leading to hybrid vigor and an increased set of genes and alleles available for selection, polyploid plants might have been better able to adapt to the drastically changed environment 65 million years ago.

| S-EPMC2667025 | biostudies-literature

The Lambda variant of SARS-CoV-2 has a better chance than the Delta variant to escape vaccines.

Project description:The newly emerging variants of SARS-CoV-2 from India (Delta variant) and South America (Lambda variant) have led to a higher infection rate of either vaccinated or unvaccinated people. We found that sera from Pfizer-BioNTech vaccine remain high reactivity toward the receptor binding domain (RBD) of Delta variant while it drops dramatically toward that of Lambda variant. Interestingly, the overall titer of antibodies of Pfizer-BioNTech vaccinated individuals drops 3-fold after 6 months, which could be one of major reasons for breakthrough infections, emphasizing the importance of potential third boost shot. While a therapeutic antibody, Bamlanivimab, decreases binding affinity to Delta variant by ~20 fold, it fully lost binding to Lambda variant. Structural modeling of complexes of RBD with human receptor, Angiotensin Converting Enzyme 2 (ACE2), and Bamlanivimab suggest the potential basis of the change of binding. The data suggest possible danger and a potential surge of Lambda variant in near future.

| S-EPMC8404886 | biostudies-literature

High-frequency neuronal signal better explains multi-phase BOLD response.

Project description:Visual stimulation-evoked blood-oxygen-level dependent (BOLD) responses can exhibit more complex temporal dynamics than a simple monophasic response. For instance, BOLD responses sometimes include a phase of positive response followed by a phase of post-stimulus undershoot. Whether the BOLD response during these phases reflects the underlying neuronal signal fluctuations or is contributed by non-neuronal physiological factors remains elusive. When presenting blocks of sustained (i.e. DC) light ON-OFF stimulations to unanesthetized rats, we observed that the response following a decrease in illumination (i.e. OFF stimulation-evoked BOLD response) in the visual cortices displayed reproducible multiple phases, including an initial positive BOLD response, followed by an undershoot and then an overshoot before the next ON trial. This multi-phase BOLD response did not result from the entrainment of the periodic stimulation structure. When we measured the neural correlates of these responses, we found that the high-frequency band from the LFP power (300 - 3000 Hz, multi-unit activity (MUA)), but not the power in the gamma band (30 - 100 Hz) exhibited the same multiphasic dynamics as the BOLD signal. This study suggests that the post-stimulus phases of the BOLD response can be better explained by the high-frequency neuronal signal.

| S-EPMC9962576 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data