Dataset Information

Overcoming the effects of false positives and threshold bias in graph theoretical analyses of neuroimaging data.

ABSTRACT: Graph theory (GT) is a powerful framework for quantifying topological features of neuroimaging-derived functional and structural networks. However, false positive (FP) connections arise frequently and influence the inferred topology of networks. Thresholding is often used to overcome this problem, but an appropriate threshold often relies on a priori assumptions, which will alter inferred network topologies. Four common network metrics (global efficiency, mean clustering coefficient, mean betweenness and smallworldness) were tested using a model tractography dataset. It was found that all four network metrics were significantly affected even by just one FP. Results also show that thresholding effectively dampens the impact of FPs, but at the expense of adding significant bias to network metrics. In a larger number (n=248) of tractography datasets, statistics were computed across random group permutations for a range of thresholds, revealing that statistics for network metrics varied significantly more than for non-network metrics (i.e., number of streamlines and number of edges). Varying degrees of network atrophy were introduced artificially to half the datasets, to test sensitivity to genuine group differences. For some network metrics, this atrophy was detected as significant (p<0.05, determined using permutation testing) only across a limited range of thresholds. We propose a multi-threshold permutation correction (MTPC) method, based on the cluster-enhanced permutation correction approach, to identify sustained significant effects across clusters of thresholds. This approach minimises requirements to determine a single threshold a priori. We demonstrate improved sensitivity of MTPC-corrected metrics to genuine group effects compared to an existing approach and demonstrate the use of MTPC on a previously published network analysis of tractography data derived from a clinical population. In conclusion, we show that there are large biases and instability induced by thresholding, making statistical comparisons of network metrics difficult. However, by testing for effects across multiple thresholds using MTPC, true group differences can be robustly identified.

SUBMITTER: Drakesmith M

PROVIDER: S-EPMC4558463 | biostudies-literature | 2015 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Overcoming the effects of false positives and threshold bias in graph theoretical analyses of neuroimaging data.

Drakesmith M M Caeyenberghs K K Dutt A A Lewis G G David A S AS Jones D K DK

NeuroImage 20150514

Graph theory (GT) is a powerful framework for quantifying topological features of neuroimaging-derived functional and structural networks. However, false positive (FP) connections arise frequently and influence the inferred topology of networks. Thresholding is often used to overcome this problem, but an appropriate threshold often relies on a priori assumptions, which will alter inferred network topologies. Four common network metrics (global efficiency, mean clustering coefficient, mean betwee ...[more]

PMID: 25982515

Similar Datasets

Project description:Background Non-invasive prenatal testing (NIPT) has had an incomparable triumph in prenatal diagnostics in the last decade. Over 1400 research articles have been published, predominantly praising the advantages of this test. Methods The present study identified among the 1400 papers 24 original and one review paper, which were suited to re-evaluate the efficacy of > 750,000 published NIPT-results. Special attention was given to false-positive and false-negative result-rates. Those were discussed under different aspects—mainly from a patient-perspective. Results A 27: 1 rate of false-positive compared to false-negative NIPT results was found. Besides, according to all reported, real-positive, chromosomally aberrant NIPT cases, 90% of those would have been aborted spontaneously before birth. These findings are here discussed under aspects like (i) How efficient is NIPT compared to first trimester screening? (ii) What are the differences in expectations towards NIPT from specialists and the public? and (iii) There should also be children born suffering from not by NIPT tested chromosomal aberrations; why are those never reported in all available NIPT studies? Conclusions Even though much research has been published on NIPT, unbiased figures concerning NIPT and first trimester screening efficacy are yet not available. While false positive rates of different NIPT tests maybe halfway accurate, reported false-negative rates are most likely too low. The latter is as NIPT-cases with negative results for tested conditions are yet not in detail followed up for cases with other genetic or teratogenic caused disorders. This promotes an image in public, that NIPT is suited to replace all invasive tests, and also to solve the problem of inborn errors in humans, if not now then in near future. Overall, it is worth discussing the usefulness of NIPT in practical clinical application. Particularly, asking for unbiased figures concerning the efficacy of first trimester-screening compared to NIPT, and for really comprehensive data on false-positive and false-negative NIPT results. Supplementary Information The online version contains supplementary material available at 10.1186/s13039-022-00612-2.

Project description:BackgroundA large number of probabilistic models used in sequence analysis assign non-zero probability values to most input sequences. To decide when a given probability is sufficient the most common way is bayesian binary classification, where the probability of the model characterizing the sequence family of interest is compared to that of an alternative probability model. We can use as alternative model a null model. This is the scoring technique used by sequence analysis tools such as HMMER, SAM and INFERNAL. The most prevalent null models are position-independent residue distributions that include: the uniform distribution, genomic distribution, family-specific distribution and the target sequence distribution. This paper presents a study to evaluate the impact of the choice of a null model in the final result of classifications. In particular, we are interested in minimizing the number of false predictions in a classification. This is a crucial issue to reduce costs of biological validation.ResultsFor all the tests, the target null model presented the lowest number of false positives, when using random sequences as a test. The study was performed in DNA sequences using GC content as the measure of content bias, but the results should be valid also for protein sequences. To broaden the application of the results, the study was performed using randomly generated sequences. Previous studies were performed on aminoacid sequences, using only one probabilistic model (HMM) and on a specific benchmark, and lack more general conclusions about the performance of null models. Finally, a benchmark test with P. falciparum confirmed these results.ConclusionsOf the evaluated models the best suited for classification are the uniform model and the target model. However, the use of the uniform model presents a GC bias that can cause more false positives for candidate sequences with extreme compositional bias, a characteristic not described in previous studies. In these cases the target model is more dependable for biological validation due to its higher specificity.

Dataset Information

Overcoming the effects of false positives and threshold bias in graph theoretical analyses of neuroimaging data.

Publications

Overcoming the effects of false positives and threshold bias in graph theoretical analyses of neuroimaging data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets