Dataset Information

ESTIMATION OF FUNCTIONALS OF SPARSE COVARIANCE MATRICES.

ABSTRACT: High-dimensional statistical tests often ignore correlations to gain simplicity and stability leading to null distributions that depend on functionals of correlation matrices such as their Frobenius norm and other ? r norms. Motivated by the computation of critical values of such tests, we investigate the difficulty of estimation the functionals of sparse correlation matrices. Specifically, we show that simple plug-in procedures based on thresholded estimators of correlation matrices are sparsity-adaptive and minimax optimal over a large class of correlation matrices. Akin to previous results on functional estimation, the minimax rates exhibit an elbow phenomenon. Our results are further illustrated in simulated data as well as an empirical study of data arising in financial econometrics.

SUBMITTER: Fan J

PROVIDER: S-EPMC4719663 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

ESTIMATION OF FUNCTIONALS OF SPARSE COVARIANCE MATRICES.

Fan Jianqing J Rigollet Philippe P Wang Weichen W

Annals of statistics 20150101 6

High-dimensional statistical tests often ignore correlations to gain simplicity and stability leading to null distributions that depend on functionals of correlation matrices such as their Frobenius norm and other ℓ <sub><i>r</i></sub> norms. Motivated by the computation of critical values of such tests, we investigate the difficulty of estimation the functionals of sparse correlation matrices. Specifically, we show that simple plug-in procedures based on thresholded estimators of correlation ma ...[more]

PMID: 26806986

Similar Datasets

Project description:In neurodegenerative disorders, a clearer understanding of the underlying aberrant networks facilitates the search for effective therapeutic targets and potential cures. [18F]-fluorodeoxyglucose (FDG) positron emission tomography (PET) imaging data of brain metabolism reflects the distribution of glucose consumption known to be directly related to neural activity. In FDG PET resting-state metabolic data, characteristic disease-related patterns have been identified in group analysis of various neurodegenerative conditions using principal component analysis of multivariate spatial covariance. Notably, among several parkinsonian syndromes, the identified Parkinson's disease-related pattern (PDRP) has been repeatedly validated as an imaging biomarker of PD in independent groups worldwide. Although the primary nodal associations of this network are known, its connectivity is not fully understood. Here, we describe a novel approach to elucidate functional principal component (PC) network connections by performing graph theoretical sparse network derivation directly within the disease relevant PC partition layer of the whole brain data rather than by searching for associations retrospectively in whole brain sparse representations. Using sparse inverse covariance estimation of each overlapping PC partition layer separately, a single coherent network is detected for each layer in contrast to more spatially modular segmentation in whole brain data analysis. Using this approach, the major nodal hubs of the PD disease network are identified and their characteristic functional pathways are clearly distinguished within the basal ganglia, midbrain and parietal areas. Network associations are further clarified using Laplacian spectral analysis of the adjacency matrices. In addition, the innate discriminative capacity of the eigenvector centrality of the graph derived networks in differentiating PD versus healthy external data provides evidence of their validity.

Project description:This article investigates a form of rank deficiency in phenotypic covariance matrices derived from geometric morphometric data, and its impact on measures of phenotypic integration. We first define a type of rank deficiency based on information theory then demonstrate that this deficiency impairs the performance of phenotypic integration metrics in a model system. Lastly, we propose methods to treat for this information rank deficiency. Our first goal is to establish how the rank of a typical geometric morphometric covariance matrix relates to the information entropy of its eigenvalue spectrum. This requires clear definitions of matrix rank, of which we define three: the full matrix rank (equal to the number of input variables), the mathematical rank (the number of nonzero eigenvalues), and the information rank or "effective rank" (equal to the number of nonredundant eigenvalues). We demonstrate that effective rank deficiency arises from a combination of methodological factors-Generalized Procrustes analysis, use of the correlation matrix, and insufficient sample size-as well as phenotypic covariance. Secondly, we use dire wolf jaws to document how differences in effective rank deficiency bias two metrics used to measure phenotypic integration. The eigenvalue variance characterizes the integration change incorrectly, and the standardized generalized variance lacks the sensitivity needed to detect subtle changes in integration. Both metrics are impacted by the inclusion of many small, but nonzero, eigenvalues arising from a lack of information in the covariance matrix, a problem that usually becomes more pronounced as the number of landmarks increases. We propose a new metric for phenotypic integration that combines the standardized generalized variance with information entropy. This metric is equivalent to the standardized generalized variance but calculated only from those eigenvalues that carry nonredundant information. It is the standardized generalized variance scaled to the effective rank of the eigenvalue spectrum. We demonstrate that this metric successfully detects the shift of integration in our dire wolf sample. Our third goal is to generalize the new metric to compare data sets with different sample sizes and numbers of variables. We develop a standardization for matrix information based on data permutation then demonstrate that Smilodon jaws are more integrated than dire wolf jaws. Finally, we describe how our information entropy-based measure allows phenotypic integration to be compared in dense semilandmark data sets without bias, allowing characterization of the information content of any given shape, a quantity we term "latent dispersion". [Canis dirus; Dire wolf; effective dispersion; effective rank; geometric morphometrics; information entropy; latent dispersion; modularity and integration; phenotypic integration; relative dispersion.].

Dataset Information

ESTIMATION OF FUNCTIONALS OF SPARSE COVARIANCE MATRICES.

Publications

ESTIMATION OF FUNCTIONALS OF SPARSE COVARIANCE MATRICES.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets