Dataset Information

The Empirical Distribution of a Large Number of Correlated Normal Variables.

ABSTRACT: Motivated by the advent of high dimensional highly correlated data, this work studies the limit behavior of the empirical cumulative distribution function (ecdf) of standard normal random variables under arbitrary correlation. First, we provide a necessary and sufficient condition for convergence of the ecdf to the standard normal distribution. Next, under general correlation, we show that the ecdf limit is a random, possible infinite, mixture of normal distribution functions that depends on a number of latent variables and can serve as an asymptotic approximation to the ecdf in high dimensions. We provide conditions under which the dimension of the ecdf limit, defined as the smallest number of effective latent variables, is finite. Estimates of the latent variables are provided and their consistency proved. We demonstrate these methods in a real high-dimensional data example from brain imaging where it is shown that, while the study exhibits apparently strongly significant results, they can be entirely explained by correlation, as captured by the asymptotic approximation developed here.

SUBMITTER: Azriel D

PROVIDER: S-EPMC4742377 | biostudies-literature | 2015 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The Empirical Distribution of a Large Number of Correlated Normal Variables.

Azriel David D Schwartzman Armin A

Journal of the American Statistical Association 20140925 511

Motivated by the advent of high dimensional highly correlated data, this work studies the limit behavior of the empirical cumulative distribution function (ecdf) of standard normal random variables under arbitrary correlation. First, we provide a necessary and sufficient condition for convergence of the ecdf to the standard normal distribution. Next, under general correlation, we show that the ecdf limit is a random, possible infinite, mixture of normal distribution functions that depends on a n ...[more]

PMID: 26858467

Similar Datasets

Project description:The global biodiversity crisis has invigorated the search for generalized patterns in most disciplines within the natural sciences. Studies based on organismal functional traits attempt to broaden implications of results by identifying the response of functional traits, instead of taxonomic units, to environmental variables. Determining the functional trait responses enables more direct comparisons with, or predictions for, communities of different taxonomic composition. The North American freshwater fish fauna is both diverse and increasingly imperiled through human mediated disturbances, including climate change. The Tennessee River, USA, contains one of the most diverse assemblages of freshwater fish in North America and has more imperiled species than other rivers, but there has been no trait-based study of community structure in the system. We identified 211 localities in the upper Tennessee River that were sampled by the Tennessee Valley Authority between 2009 and 2011 and compiled fish functional traits for the observed species and environmental variables for each locality. Using fourth corner analysis, we identified significant correlations between many fish functional traits and environmental variables. Functional traits associated with an opportunistic life history strategy were correlated with localities subject to greater land use disturbance and less flow regulation, while functional traits associated with a periodic life history strategy were correlated with localities subject to regular disturbance and regulated flow. These are patterns observed at the continental scale, highlighting the generalizability of trait-based methods. Contrary to studies that found no community structure differences when considering riparian buffer zones, we found that fish functional traits were correlated with different environmental variables between analyses with buffer zones vs. entire catchment area land cover proportions. Using existing databases and fourth corner analysis, our results support the broad application potential for trait-based methods and indicate trait-based methods can detect environmental filtering by riparian zone land cover.

Project description:Both climatic and edaphic conditions determine plant distribution, however many species distribution models do not include edaphic variables especially over large geographical extent. Using an exceptional database of vegetation plots (n?=?4839) covering an extent of ?55,000 km2, we tested whether the inclusion of fine scale edaphic variables would improve model predictions of plant distribution compared to models using only climate predictors. We also tested how well these edaphic variables could predict distribution on their own, to evaluate the assumption that at large extents, distribution is governed largely by climate. We also hypothesized that the relative contribution of edaphic and climatic data would vary among species depending on their growth forms and biogeographical attributes within the study area. We modelled 128 native plant species from diverse taxa using four statistical model types and three sets of abiotic predictors: climate, edaphic, and edaphic-climate. Model predictive accuracy and variable importance were compared among these models and for species' characteristics describing growth form, range boundaries within the study area, and prevalence. For many species both the climate-only and edaphic-only models performed well, however the edaphic-climate models generally performed best. The three sets of predictors differed in the spatial information provided about habitat suitability, with climate models able to distinguish range edges, but edaphic models able to better distinguish within-range variation. Model predictive accuracy was generally lower for species without a range boundary within the study area and for common species, but these effects were buffered by including both edaphic and climatic predictors. The relative importance of edaphic and climatic variables varied with growth forms, with trees being more related to climate whereas lower growth forms were more related to edaphic conditions. Our study identifies the potential for non-climate aspects of the environment to pose a constraint to range expansion under climate change.

Dataset Information

The Empirical Distribution of a Large Number of Correlated Normal Variables.

Publications

The Empirical Distribution of a Large Number of Correlated Normal Variables.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets