Dataset Information

Blessing of dimensionality: mathematical foundations of the statistical physics of data.

ABSTRACT: The concentrations of measure phenomena were discovered as the mathematical background to statistical mechanics at the end of the nineteenth/beginning of the twentieth century and have been explored in mathematics ever since. At the beginning of the twenty-first century, it became clear that the proper utilization of these phenomena in machine learning might transform the curse of dimensionality into the blessing of dimensionality This paper summarizes recently discovered phenomena of measure concentration which drastically simplify some machine learning problems in high dimension, and allow us to correct legacy artificial intelligence systems. The classical concentration of measure theorems state that i.i.d. random points are concentrated in a thin layer near a surface (a sphere or equators of a sphere, an average or median-level set of energy or another Lipschitz function, etc.). The new stochastic separation theorems describe the thin structure of these thin layers: the random points are not only concentrated in a thin layer but are all linearly separable from the rest of the set, even for exponentially large random sets. The linear functionals for separation of points can be selected in the form of the linear Fisher's discriminant. All artificial intelligence systems make errors. Non-destructive correction requires separation of the situations (samples) with errors from the samples corresponding to correct behaviour by a simple and robust classifier. The stochastic separation theorems provide us with such classifiers and determine a non-iterative (one-shot) procedure for their construction.This article is part of the theme issue 'Hilbert's sixth problem'.

SUBMITTER: Gorban AN

PROVIDER: S-EPMC5869543 | biostudies-other | 2018 Apr

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Blessing of dimensionality: mathematical foundations of the statistical physics of data.

Gorban A N AN Tyukin I Y IY

Philosophical transactions. Series A, Mathematical, physical, and engineering sciences 20180401 2118

The concentrations of measure phenomena were discovered as the mathematical background to statistical mechanics at the end of the nineteenth/beginning of the twentieth century and have been explored in mathematics ever since. At the beginning of the twenty-first century, it became clear that the proper utilization of these phenomena in machine learning might transform the <i>curse of dimensionality</i> into the <i>blessing of dimensionality</i> This paper summarizes recently discovered phenomena ...[more]

PMID: 29555807

Similar Datasets

Project description:A variety of pulmonary insults can prompt the need for life-saving mechanical ventilation; however, misuse, prolonged use, or an excessive inflammatory response, can result in ventilator-induced lung injury. Past research has observed an increased instance of respiratory distress in older patients and differences in the inflammatory response. To address this, we performed high pressure ventilation on young (2-3 months) and old (20-25 months) mice for 2 hours and collected data for macrophage phenotypes and lung tissue integrity. Large differences in macrophage activation at baseline and airspace enlargement after ventilation were observed in the old mice. The experimental data was used to determine plausible trajectories for a mathematical model of the inflammatory response to lung injury which includes variables for the innate inflammatory cells and mediators, epithelial cells in varying states, and repair mediators. Classification methods were used to identify influential parameters separating the parameter sets associated with the young or old data and separating the response to ventilation, which was measured by changes in the epithelial state variables. Classification methods ranked parameters involved in repair and damage to the epithelial cells and those associated with classically activated macrophages to be influential. Sensitivity results were used to determine candidate in-silico interventions and these interventions were most impact for transients associated with the old data, specifically those with poorer lung health prior to ventilation. Model results identified dynamics involved in M1 macrophages as a focus for further research, potentially driving the age-dependent differences in all macrophage phenotypes. The model also supported the pro-inflammatory response as a potential indicator of age-dependent differences in response to ventilation. This mathematical model can serve as a baseline model for incorporating other pulmonary injuries.

Project description:Backgrounddata generated using 'omics' technologies are characterized by high dimensionality, where the number of features measured per subject vastly exceeds the number of subjects in the study. In this paper, we consider issues relevant in the design of biomedical studies in which the goal is the discovery of a subset of features and an associated algorithm that can predict a binary outcome, such as disease status. We compare the performance of four commonly used classifiers (K-Nearest Neighbors, Prediction Analysis for Microarrays, Random Forests and Support Vector Machines) in high-dimensionality data settings. We evaluate the effects of varying levels of signal-to-noise ratio in the dataset, imbalance in class distribution and choice of metric for quantifying performance of the classifier. To guide study design, we present a summary of the key characteristics of 'omics' data profiled in several human or animal model experiments utilizing high-content mass spectrometry and multiplexed immunoassay based techniques.Resultsthe analysis of data from seven 'omics' studies revealed that the average magnitude of effect size observed in human studies was markedly lower when compared to that in animal studies. The data measured in human studies were characterized by higher biological variation and the presence of outliers. The results from simulation studies indicated that the classifier Prediction Analysis for Microarrays (PAM) had the highest power when the class conditional feature distributions were Gaussian and outcome distributions were balanced. Random Forests was optimal when feature distributions were skewed and when class distributions were unbalanced. We provide a free open-source R statistical software library (MVpower) that implements the simulation strategy proposed in this paper.Conclusionno single classifier had optimal performance under all settings. Simulation studies provide useful guidance for the design of biomedical studies involving high-dimensionality data.

Dataset Information

Blessing of dimensionality: mathematical foundations of the statistical physics of data.

Publications

Blessing of dimensionality: mathematical foundations of the statistical physics of data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets