Distribution-free tests of independence in high dimensions.
Ontology highlight
ABSTRACT:
SUBMITTER: Han F
PROVIDER: S-EPMC5793489 | biostudies-literature | 2017 Dec
REPOSITORIES: biostudies-literature
Publications
Distribution-free tests of independence in high dimensions.
Han Fang F Chen Shizhe S Liu Han H
Biometrika 20171003 4
We consider the testing of mutual independence among all entries in a [Formula: see text]-dimensional random vector based on [Formula: see text] independent observations. We study two families of distribution-free test statistics, which include Kendall's tau and Spearman's rho as important examples. We show that under the null hypothesis the test statistics of these two families converge weakly to Gumbel distributions, and we propose tests that control the Type I error in the high-dimensional se ...[more]
Similar Datasets
Project description:BackgroundEvaluating severity of illness of patients with prolonged mechanical ventilation (PMV) is important to adopt the best appropriate care management for each individual. Yet, no severity-of-illness scoring system has been specifically designed for this type of patients. The aim of this study was to develop and validate a new instrument, the Multi-INdependence Dimensions (MIND) questionnaire designed to comprehensively measure the severity of illness of patients under PMV.MethodsThe validation of the MIND questionnaire was performed during a longitudinal observational study conducted with PMV subjects in weaning facilities in three countries (Argentina, Colombia and Germany). The questionnaire validity was tested in 3 stages: 1) Specification of components, with description of item responses, inter-item and Cronbach alpha correlations; 2) Creation of the composite scores; 3) Measurement properties determination including test-retest reliability after 30 days, clinical validity (Medical Research Council (MRC) muscle strength score, Sepsis-related Organ Failure Assessment (SOFA), Glasgow Coma Scale (GCS), Dependence Nursing Scale and EuroQol-5 Dimension evaluated at inclusion), and ability to detect change.ResultsA total of 128 subjects participated in the validation study. Eleven component scores and four composite scores were created. MIND scores significantly correlated with MRC muscle strength, SOFA, DNS, GCS and EQ-5D, supporting the validity of the new scores. Intraclass Correlation Coefficient greater than 0.82 were observed for all composite scores, indicating good test-retest reliability. MIND scores were able to detect improvement in subject severity of illness.ConclusionThe MIND questionnaire is a valid and reliable instrument for measuring comprehensively the multiple dimensions characterizing the severity of illness of PMV patients.Trial registrationNCT02255058 .
| S-EPMC6585039 | biostudies-literature
Project description:Most papers on high-dimensional statistics are based on the assumption that none of the regressors are correlated with the regression error, namely, they are exogenous. Yet, endogeneity can arise incidentally from a large pool of regressors in a high-dimensional regression. This causes the inconsistency of the penalized least-squares method and possible false scientific discoveries. A necessary condition for model selection consistency of a general class of penalized regression methods is given, which allows us to prove formally the inconsistency claim. To cope with the incidental endogeneity, we construct a novel penalized focused generalized method of moments (FGMM) criterion function. The FGMM effectively achieves the dimension reduction and applies the instrumental variable methods. We show that it possesses the oracle property even in the presence of endogenous predictors, and that the solution is also near global minimum under the over-identification assumption. Finally, we also show how the semi-parametric efficiency of estimation can be achieved via a two-step approach.
| S-EPMC4286899 | biostudies-literature
Project description:Projection pursuit is a classical exploratory data analysis method to detect interesting low-dimensional structures in multivariate data. Originally, projection pursuit was applied mostly to data of moderately low dimension. Motivated by contemporary applications, we here study its properties in high-dimensional settings. Specifically, we analyze the asymptotic properties of projection pursuit on structureless multivariate Gaussian data with an identity covariance, as both dimension p and sample size n tend to infinity, with [Formula: see text] Our main results are that (i) if [Formula: see text] then there exist projections whose corresponding empirical cumulative distribution function can approximate any arbitrary distribution; and (ii) if [Formula: see text], not all limiting distributions are possible. However, depending on the value of ?, various non-Gaussian distributions may still be approximated. In contrast, if we restrict to sparse projections, involving only a few of the p variables, then asymptotically all empirical cumulative distribution functions are Gaussian. And (iii) if [Formula: see text], then asymptotically all projections are Gaussian. Some of these results extend to mean-centered sub-Gaussian data and to projections into k dimensions. Hence, in the "small n, large p" setting, unless sparsity is enforced, and regardless of the chosen projection index, projection pursuit may detect an apparent structure that has no statistical significance. Furthermore, our work reveals fundamental limitations on the ability to detect non-Gaussian signals in high-dimensional data, in particular through independent component analysis and related non-Gaussian component analysis.
| S-EPMC6140545 | biostudies-literature
Project description:There have been recent proposals advocating the use of additive gene-environment interaction instead of the widely used multiplicative scale, as a more relevant public health measure. Using gene-environment independence enhances statistical power for testing multiplicative interaction in case-control studies. However, under departure from this assumption, substantial bias in the estimates and inflated type I error in the corresponding tests can occur. In this paper, we extend the empirical Bayes (EB) approach previously developed for multiplicative interaction, which trades off between bias and efficiency in a data-adaptive way, to the additive scale. An EB estimator of the relative excess risk due to interaction is derived, and the corresponding Wald test is proposed with a general regression setting under a retrospective likelihood framework. We study the impact of gene-environment association on the resultant test with case-control data. Our simulation studies suggest that the EB approach uses the gene-environment independence assumption in a data-adaptive way and provides a gain in power compared with the standard logistic regression analysis and better control of type I error when compared with the analysis assuming gene-environment independence. We illustrate the methods with data from the Ovarian Cancer Association Consortium.
| S-EPMC5860584 | biostudies-literature
Project description:ObjectiveAt least half of youths with mental disorders are unrecognized and untreated. Rapid, accurate assessment of child mental disorders could facilitate identification and referral and potentially reduce the occurrence of functional disability that stems from early-onset mental disorders.MethodComputerized adaptive tests (CATs) based on multidimensional item response theory were developed for depression, anxiety, mania/hypomania, attention-deficit/hyperactivity disorder, conduct disorder, oppositional defiant disorder, and suicidality, based on parent and child ratings of 1,060 items each. In phase 1, CATs were developed from 801 participants. In phase 2, predictive, discriminant, and convergent validity were tested against semi-structured research interviews for diagnoses and suicidality in 497 patients and 104 healthy controls. Overall strength of association was determined by area under the receiver operating characteristic curve (AUC).ResultsThe child and parent independently completed the Kiddie-Computerized Adaptive Tests (K-CATs) in a median time of 7.56 and 5.03 minutes, respectively, with an average of 7 items per domain. The K-CATs accurately captured the presence of diagnoses (AUCs from 0.83 for generalized anxiety disorder to 0.92 for major depressive disorder) and suicidal ideation (AUC = 0.996). Strong correlations with extant measures were found (r ≥ 0.60). Test-retest reliability averaged r = 0.80.ConclusionThese K-CATs provide a new approach to child psychopathology screening and measurement. Testing can be completed by child and parent in less than 8 minutes and yields results that are highly convergent with much more time-consuming structured clinical interviews and dimensional severity assessment and measurement. Testing of the implementation of the K-CAT is now indicated.
| S-EPMC7042076 | biostudies-literature
Project description:Quantum entanglement is a fundamental resource in quantum information processing and its distribution between distant parties is a key challenge in quantum communications. Increasing the dimensionality of entanglement has been shown to improve robustness and channel capacities in secure quantum communications. Here we report on the distribution of genuine high-dimensional entanglement via a 1.2-km-long free-space link across Vienna. We exploit hyperentanglement, that is, simultaneous entanglement in polarization and energy-time bases, to encode quantum information, and observe high-visibility interference for successive correlation measurements in each degree of freedom. These visibilities impose lower bounds on entanglement in each subspace individually and certify four-dimensional entanglement for the hyperentangled system. The high-fidelity transmission of high-dimensional entanglement under real-world atmospheric link conditions represents an important step towards long-distance quantum communications with more complex quantum systems and the implementation of advanced quantum experiments with satellite links.
| S-EPMC5527279 | biostudies-literature
Project description:Local independence is a central assumption of commonly used item response theory models. Violations of this assumption are usually tested using test statistics based on item pairs. This study presents two quasi-exact tests based on the Q3 statistic for testing the hypothesis of local independence in the Rasch model. The proposed tests do not require the estimation of item parameters and can also be applied to small data sets. The authors evaluate the tests with three simulation studies. Their results indicate that the quasi-exact tests hold their alpha level under the Rasch model and have higher power against different forms of local dependence than several alternative parametric and nonparametric model tests for local independence.
| S-EPMC7003184 | biostudies-literature
Project description:In recent years, due to the difficulty and inefficiency of experimental methods, numerous computational methods have been introduced for inferring the structure of Gene Regulatory Networks (GRNs). The Path Consistency (PC) algorithm is one of the popular methods to infer the structure of GRNs. However, this group of methods still has limitations and there is a potential for improvements in this field. For example, the PC-based algorithms are still sensitive to the ordering of nodes i.e. different node orders results in different network structures. The second is that the networks inferred by these methods are highly dependent on the threshold used for independence testing. Also, it is still a challenge to select the set of conditional genes in an optimal way, which affects the performance and computation complexity of the PC-based algorithm. We introduce a novel algorithm, namely Order Independent PC-based algorithm using Quantile value (OIPCQ), which improves the accuracy of the learning process of GRNs and solves the order dependency issue. The quantile-based thresholds are considered for different orders of CMI tests. For conditional gene selection, we consider the paths between genes with length equal or greater than 2 while other well-known PC-based methods only consider the paths of length 2. We applied OIPCQ on the various networks of the DREAM3 and DREAM4 in silico challenges. As a real-world case study, we used OIPCQ to reconstruct SOS DNA network obtained from Escherichia coli and GRN for acute myeloid leukemia based on the RNA sequencing data from The Cancer Genome Atlas. The results show that OIPCQ produces the same network structure for all the permutations of the genes and improves the resulted GRN through accurately quantifying the causal regulation strength in comparison with other well-known PC-based methods. According to the GRN constructed by OIPCQ, for acute myeloid leukemia, two regulators BCLAF1 and NRSF reported previously are significantly important. However, the highest degree nodes in this GRN are ZBTB7A and PU1 which play a significant role in cancer, especially in leukemia. OIPCQ is freely accessible at https://github.com/haammim/OIPCQ-and-OIPCQ2 .
| S-EPMC8027014 | biostudies-literature
Project description:Information about the use and donor site morbidity of periosteal free flaps in head and neck reconstruction is limited. The aim of this study was to examine potential periosteal free flap donor sites with respect to their dimensions, tissue and pedicle characteristics, and predicted donor site morbidity in a cadaveric model. The following cadaveric periosteal specimens with a vascular pedicle were harvested using standard surgical approaches: skull, chest wall, sternum, scapula, iliac crest, femur, and humerus. Data relating to the periosteum size and quality, vascular pedicle, surgical factors, feasibility of use, and the potential donor-site morbidity were recorded. One female (age: 78 years, height: 152 cm) and one male (age: 65 years, height: 186 cm) cadaver were used for flap harvest. The skull, chest wall, scapula, and femur were suitable in terms of the size of the periosteum harvested. The procedure to remove the periosteum from the scalp, chest wall, and scapula had the least predicted donor-site morbidity. The pedicle length and vessel caliber from the periosteal flaps were most favorable from the skull, scapula, and iliac crest. Considering all factors, the periosteum harvested from the skull and scapula were the most promising.
| S-EPMC8489887 | biostudies-literature
Project description:In modern statistical applications, the dimension of covariates can be much larger than the sample size. In the context of linear models, correlation screening (Fan and Lv, 2008) has been shown to reduce the dimension of such data effectively while achieving the sure screening property, i.e., all of the active variables can be retained with high probability. However, screening based on the Pearson correlation does not perform well when applied to contaminated covariates and/or censored outcomes. In this paper, we study censored rank independence screening of high-dimensional survival data. The proposed method is robust to predictors that contain outliers, works for a general class of survival models, and enjoys the sure screening property. Simulations and an analysis of real data demonstrate that the proposed method performs competitively on survival data sets of moderate size and high-dimensional predictors, even when these are contaminated.
| S-EPMC4318124 | biostudies-literature