Dataset Information

An Efficient and Provable Approach for Mixture Proportion Estimation Using Linear Independence Assumption.

ABSTRACT: In this paper, we study the mixture proportion estimation (MPE) problem in a new setting: given samples from the mixture and the component distributions, we identify the proportions of the components in the mixture distribution. To address this problem, we make use of a linear independence assumption, i.e., the component distributions are independent from each other, which is much weaker than assumptions exploited in the previous MPE methods. Based on this assumption, we propose a method (1) that uniquely identifies the mixture proportions, (2) whose output provably converges to the optimal solution, and (3) that is computationally efficient. We show the superiority of the proposed method over the state-of-the-art methods in two applications including learning with label noise and semi-supervised learning on both synthetic and real-world datasets.

SUBMITTER: Yu X

PROVIDER: S-EPMC7034929 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An Efficient and Provable Approach for Mixture Proportion Estimation Using Linear Independence Assumption.

Yu Xiyu X Liu Tongliang T Gong Mingming M Batmanghelich Kayhan K Tao Dacheng D

Conference on Computer Vision and Pattern Recognition Workshops. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Workshops 20180601

In this paper, we study the mixture proportion estimation (MPE) problem in a new setting: given samples from the mixture and the component distributions, we identify the proportions of the components in the mixture distribution. To address this problem, we make use of a linear independence assumption, i.e., the component distributions are independent from each other, which is much weaker than assumptions exploited in the previous MPE methods. Based on this assumption, we propose a method (1) tha ...[more]

PMID: 32089968

Similar Datasets

Project description:In this study we propose an extension of the N-mixture family of models that targets an improvement of the statistical properties of rare species abundance estimators when sample sizes are low, yet typical for tropical studies. The proposed method harnesses information from other species in an ecological community to correct each species' estimator. We provide guidance to determine the sample size required to estimate accurately the abundance of rare tropical species when attempting to estimate the abundance of single species.We evaluate the proposed methods using an assumption of 50 m radius plots and perform simulations comprising a broad range of sample sizes, true abundances and detectability values and a complex data generating process. The extension of the N-mixture model is achieved by assuming that the detection probabilities are drawn at random from a beta distribution in a multi-species fashion. This hierarchical model avoids having to specify a single detection probability parameter per species in the targeted community. Parameter estimation is done via Maximum Likelihood.We compared our multi-species approach with previously proposed multi-species N-mixture models, which we show are biased when the true densities of species in the community are less than seven individuals per 100 hectares. The beta N-mixture model proposed here outperforms the traditional Multi-species N-mixture model by allowing the estimation of organisms at lower densities and controlling the bias in the estimation.We illustrate how our methodology can be used to suggest sample sizes required to estimate the abundance of organisms, when these are either rare, common or abundant. When the interest is full communities, we show how the multi-species approaches, and in particular our beta model and estimation methodology, can be used as a practical solution to estimate organism densities from rapid inventory datasets. The statistical inferences done with our model via Maximum Likelihood can also be used to group species in a community according to their detectabilities.

Dataset Information

An Efficient and Provable Approach for Mixture Proportion Estimation Using Linear Independence Assumption.

Publications

An Efficient and Provable Approach for Mixture Proportion Estimation Using Linear Independence Assumption.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets