Subsampling scaling.
Ontology highlight
ABSTRACT:
SUBMITTER: Levina A
PROVIDER: S-EPMC5418619 | biostudies-literature | 2017 May
REPOSITORIES: biostudies-literature
Publications
In real-world applications, observations are often constrained to a small fraction of a system. Such spatial subsampling can be caused by the inaccessibility or the sheer size of the system, and cannot be overcome by longer sampling. Spatial subsampling can strongly bias inferences about a system's aggregated properties. To overcome the bias, we derive analytically a subsampling scaling framework that is applicable to different observables, including distributions of neuronal avalanches, of numb ...[more]
Similar Datasets
Project description:BackgroundAnalysis of species count data in ecology often requires normalization to an identical sample size. Rarefying (random subsampling without replacement), which is the current standard method for normalization, has been widely criticized for its poor reproducibility and potential distortion of the community structure. In the context of microbiome count data, researchers explicitly advised against the use of rarefying. Here we introduce a normalization method for species count data called scaling with ranked subsampling (SRS) and demonstrate its suitability for the analysis of microbial communities.MethodsSRS consists of two steps. In the scaling step, the counts for all species or operational taxonomic units (OTUs) are divided by a scaling factor chosen in such a way that the sum of scaled counts equals the selected total number of counts Cmin. The relative frequencies of all OTUs remain unchanged. In the subsequent ranked subsampling step, non-integer count values are converted into integers by an algorithm that minimizes subsampling error with regard to the population structure (relative frequencies of species or OTUs) while keeping the total number of counts equal Cmin. SRS and rarefying were compared by normalizing a test library representing a soil bacterial community. Common parameters of biodiversity and population structure (Shannon index H', species richness, species composition, and relative abundances of OTUs) were determined for libraries normalized to different size by rarefying as well as SRS with 10,000 replications each. An implementation of SRS in R is available for download (https://doi.org/10.20387/BONARES-2657-1NP3).ResultsSRS showed greater reproducibility and preserved OTU frequencies and alpha diversity better than rarefying. The variance in Shannon diversity increased with the reduction of the library size after rarefying but remained zero for SRS. Relative abundances of OTUs strongly varied among libraries generated by rarefying, whereas libraries normalized by SRS showed only negligible variation. Bray-Curtis index of dissimilarity among replicates of the same library normalized by rarefying revealed a large variation in species composition, which reached complete dissimilarity (not a single OTU shared) among some libraries rarefied to a small size. The dissimilarity among replicated libraries normalized by SRS remained negligibly low at each library size. The variance in dissimilarity increased with the decreasing library size after rarefying, whereas it remained either zero or negligibly low after SRS.ConclusionsNormalization of OTU or species counts by scaling with ranked subsampling preserves the original community structure by minimizing subsampling errors. We therefore propose SRS for the normalization of biological count data.
| S-EPMC7409812 | biostudies-literature
Project description:It is often necessary to extract a small amount of a suspension, such as blood, from a larger sample of the same material for the purposes of diagnostics, testing or imaging. A practical challenge is that the cells in blood sediment noticeably on the time scale of a few minutes, making a representative subsampling of the original sample challenging. Guided by experimental data, we develop a Kynch sedimentation model to discuss design considerations that ensure a representative subsampling of blood, from a container of constant cross-sectional area, for the entire range of physiologically relevant hematocrit over a specified time of interest. Additionally, we show that this design may be modified to exploit the sedimentation and perform subsampling to achieve either higher or lower hematocrit relative to that of the original sample. Thus, our method provides a simple tool to either concentrate or dilute small quantities of blood or other sedimenting suspensions.
| S-EPMC6694302 | biostudies-literature
Project description:For massive data, the family of subsampling algorithms is popular to downsize the data volume and reduce computational burden. Existing studies focus on approximating the ordinary least squares estimate in linear regression, where statistical leverage scores are often used to define subsampling probabilities. In this paper, we propose fast subsampling algorithms to efficiently approximate the maximum likelihood estimate in logistic regression. We first establish consistency and asymptotic normality of the estimator from a general subsampling algorithm, and then derive optimal subsampling probabilities that minimize the asymptotic mean squared error of the resultant estimator. An alternative minimization criterion is also proposed to further reduce the computational cost. The optimal subsampling probabilities depend on the full data estimate, so we develop a two-step algorithm to approximate the optimal subsampling procedure. This algorithm is computationally efficient and has a significant reduction in computing time compared to the full data approach. Consistency and asymptotic normality of the estimator from a two-step algorithm are also established. Synthetic and real data sets are used to evaluate the practical performance of the proposed method.
| S-EPMC6075720 | biostudies-literature
Project description:Biased population samples pose a prevalent problem in the social sciences. Therefore, we present two novel methods that are based on positive-unlabeled learning to mitigate bias. Both methods leverage auxiliary information from a representative data set and train machine learning classifiers to determine the sample weights. The first method, named maximum representative subsampling (MRS), uses a classifier to iteratively remove instances, by assigning a sample weight of 0, from the biased data set until it aligns with the representative one. The second method is a variant of MRS - Soft-MRS - that iteratively adapts sample weights instead of removing samples completely. To assess the effectiveness of our approach, we induced artificial bias in a public census data set and examined the corrected estimates. We compare the performance of our methods against existing techniques, evaluating the ability of sample weights created with Soft-MRS or MRS to minimize differences and improve downstream classification tasks. Lastly, we demonstrate the applicability of the proposed methods in a real-world study of resilience research, exploring the influence of resilience on voting behavior. Through our work, we address the issue of bias in social science, amongst others, and provide a versatile methodology for bias reduction based on machine learning. Based on our experiments, we recommend to use MRS for downstream classification tasks and Soft-MRS for downstream tasks where the relative bias of the dependent variable is relevant.
| S-EPMC10684887 | biostudies-literature
Project description:Combinatorial genetics reveals a scaling law for the effects of mutations on splicing
| PRJEB24588 | ENA
Project description:Career planning consists of a series of decisions that will significantly impact one's life. However, current recommendation systems have serious limitations, including the lack of effective artificial intelligence algorithms for long-term career planning, and the lack of efficient reinforcement learning (RL) methods for dynamic systems. To improve the long-term recommendation, this work proposes an intelligent sequential career planning system featuring a career path rating mechanism and a new RL method coined as the stochastic subsampling reinforcement learning (SSRL) framework. After proving the effectiveness of this new recommendation system theoretically, we evaluate it computationally by gauging it against several benchmarks under different scenarios representing different user preferences in career planning. Numerical results have demonstrated that our system is superior to other benchmarks in locating promising optimal career paths for users in long-term planning. Case studies have further revealed that our SSRL career path recommendation system would encourage people to gradually improve their career paths to maximize long-term benefits. Moreover, we have shown that the initial state (i.e., the first job) can have a significant impact, positively or negatively, on one's career, while in the long-term view, a carefully planned career path following our recommendation system may mitigate the negative impact of a lackluster beginning in one's career life.
| S-EPMC9117248 | biostudies-literature
Project description:BACKGROUND:The analysis of large mosquito samples is expensive and time-consuming, delaying the efficient timing of vector control measurements. Processing a fraction of a sample using a subsampling method can significantly reduce the processing effort. However, a comprehensive evaluation of the reliability of different subsampling methods is missing. METHODS:A total of 23 large mosquito samples (397-4713 specimens per sample) were compared in order to evaluate five subsampling methods for the estimation of the number of specimens and species: area, volume, weight, selection of 200 random specimens and analyses with an image processing software. Each sample was distributed over a grid paper (21.0?×?29.7 cm; 25 grid cells of 4.2?×?5.9 cm) with 200 randomly distributed points. After taking pictures, mosquito specimens closest to each of the 200 points on the paper were selected. All mosquitoes per grid cell were identified by morphology and transferred to scaled tubes to estimate the volume. Finally, the fresh and dry weights were determined. RESULTS:The estimated number of specimens and species did not differ between the area-, volume- and weight-based method. Subsampling 20% of the sample gave an error rate of approximately 12% for the number of specimens, 6% for the proportion of the most abundant species and between 6-40% for the number of species per sample. The error for the estimated number of specimens using the picture processing software ImageJ gave a similar error rate when analyzing 15-20% of the total sample. By using 200 randomly selected specimens it was possible to give a precise estimation of the proportion of the most abundant species (r?=?0.97, P?<?0.001), but the number of species per sample was underestimated by 28% on average. Selecting adjacent grid cells instead of sampling randomly chosen grid cells and using dry weight instead of wet weight did not increase the accuracy of estimates. CONCLUSIONS:Different subsampling methods have various advantages and disadvantages. However, the area-based analysis of 20% of the sample is probably the most suitable approach for most kinds of mosquito studies, giving sufficiently precise estimations of the number of specimens and species, which is slightly less laborious compared to the other methods tested.
| S-EPMC6636137 | biostudies-literature
Project description:BackgroundMany systems in nature are characterized by complex behaviour where large cascades of events, or avalanches, unpredictably alternate with periods of little activity. Snow avalanches are an example. Often the size distribution f(s) of a system's avalanches follows a power law, and the branching parameter sigma, the average number of events triggered by a single preceding event, is unity. A power law for f(s), and sigma = 1, are hallmark features of self-organized critical (SOC) systems, and both have been found for neuronal activity in vitro. Therefore, and since SOC systems and neuronal activity both show large variability, long-term stability and memory capabilities, SOC has been proposed to govern neuronal dynamics in vivo. Testing this hypothesis is difficult because neuronal activity is spatially or temporally subsampled, while theories of SOC systems assume full sampling. To close this gap, we investigated how subsampling affects f(s) and sigma by imposing subsampling on three different SOC models. We then compared f(s) and sigma of the subsampled models with those of multielectrode local field potential (LFP) activity recorded in three macaque monkeys performing a short term memory task.ResultsNeither the LFP nor the subsampled SOC models showed a power law for f(s). Both, f(s) and sigma, depended sensitively on the subsampling geometry and the dynamics of the model. Only one of the SOC models, the Abelian Sandpile Model, exhibited f(s) and sigma similar to those calculated from LFP activity.ConclusionSince subsampling can prevent the observation of the characteristic power law and sigma in SOC systems, misclassifications of critical systems as sub- or supercritical are possible. Nevertheless, the system specific scaling of f(s) and sigma under subsampling conditions may prove useful to select physiologically motivated models of brain function. Models that better reproduce f(s) and sigma calculated from the physiological recordings may be selected over alternatives.
| S-EPMC2697147 | biostudies-literature
Project description:ObjectiveDespite the move to at-home, small-volume collection kits to facilitate large population-based studies of faecal microbial compositional profiling, there remains limited reporting on potential impacts of faecal subsampling approaches on compositional profiles. This study aimed to compare the microbial composition from faecal subsamples (< 5 g) collected from the beginning and end of a single bowel movement in ten otherwise healthy adults (6 female, 4 male; age: 24-55 years). Microbial composition was determined by V3-V4 16s rRNA sequencing and compared between subsamples.ResultsThere were no significant differences in OTU count (p = 0.32) or Shannon diversity index (p = 0.29) between the subsamples. Comparison of relative abundance for identified taxa revealed very few differences between subsamples. At the lower levels of taxonomic classification differences in abundance of the Bacillales (p = 0.02) and the Eubacteriaceae family (p = 0.03), and the Eubacterium genera (p = 0.03) were noted. The observation of consistent microbial compositional profiles between faecal subsamples from the beginning and end of a single bowel movement is an important outcome for study designs employing this approach to faecal sample collection. These findings provide assurance that use of a faecal subsample for microbial composition profiling is generally representative of the gut luminal contents more broadly.
| S-EPMC8842933 | biostudies-literature
Project description:Phylogenomic subsampling is a procedure by which small sets of loci are selected from large genome-scale data sets and used for phylogenetic inference. This step is often motivated by either computational limitations associated with the use of complex inference methods or as a means of testing the robustness of phylogenetic results by discarding loci that are deemed potentially misleading. Although many alternative methods of phylogenomic subsampling have been proposed, little effort has gone into comparing their behavior across different data sets. Here, I calculate multiple gene properties for a range of phylogenomic data sets spanning animal, fungal, and plant clades, uncovering a remarkable predictability in their patterns of covariance. I also show how these patterns provide a means for ordering loci by both their rate of evolution and their relative phylogenetic usefulness. This method of retrieving phylogenetically useful loci is found to be among the top performing when compared with alternative subsampling protocols. Relatively common approaches such as minimizing potential sources of systematic bias or increasing the clock-likeness of the data are found to fare worse than selecting loci at random. Likewise, the general utility of rate-based subsampling is found to be limited: loci evolving at both low and high rates are among the least effective, and even those evolving at optimal rates can still widely differ in usefulness. This study shows that many common subsampling approaches introduce unintended effects in off-target gene properties and proposes an alternative multivariate method that simultaneously optimizes phylogenetic signal while controlling for known sources of bias.
| S-EPMC8382905 | biostudies-literature