Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Detecting Arbitrarily Oriented Subspace Clusters in Data Streams Using Hough Transform

ABSTRACT: When facing high-dimensional data streams, clustering algorithms quickly reach the boundaries of their usefulness as most of these methods are not designed to deal with the curse of dimensionality. Due to inherent sparsity in high-dimensional data, distances between objects tend to become meaningless since the distances between any two objects measured in the full dimensional space tend to become the same for all pairs of objects. In this work, we present a novel oriented subspace clustering algorithm that is able to deal with such issues and detects arbitrarily oriented subspace clusters in high-dimensional data streams. Data streams generally implicate the challenge that the data cannot be stored entirely and hence there is a general demand for suitable data handling strategies for clustering algorithms such that the data can be processed within a single scan. We therefore propose the CashStream algorithm that unites state-of-the-art stream processing techniques and additionally relies on the Hough transform to detect arbitrarily oriented subspace clusters. Our experiments compare CashStream to its static counterpart and show that the amount of consumed memory is significantly decreased while there is no loss in terms of runtime. Electronic supplementary material The online version of this chapter (10.1007/978-3-030-47426-3_28) contains supplementary material, which is available to authorized users.

SUBMITTER: Lauw H

PROVIDER: S-EPMC7206268 | biostudies-literature | 2020 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Identifying subspace gene clusters from microarray data using low-rank representation.

Project description:Identifying subspace gene clusters from the gene expression data is useful for discovering novel functional gene interactions. In this paper, we propose to use low-rank representation (LRR) to identify the subspace gene clusters from microarray data. LRR seeks the lowest-rank representation among all the candidates that can represent the genes as linear combinations of the bases in the dataset. The clusters can be extracted based on the block diagonal representation matrix obtained using LRR, and they can well capture the intrinsic patterns of genes with similar functions. Meanwhile, the parameter of LRR can balance the effect of noise so that the method is capable of extracting useful information from the data with high level of background noise. Compared with traditional methods, our approach can identify genes with similar functions yet without similar expression profiles. Also, it could assign one gene into different clusters. Moreover, our method is robust to the noise and can identify more biologically relevant gene clusters. When applied to three public datasets, the results show that the LRR based method is superior to existing methods for identifying subspace gene clusters.

| S-EPMC3602020 | biostudies-literature

Accelerating Fourier Transform-Ion Cyclotron Resonance Mass Spectrometry Imaging Using a Subspace Approach.

Project description:We present a subspace method that accelerates data acquisition using Fourier transform-ion cyclotron resonance (FT-ICR) mass spectrometry imaging (MSI). For MSI of biological tissue samples, there is a finite number of heterogeneous tissue types with distinct chemical profiles that introduce redundancy in the high-dimensional measurements. Our subspace model exploits the redundancy in data measured from whole-slice tissue samples by decomposing the transient signals into linear combinations of a set of basis transients with the desired spectral resolution. This decomposition allowed us to design a strategy that acquires a subset of long transients for basis determination and short transients for the remaining pixels, drastically reducing the acquisition time. The computational reconstruction strategy can maintain high-mass-resolution and spatial-resolution MSI while providing a 10-fold improvement in throughput. We validated the capability of the subspace model using a rat sagittal brain slice imaging data set. Comprehensive evaluation of the quality of the mass spectral and ion images demonstrated that the reconstructed data produced by the reported method required only 15% of the typical acquisition time and exhibited both qualitative and quantitative consistency when compared to the original data. Our method enables either higher sample throughput or higher-resolution images at similar acquisition lengths, providing greater flexibility in obtaining FT-ICR MSI measurements.

| S-EPMC7682253 | biostudies-literature

Radon Cumulative Distribution Transform Subspace Modeling for Image Classification.

Project description:We present a new supervised image classification method applicable to a broad class of image deformation models. The method makes use of the previously described Radon Cumulative Distribution Transform (R-CDT) for image data, whose mathematical properties are exploited to express the image data in a form that is more suitable for machine learning. While certain operations such as translation, scaling, and higher-order transformations are challenging to model in native image space, we show the R-CDT can capture some of these variations and thus render the associated image classification problems easier to solve. The method - utilizing a nearest-subspace algorithm in the R-CDT space - is simple to implement, non-iterative, has no hyper-parameters to tune, is computationally efficient, label efficient, and provides competitive accuracies to state-of-the-art neural networks for many types of classification problems. In addition to the test accuracy performances, we show improvements (with respect to neural network-based methods) in terms of computational efficiency (it can be implemented without the use of GPUs), number of training samples needed for training, as well as out-of-distribution generalization. The Python code for reproducing our results is available at [1].

| S-EPMC9032314 | biostudies-literature

Detecting Anatomical Landmarks From Limited Medical Imaging Data Using Two-Stage Task-Oriented Deep Neural Networks.

Project description:One of the major challenges in anatomical landmark detection, based on deep neural networks, is the limited availability of medical imaging data for network learning. To address this problem, we present a two-stage task-oriented deep learning method to detect large-scale anatomical landmarks simultaneously in real time, using limited training data. Specifically, our method consists of two deep convolutional neural networks (CNN), with each focusing on one specific task. Specifically, to alleviate the problem of limited training data, in the first stage, we propose a CNN based regression model using millions of image patches as input, aiming to learn inherent associations between local image patches and target anatomical landmarks. To further model the correlations among image patches, in the second stage, we develop another CNN model, which includes a) a fully convolutional network that shares the same architecture and network weights as the CNN used in the first stage and also b) several extra layers to jointly predict coordinates of multiple anatomical landmarks. Importantly, our method can jointly detect large-scale (e.g., thousands of) landmarks in real time. We have conducted various experiments for detecting 1200 brain landmarks from the 3D T1-weighted magnetic resonance images of 700 subjects, and also 7 prostate landmarks from the 3D computed tomography images of 73 subjects. The experimental results show the effectiveness of our method regarding both accuracy and efficiency in the anatomical landmark detection.

| S-EPMC5729285 | biostudies-literature

Detecting and removing data artifacts in Hadamard transform ion mobility-mass spectrometry measurements.

Project description:Applying Hadamard transform multiplexing to ion mobility separations (IMS) can significantly improve the signal-to-noise ratio and throughput for IMS coupled mass spectrometry (MS) measurements by increasing the ion utilization efficiency. However, it has been determined that fluctuations in ion intensity as well as spatial shifts in the multiplexed data lower the signal-to-noise ratios and appear as noise in downstream processing of the data. To address this problem, we have developed a novel algorithm that discovers and eliminates data artifacts. The algorithm employs an analytical approach to identify and remove artifacts from the data, decreasing the likelihood of false identifications in subsequent data processing. Following application of the algorithm, IMS-MS measurement sensitivity is greatly increased and artifacts that previously limited the utility of applying the Hadamard transform to IMS are avoided. Figure ?

| S-EPMC4223016 | biostudies-literature

Detecting clusters of mutations.

Project description:Positive selection for protein function can lead to multiple mutations within a small stretch of DNA, i.e., to a cluster of mutations. Recently, Wagner proposed a method to detect such mutation clusters. His method, however, did not take into account that residues with high solvent accessibility are inherently more variable than residues with low solvent accessibility. Here, we propose a new algorithm to detect clustered evolution. Our algorithm controls for different substitution probabilities at buried and exposed sites in the tertiary protein structure, and uses random permutations to calculate accurate P values for inferred clusters. We apply the algorithm to genomes of bacteria, fly, and mammals, and find several clusters of mutations in functionally important regions of proteins. Surprisingly, clustered evolution is a relatively rare phenomenon. Only between 2% and 10% of the genes we analyze contain a statistically significant mutation cluster. We also find that not controlling for solvent accessibility leads to an excess of clusters in terminal and solvent-exposed regions of proteins. Our algorithm provides a novel method to identify functionally relevant divergence between groups of species. Moreover, it could also be useful to detect artifacts in automatically assembled genomes.

| S-EPMC2582452 | biostudies-literature

To transform or not to transform: using generalized linear mixed models to analyse reaction time data.

Project description:Linear mixed-effect models (LMMs) are being increasingly widely used in psychology to analyse multi-level research designs. This feature allows LMMs to address some of the problems identified by Speelman and McGann (2013) about the use of mean data, because they do not average across individual responses. However, recent guidelines for using LMM to analyse skewed reaction time (RT) data collected in many cognitive psychological studies recommend the application of non-linear transformations to satisfy assumptions of normality. Uncritical adoption of this recommendation has important theoretical implications which can yield misleading conclusions. For example, Balota et al. (2013) showed that analyses of raw RT produced additive effects of word frequency and stimulus quality on word identification, which conflicted with the interactive effects observed in analyses of transformed RT. Generalized linear mixed-effect models (GLMM) provide a solution to this problem by satisfying normality assumptions without the need for transformation. This allows differences between individuals to be properly assessed, using the metric most appropriate to the researcher's theoretical context. We outline the major theoretical decisions involved in specifying a GLMM, and illustrate them by reanalysing Balota et al.'s datasets. We then consider the broader benefits of using GLMM to investigate individual differences.

| S-EPMC4528092 | biostudies-literature

Oriented assembly of polyhedral plasmonic nanoparticle clusters.

Project description:Shaped colloids can be used as nanoscale building blocks for the construction of composite, functional materials that are completely assembled from the bottom up. Assemblies of noble metal nanostructures have unique optical properties that depend on key structural features requiring precise control of both position and connectivity spanning nanometer to micrometer length scales. Identifying and optimizing structures that strongly couple to light is important for understanding the behavior of surface plasmons in small nanoparticle clusters, and can result in highly sensitive chemical and biochemical sensors using surface-enhanced Raman spectroscopy (SERS). We use experiment and simulation to examine the local surface plasmon resonances of different arrangements of Ag polyhedral clusters. High-resolution transmission electron microscopy shows that monodisperse, atomically smooth Ag polyhedra can self-assemble into uniform interparticle gaps that result in reproducible SERS enhancement factors from assembly to assembly. We introduce a large-scale, gravity-driven assembly method that can generate arbitrary nanoparticle clusters based on the size and shape of a patterned template. These templates enable the systematic examination of different cluster arrangements and provide a means of constructing scalable and reliable SERS sensors.

| S-EPMC3637711 | biostudies-literature

Motion correction for PET data using subspace-based real-time MR imaging in simultaneous PET/MR.

Project description:Image quality of positron emission tomography (PET) reconstructions is degraded by subject motion occurring during the acquisition. Magnetic resonance (MR)-based motion correction approaches have been studied for PET/MR scanners and have been successful at capturing regular motion patterns, when used in conjunction with surrogate signals (e.g. navigators) to detect motion. However, handling irregular respiratory motion and bulk motion remains challenging. In this work, we propose an MR-based motion correction method relying on subspace-based real-time MR imaging to estimate motion fields used to correct PET reconstructions. We take advantage of the low-rank characteristics of dynamic MR images to reconstruct high-resolution MR images at high frame rates from highly undersampled k-space data. Reconstructed dynamic MR images are used to determine motion phases for PET reconstruction and estimate phase-to-phase nonrigid motion fields able to capture complex motion patterns such as irregular respiratory and bulk motion. MR-derived binning and motion fields are used for PET reconstruction to generate motion-corrected PET images. The proposed method was evaluated on in vivo data with irregular motion patterns. MR reconstructions accurately captured motion, outperforming state-of-the-art dynamic MR reconstruction techniques. Evaluation of PET reconstructions demonstrated the benefits of the proposed method in terms of motion artifacts reduction, improving the contrast-to-noise ratio by up to a factor 3 and achieveing a target-to-background ratio up to 90% superior compared to standard/uncorrected methods. The proposed method can improve the image quality of motion-corrected PET reconstructions in clinical applications.

| S-EPMC7985095 | biostudies-literature

RNA arbitrarily primed PCR and fourier transform infrared spectroscopy reveal plasticity in the acid tolerance response of Streptococcus macedonicus.

Project description:We have previously reported that an acid tolerance response (ATR) can be induced in Streptococcus macedonicus cells at mid-log phase after autoacidification, transient exposure to acidic pH, or acid habituation, as well as at stationary phase. Here, we compared the transcriptional profiles of these epigenetic phenotypes, by RNA arbitrarily primed PCR (RAP-PCR), and their whole-cell chemical compositions, by Fourier transform infrared spectroscopy (FT-IR). RAP-PCR fingerprints revealed significant differences among the phenotypes, indicating that gene expression during the ATR is influenced not only by the growth phase but also by the treatments employed to induce the response. The genes coding for the mannose-specific IID component, the 1,2-diacylglycerol 3-glucosyltransferase, the 3-oxoacyl-acyl carrier protein, the large subunit of carbamoyl-phosphate synthase, and a hypothetical protein were found to be induced at least under some of the acid-adapting conditions. Furthermore, principal component analysis of the second-derivative-transformed FT-IR spectra segregated S. macedonicus phenotypes individually in all spectral regions that are characteristic for major cellular constituents like the polysaccharides of the cell wall, fatty acids of the cell membrane, proteins, and other compounds that absorb in these regions. These findings provide evidence for major changes in cellular composition due to acid adaptation that were clearly different to some extent among the phenotypes. Overall, our data demonstrate the plasticity in the ATR of S. macedonicus, which reflects the inherent ability of the bacterium to adjust the response to the distinctiveness of the imposed stress condition, probably to maximize its adaptability.

| S-EPMC2565966 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data