Dataset Information

A fully scalable online pre-processing algorithm for short oligonucleotide microarray atlases.

ABSTRACT: Rapid accumulation of large and standardized microarray data collections is opening up novel opportunities for holistic characterization of genome function. The limited scalability of current preprocessing techniques has, however, formed a bottleneck for full utilization of these data resources. Although short oligonucleotide arrays constitute a major source of genome-wide profiling data, scalable probe-level techniques have been available only for few platforms based on pre-calculated probe effects from restricted reference training sets. To overcome these key limitations, we introduce a fully scalable online-learning algorithm for probe-level analysis and pre-processing of large microarray atlases involving tens of thousands of arrays. In contrast to the alternatives, our algorithm scales up linearly with respect to sample size and is applicable to all short oligonucleotide platforms. The model can use the most comprehensive data collections available to date to pinpoint individual probes affected by noise and biases, providing tools to guide array design and quality control. This is the only available algorithm that can learn probe-level parameters based on sequential hyperparameter updates at small consecutive batches of data, thus circumventing the extensive memory requirements of the standard approaches and opening up novel opportunities to take full advantage of contemporary microarray collections.

SUBMITTER: Lahti L

PROVIDER: S-EPMC3664815 | biostudies-literature | 2013 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A fully scalable online pre-processing algorithm for short oligonucleotide microarray atlases.

Lahti Leo L Torrente Aurora A Elo Laura L LL Brazma Alvis A Rung Johan J

Nucleic acids research 20130405 10

Rapid accumulation of large and standardized microarray data collections is opening up novel opportunities for holistic characterization of genome function. The limited scalability of current preprocessing techniques has, however, formed a bottleneck for full utilization of these data resources. Although short oligonucleotide arrays constitute a major source of genome-wide profiling data, scalable probe-level techniques have been available only for few platforms based on pre-calculated probe eff ...[more]

PMID: 23563154

Similar Datasets

Project description:BackgroundThe human gastrointestinal (GI) tract contains a diverse collection of bacteria, most of which are unculturable by conventional microbiological methods. Increasingly molecular profiling techniques are being employed to examine this complex microbial community. The purpose of this study was to develop a microarray technique based on 16S ribosomal gene sequences for rapidly monitoring the microbial population of the GI tract.ResultsWe have developed a culture-independent, semi-quantitative, rapid method for detection of gut bacterial populations based on 16S rDNA probes using a DNA microarray. We compared the performance of microarrays based on long (40- and 50-mer) and short (16-21-mer) oligonucleotides. Short oligonucleotides consistently gave higher specificity. Optimal DNA amplification and labelling, hybridisation and washing conditions were determined using a probe with an increasing number of nucleotide mismatches, identifying the minimum number of nucleotides needed to distinguish between perfect and mismatch probes. An independent PCR-based control was used to normalise different hybridisation results, and to make comparisons between different samples, greatly improving the detection of changes in the gut bacterial population. The sensitivity of the microarray was determined to be 8.8 x 104 bacterial cells g-1 faecal sample, which is more sensitive than a number of existing profiling methods. The short oligonucleotide microarray was used to compare the faecal flora from healthy individuals and a patient suffering from Ulcerative Colitis (UC) during the active and remission states. Differences were identified in the bacterial profiles between healthy individuals and a UC patient. These variations were verified by Denaturing Gradient Gel Electrophoresis (DGGE) and DNA sequencing.ConclusionIn this study we demonstrate the design, testing and application of a highly sensitive, short oligonucleotide community microarray. Our approach allows the rapid discrimination of bacteria inhabiting the human GI tract, at taxonomic levels ranging from species to the superkingdom bacteria. The optimised protocol is available at: http://www.ifr.ac.uk/safety/microarrays/#protocols. It offers a high throughput method for studying the dynamics of the bacterial population over time and between individuals.

Dataset Information

A fully scalable online pre-processing algorithm for short oligonucleotide microarray atlases.

Publications

A fully scalable online pre-processing algorithm for short oligonucleotide microarray atlases.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets