Meffil: efficient normalization and analysis of very large DNA methylation datasets.
Ontology highlight
ABSTRACT: Motivation
DNA methylation datasets are growing ever larger both in sample size and genome coverage. Novel computational solutions are required to efficiently handle these data.Results
We have developed meffil, an R package designed for efficient quality control, normalization and epigenome-wide association studies of large samples of Illumina Methylation BeadChip microarrays. A complete re-implementation of functional normalization minimizes computational memory without increasing running time. Incorporating fixed and random effects within functional normalization, and automated estimation of functional normalization parameters reduces technical variation in DNA methylation levels, thus reducing false positive rates and improving power. Support for normalization of datasets distributed across physically different locations without needing to share biologically-based individual-level data means that meffil can be used to reduce heterogeneity in meta-analyses of epigenome-wide association studies.Availability and implementation
https://github.com/perishky/meffil/.Supplementary information
Supplementary data are available at Bioinformatics online.
SUBMITTER: Min JL
PROVIDER: S-EPMC6247925 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:MotivationThe datasets generated by DNA methylation analyses are getting bigger. With the release of the HumanMethylationEPIC micro-array and datasets containing thousands of samples, analyses of these large datasets using R are becoming impractical due to large memory requirements. As a result there is an increasing need for computationally efficient methodologies to perform meaningful analysis on high dimensional data.ResultsHere we introduce the bigmelon R package, which provides a memory efficient workflow that enables users to perform the complex, large scale analyses required in epigenome wide association studies (EWAS) without the need for large RAM. Building on top of the CoreArray Genomic Data Structure file format and libraries packaged in the gdsfmt package, we provide a practical workflow that facilitates the reading-in, preprocessing, quality control and statistical analysis of DNA methylation data.We demonstrate the capabilities of the bigmelon package using a large dataset consisting of 1193 human blood samples from the Understanding Society: UK Household Longitudinal Study, assayed on the EPIC micro-array platform.Availability and implementationThe bigmelon package is available on Bioconductor (http://bioconductor.org/packages/bigmelon/). The Understanding Society dataset is available at https://www.understandingsociety.ac.uk/about/health/data upon request.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC6419913 | biostudies-literature
Project description:MotivationThe rapidly expanding microbiomics field is generating increasingly larger datasets, characterizing the microbiota in diverse environments. Although classical numerical ecology methods provide a robust statistical framework for their analysis, software currently available is inadequate for large datasets and some computationally intensive tasks, like rarefaction and associated analysis.ResultsHere we present a software package for rarefaction analysis of large count matrices, as well as estimation and visualization of diversity, richness and evenness. Our software is designed for ease of use, operating at least 7x faster than existing solutions, despite requiring 10x less memory.Availability and implementationC ++ and R source code (GPL v.2) as well as binaries are available from https://github.com/hildebra/Rarefaction and from CRAN (https://cran.r-project.org/).Contactbork@embl.de or falk.hildebrand@embl.de.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC5870771 | biostudies-literature
Project description:BACKGROUND: The scale and diversity of metagenomic sequencing projects challenge both our technical and conceptual approaches in gene and genome annotations. The recent Sorcerer II Global Ocean Sampling (GOS) expedition yielded millions of predicted protein sequences, which significantly altered the landscape of known protein space by more than doubling its size and adding thousands of new families (Yooseph et al., 2007 PLoS Biol 5, e16). Such datasets, not only by their sheer size, but also by many other features, defy conventional analysis and annotation methods. METHODOLOGY/PRINCIPAL FINDINGS: In this study, we describe an approach for rapid analysis of the sequence diversity and the internal structure of such very large datasets by advanced clustering strategies using the newly modified CD-HIT algorithm. We performed a hierarchical clustering analysis on the 17.4 million Open Reading Frames (ORFs) identified from the GOS study and found over 33 thousand large predicted protein clusters comprising nearly 6 million sequences. Twenty percent of these clusters did not match known protein families by sequence similarity search and might represent novel protein families. Distributions of the large clusters were illustrated on organism composition, functional class, and sample locations. CONCLUSION/SIGNIFICANCE: Our clustering took about two orders of magnitude less computational effort than the similar protein family analysis of original GOS study. This approach will help to analyze other large metagenomic datasets in the future. A Web server with our clustering results and annotations of predicted protein clusters is available online at http://tools.camera.calit2.net/gos under the CAMERA project.
| S-EPMC2557142 | biostudies-literature
Project description:Advancements in cytometry technologies have enabled quantification of up to 50 proteins across millions of cells at single cell resolution. Analysis of cytometry data routinely involves tasks such as data integration, clustering, and dimensionality reduction. While numerous tools exist, many require extensive run times when processing large cytometry data containing millions of cells. Existing solutions, such as random subsampling, are inadequate as they risk excluding rare cell subsets. To address this, we propose SuperCellCyto, an R package that builds on the SuperCell tool which groups highly similar cells into supercells. SuperCellCyto is available on GitHub ( https://github.com/phipsonlab/SuperCellCyto ) and Zenodo ( https://doi.org/10.5281/zenodo.10521294 ).
| S-EPMC11003185 | biostudies-literature
Project description:MotivationAdvances in high-throughput sequencing have resulted in rapid growth in large, high-quality datasets including those arising from transcription factor (TF) ChIP-seq experiments. While there are many existing tools for discovering TF binding site motifs in such datasets, most web-based tools cannot directly process such large datasets.ResultsThe MEME-ChIP web service is designed to analyze ChIP-seq 'peak regions'--short genomic regions surrounding declared ChIP-seq 'peaks'. Given a set of genomic regions, it performs (i) ab initio motif discovery, (ii) motif enrichment analysis, (iii) motif visualization, (iv) binding affinity analysis and (v) motif identification. It runs two complementary motif discovery algorithms on the input data--MEME and DREME--and uses the motifs they discover in subsequent visualization, binding affinity and identification steps. MEME-ChIP also performs motif enrichment analysis using the AME algorithm, which can detect very low levels of enrichment of binding sites for TFs with known DNA-binding motifs. Importantly, unlike with the MEME web service, there is no restriction on the size or number of uploaded sequences, allowing very large ChIP-seq datasets to be analyzed. The analyses performed by MEME-ChIP provide the user with a varied view of the binding and regulatory activity of the ChIP-ed TF, as well as the possible involvement of other DNA-binding TFs.AvailabilityMEME-ChIP is available as part of the MEME Suite at http://meme.nbcr.net.
| S-EPMC3106185 | biostudies-literature
Project description:Information processing functions are essential for organisms to perceive and react to their complex environment, and for humans to analyze and rationalize them. While our brain is extraordinary at processing complex information, winner-take-all, as a type of biased competition is one of the simplest models of lateral inhibition and competition among biological neurons. It has been implemented as DNA-based neural networks, for example, to mimic pattern recognition. However, the utility of DNA-based computation in information processing for real biotechnological applications remains to be demonstrated. In this paper, a biased competition method for nonlinear manipulation and analysis of mixtures of DNA sequences was developed. Unlike conventional biological experiments, selected species were not directly subjected to analysis. Instead, parallel computation among a myriad of different DNA sequences was carried out to reduce the information entropy. The method could be used for various oligonucleotide-encoded libraries, as we have demonstrated its application in decoding and data analysis for selection experiments with DNA-encoded chemical libraries against protein targets.
| S-EPMC9410889 | biostudies-literature
Project description:The elucidation of orthology relationships is an important step both in gene function prediction as well as towards understanding patterns of sequence evolution. Orthology assignments are usually derived directly from sequence similarities for large data because more exact approaches exhibit too high computational costs. Here we present PoFF, an extension for the standalone tool Proteinortho, which enhances orthology detection by combining clustering, sequence similarity, and synteny. In the course of this work, FFAdj-MCS, a heuristic that assesses pairwise gene order using adjacencies (a similarity measure related to the breakpoint distance) was adapted to support multiple linear chromosomes and extended to detect duplicated regions. PoFF largely reduces the number of false positives and enables more fine-grained predictions than purely similarity-based approaches. The extension maintains the low memory requirements and the efficient concurrency options of its basis Proteinortho, making the software applicable to very large datasets.
| S-EPMC4138177 | biostudies-literature
Project description:BackgroundDNA methylation is an important mechanism of epigenetic regulation in development and disease. New generation sequencers allow genome-wide measurements of the methylation status by reading short stretches of the DNA sequence (Methyl-seq). Several software tools for methylation analysis have been proposed over recent years. However, the current trend is that the new sequencers and the ones expected for an upcoming future yield sequences of increasing length, making these software tools inefficient and obsolete.ResultsIn this paper, we propose a new software based on a strategy for methylation analysis of Methyl-seq sequencing data that requires much shorter execution times while yielding a better level of sensitivity, particularly for datasets composed of long reads. This strategy can be exported to other methylation, DNA and RNA analysis tools.ConclusionsThe developed software tool achieves execution times one order of magnitude shorter than the existing tools, while yielding equal sensitivity for short reads and even better sensitivity for long reads.
| S-EPMC5343294 | biostudies-literature
Project description:Oligodendrocyte progenitor cells (OPCs) are the principal source of new myelin in the central nervous system. A better understanding of how they mature into myelin-forming cells is of high relevance for remyelination. It has recently been demonstrated that during developmental myelination, the DNA methyltransferase 1 (DNMT1), but not DNMT3A, is critical for regulating proliferation and differentiation of OPCs into myelinating oligodendrocytes (OLs). However, it remains to be determined whether DNA methylation is also critical for the differentiation of adult OPCs during remyelination. After lysolecithin-induced demyelination in the ventrolateral spinal cord white matter of adult mice of either sex, we detected increased levels of DNA methylation and higher expression levels of the DNA methyltransferase DNMT3A and lower levels of DNMT1 in differentiating adult OLs. To functionally assess the role of DNMT1 and DNMT3 in adult OPCs, we used mice with inducible and lineage-specific ablation of Dnmt3a and/or Dnmt1 (i.e., Plp-creER(t);Dnmt3a-flox, Plp-creER(t);Dnmt1-flox, Plp-creER(t);Dnmt1-flox;Dnmt3a-flox). Upon lysolecithin injection in the spinal cord of these transgenic mice, we detected defective OPC differentiation and inefficient remyelination in the Dnmt3a null and Dnmt1/Dnmt3a null mice, but not in the Dnmt1 null mice. Taken together with previous results in the developing spinal cord, these data suggest an age-dependent role of distinct DNA methyltransferases in the oligodendrocyte lineage, with a dominant role for DNMT1 in neonatal OPCs and for DNMT3A in adult OPCs.
| S-EPMC5394940 | biostudies-literature
Project description:BackgroundIn recent years, the demand for computational power in computational biology has increased due to rapidly growing data sets from microarray and other high-throughput technologies. This demand is likely to increase. Standard algorithms for analyzing data, such as cluster algorithms, need to be parallelized for fast processing. Unfortunately, most approaches for parallelizing algorithms largely rely on network communication protocols connecting and requiring multiple computers. One answer to this problem is to utilize the intrinsic capabilities in current multi-core hardware to distribute the tasks among the different cores of one computer.ResultsWe introduce a multi-core parallelization of the k-means and k-modes cluster algorithms based on the design principles of transactional memory for clustering gene expression microarray type data and categorial SNP data. Our new shared memory parallel algorithms show to be highly efficient. We demonstrate their computational power and show their utility in cluster stability and sensitivity analysis employing repeated runs with slightly changed parameters. Computation speed of our Java based algorithm was increased by a factor of 10 for large data sets while preserving computational accuracy compared to single-core implementations and a recently published network based parallelization.ConclusionsMost desktop computers and even notebooks provide at least dual-core processors. Our multi-core algorithms show that using modern algorithmic concepts, parallelization makes it possible to perform even such laborious tasks as cluster sensitivity and cluster number estimation on the laboratory computer.
| S-EPMC2865495 | biostudies-literature