Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

RestfulSE: A semantically rich interface for cloud-scale genomics with Bioconductor.

ABSTRACT: Bioconductor's SummarizedExperiment class unites numerical assay quantifications with sample- and experiment-level metadata. SummarizedExperiment is the standard Bioconductor class for assays that produce matrix-like data, used by over 200 packages. We describe the restfulSE package, a deployment of this data model that supports remote storage. We illustrate use of SummarizedExperiment with remote HDF5 and Google BigQuery back ends, with two applications in cancer genomics. Our intent is to allow the use of familiar and semantically meaningful programmatic idioms to query genomic data, while abstracting the remote interface from end users and developers.

SUBMITTER: Gopaulakrishnan S

PROVIDER: S-EPMC6392152 | biostudies-other | 2019

REPOSITORIES: biostudies-other

ACCESS DATA

Json Xml

Publications

restfulSE: A semantically rich interface for cloud-scale genomics with Bioconductor.

Gopaulakrishnan Shweta S Pollack Samuela S Stubbs B J BJ Pagès Hervé H Readey John J Davis Sean S Waldron Levi L Morgan Martin M Carey Vincent V

F1000Research 20190107

Bioconductor's SummarizedExperiment class unites numerical assay quantifications with sample- and experiment-level metadata. SummarizedExperiment is the standard Bioconductor class for assays that produce matrix-like data, used by over 200 packages. We describe the restfulSE package, a deployment of this data model that supports remote storage. We illustrate use of SummarizedExperiment with remote HDF5 and Google BigQuery back ends, with two applications in cancer genomics. Our intent is to ...[more]

PMID: 30828438

Similar Datasets

Menu-driven cloud computing and resource sharing for R and Bioconductor.

Project description:SummaryWe report CRdata.org, a cloud-based, free, open-source web server for running analyses and sharing data and R scripts with others. In addition to using the free, public service, CRdata users can launch their own private Amazon Elastic Computing Cloud (EC2) nodes and store private data and scripts on Amazon's Simple Storage Service (S3) with user-controlled access rights. All CRdata services are provided via point-and-click menus.Availability and implementationCRdata is open-source and free under the permissive MIT License (opensource.org/licenses/mit-license.php). The source code is in Ruby (ruby-lang.org/en/) and available at: github.com/seerdata/crdata.Contacthbolouri@fhcrc.org.

| S-EPMC3150038 | biostudies-literature

Cloud computing for comparative genomics.

Project description:BackgroundLarge comparative genomics studies and tools are becoming increasingly more compute-expensive as the number of available genome sequences continues to rise. The capacity and cost of local computing infrastructures are likely to become prohibitive with the increase, especially as the breadth of questions continues to rise. Alternative computing architectures, in particular cloud computing environments, may help alleviate this increasing pressure and enable fast, large-scale, and cost-effective comparative genomics strategies going forward. To test this, we redesigned a typical comparative genomics algorithm, the reciprocal smallest distance algorithm (RSD), to run within Amazon's Elastic Computing Cloud (EC2). We then employed the RSD-cloud for ortholog calculations across a wide selection of fully sequenced genomes.ResultsWe ran more than 300,000 RSD-cloud processes within the EC2. These jobs were farmed simultaneously to 100 high capacity compute nodes using the Amazon Web Service Elastic Map Reduce and included a wide mix of large and small genomes. The total computation time took just under 70 hours and cost a total of $6,302 USD.ConclusionsThe effort to transform existing comparative genomics algorithms from local compute infrastructures is not trivial. However, the speed and flexibility of cloud computing environments provides a substantial boost with manageable cost. The procedure designed to transform the RSD algorithm into a cloud-ready application is readily adaptable to similar comparative genomics problems.

| S-EPMC3098063 | biostudies-literature

Studying Scale Dependency of Aerosol Cloud Interactions using Multi-Scale Cloud Formulations.

Project description:The WRF-ACI model configuration is used to investigate the scale dependency of aerosol-cloud interactions (ACI) across the "grey zone" scales for grid and subgrid-scale clouds. The impacts of ACI on weather are examined across regions in the eastern and western U. S. at 36, 12, 4, and 1 km grid spacing for short-term periods during the summer of 2006. ACI impacts are determined by comparing simulations with current climatological aerosol levels to simulations with aerosol levels reduced by 90%. The aerosol-cloud lifetime effect is found to be the dominant process leading to suppressed precipitation in regions of the eastern U.S., while regions in the western U. S. experience offsetting impacts on precipitation from the cloud lifetime effect and other effects that enhance precipitation. Generally, the cloud lifetime effect weakens with decreasing grid spacing due to a decrease in relative importance of autoconversion compared to accretion. Subgrid-scale ACI are dominant at 36 km, while grid-scale ACI are dominant at 4 and 1 km. At 12 km grid spacing, grid-scale and subgrid-scale ACI processes are comparable in magnitude and spatial coverage, but random perturbations in grid-scale-ACI impacts make the overall grid-scale-ACI impact appear muted. This competing behavior of grid and subgrid-scale clouds complicate the understanding of ACI at 12 km within the current WRF modeling framework. The work implies including subgrid-scale-cloud microphysics and ice/mixed phase cloud ACI processes may be necessary in weather and climate models to study ACI effectively.

| S-EPMC7886284 | biostudies-literature

The Cancer Genomics Cloud: Collaborative, Reproducible, and Democratized-A New Paradigm in Large-Scale Computational Research.

Project description:The Seven Bridges Cancer Genomics Cloud (CGC; www.cancergenomicscloud.org) enables researchers to rapidly access and collaborate on massive public cancer genomic datasets, including The Cancer Genome Atlas. It provides secure on-demand access to data, analysis tools, and computing resources. Researchers from diverse backgrounds can easily visualize, query, and explore cancer genomic datasets visually or programmatically. Data of interest can be immediately analyzed in the cloud using more than 200 preinstalled, curated bioinformatics tools and workflows. Researchers can also extend the functionality of the platform by adding their own data and tools via an intuitive software development kit. By colocalizing these resources in the cloud, the CGC enables scalable, reproducible analyses. Researchers worldwide can use the CGC to investigate key questions in cancer genomics. Cancer Res; 77(21); e3-6. ©2017 AACR.

| S-EPMC5832960 | biostudies-literature

TCGA Workflow: Analyze cancer genomics and epigenomics data using Bioconductor packages.

Project description:Biotechnological advances in sequencing have led to an explosion of publicly available data via large international consortia such as The Cancer Genome Atlas (TCGA), The Encyclopedia of DNA Elements (ENCODE), and The NIH Roadmap Epigenomics Mapping Consortium (Roadmap). These projects have provided unprecedented opportunities to interrogate the epigenome of cultured cancer cell lines as well as normal and tumor tissues with high genomic resolution. The Bioconductor project offers more than 1,000 open-source software and statistical packages to analyze high-throughput genomic data. However, most packages are designed for specific data types (e.g. expression, epigenetics, genomics) and there is no one comprehensive tool that provides a complete integrative analysis of the resources and data provided by all three public projects. A need to create an integration of these different analyses was recently proposed. In this workflow, we provide a series of biologically focused integrative analyses of different molecular data. We describe how to download, process and prepare TCGA data and by harnessing several key Bioconductor packages, we describe how to extract biologically meaningful genomic and epigenomic data. Using Roadmap and ENCODE data, we provide a work plan to identify biologically relevant functional epigenomic elements associated with cancer. To illustrate our workflow, we analyzed two types of brain tumors: low-grade glioma (LGG) versus high-grade glioma (glioblastoma multiform or GBM). This workflow introduces the following Bioconductor packages: AnnotationHub, ChIPSeeker, ComplexHeatmap, pathview, ELMER, GAIA, MINET, RTCGAToolbox, TCGAbiolinks.

| S-EPMC5302158 | biostudies-other

RamiGO: an R/Bioconductor package providing an AmiGO visualize interface.

Project description:SummaryThe R/Bioconductor package RamiGO is an R interface to AmiGO that enables visualization of Gene Ontology (GO) trees. Given a list of GO terms, RamiGO uses the AmiGO visualize API to import Graphviz-DOT format files into R, and export these either as images (SVG, PNG) or into Cytoscape for extended network analyses. RamiGO provides easy customization of annotation, highlighting of specific GO terms, colouring of terms by P-value or export of a simplified summary GO tree. We illustrate RamiGO functionalities in a genome-wide gene set analysis of prognostic genes in breast cancer.Availability and implementationRamiGO is provided in R/Bioconductor, is open source under the Artistic-2.0 License and is available with a user manual containing installation, operating instructions and tutorials. It requires R version 2.15.0 or higher. URL: http://bioconductor.org/packages/release/bioc/html/RamiGO.html

| S-EPMC3582261 | biostudies-literature

iFlow: A Graphical User Interface for Flow Cytometry Tools in Bioconductor.

Project description:Flow cytometry (FCM) has become an important analysis technology in health care and medical research, but the large volume of data produced by modern high-throughput experiments has presented significant new challenges for computational analysis tools. The development of an FCM software suite in Bioconductor represents one approach to overcome these challenges. In the spirit of the R programming language (Tree Star Inc., "FlowJo," http://www.owjo.com), these tools are predominantly console-driven, allowing for programmatic access and rapid development of novel algorithms. Using this software requires a solid understanding of programming concepts and of the R language. However, some of these tools|in particular the statistical graphics and novel analytical methods|are also useful for nonprogrammers. To this end, we have developed an open source, extensible graphical user interface (GUI) iFlow, which sits on top of the Bioconductor backbone, enabling basic analyses by means of convenient graphical menus and wizards. We envision iFlow to be easily extensible in order to quickly integrate novel methodological developments.

| S-EPMC2798115 | biostudies-literature

Large scale microbiome profiling in the cloud.

Project description:MotivationBacterial metagenomics profiling for metagenomic whole sequencing (mWGS) usually starts by aligning sequencing reads to a collection of reference genomes. Current profiling tools are designed to work against a small representative collection of genomes, and do not scale very well to larger reference genome collections. However, large reference genome collections are capable of providing a more complete and accurate profile of the bacterial population in a metagenomics dataset. In this paper, we discuss a scalable, efficient and affordable approach to this problem, bringing big data solutions within the reach of laboratories with modest resources.ResultsWe developed Flint, a metagenomics profiling pipeline that is built on top of the Apache Spark framework, and is designed for fast real-time profiling of metagenomic samples against a large collection of reference genomes. Flint takes advantage of Spark's built-in parallelism and streaming engine architecture to quickly map reads against a large (170 GB) reference collection of 43 552 bacterial genomes from Ensembl. Flint runs on Amazon's Elastic MapReduce service, and is able to profile 1 million Illumina paired-end reads against over 40 K genomes on 64 machines in 67 s-an order of magnitude faster than the state of the art, while using a much larger reference collection. Streaming the sequencing reads allows this approach to sustain mapping rates of 55 million reads per hour, at an hourly cluster cost of $8.00 USD, while avoiding the necessity of storing large quantities of intermediate alignments.Availability and implementationFlint is open source software, available under the MIT License (MIT). Source code is available at https://github.com/camilo-v/flint.Supplementary informationSupplementary data are available at Bioinformatics online.

| S-EPMC6612844 | biostudies-literature

Aether: leveraging linear programming for optimal cloud computing in genomics.

Project description:Motivation:Across biology, we are seeing rapid developments in scale of data production without a corresponding increase in data analysis capabilities. Results:Here, we present Aether (http://aether.kosticlab.org), an intuitive, easy-to-use, cost-effective and scalable framework that uses linear programming to optimally bid on and deploy combinations of underutilized cloud computing resources. Our approach simultaneously minimizes the cost of data analysis and provides an easy transition from users' existing HPC pipelines. Availability and implementation:Data utilized are available at https://pubs.broadinstitute.org/diabimmune and with EBI SRA accession ERP005989. Source code is available at (https://github.com/kosticlab/aether). Examples, documentation and a tutorial are available at http://aether.kosticlab.org. Contact:chirag_patel@hms.harvard.edu or aleksandar.kostic@joslin.harvard.edu. Supplementary information:Supplementary data are available at Bioinformatics online.

| S-EPMC5925767 | biostudies-literature

warpDOCK: Large-Scale Virtual Drug Discovery Using Cloud Infrastructure.

Project description:warpDOCK is an open-source pipeline for virtual small-molecule drug discovery using cloud infrastructure. warpDOCK is designed from the ground up for the Oracle Cloud Infrastructure (OCI), enabling harmonious parallelism of docking calculations over thousands to hundreds of thousands of cores. This enables cost-effective sampling of ultra-large chemical libraries, potentially reducing the time to identify lead drug candidates by orders of magnitude. By utilizing established docking software and automating each step of the process, warpDOCK makes large-scale virtual screening accessible to a broad user group. The warpDOCK code can be found at the BruningLab GitHub repository (https://github.com/BruningLab/warpDOCK).

| S-EPMC10433467 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data