Profiling SLAs for cloud system infrastructures and user interactions
Ontology highlight
ABSTRACT:
SUBMITTER: Cambronero M
PROVIDER: S-EPMC8157057 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:A fundamental problem in bioinformatics is genome assembly. Next-generation sequencing (NGS) technologies produce large volumes of fragmented genome reads, which require large amounts of memory to assemble the complete genome efficiently. With recent improvements in DNA sequencing technologies, it is expected that the memory footprint required for the assembly process will increase dramatically and will emerge as a limiting factor in processing widely available NGS-generated reads. In this report, we compare current memory-efficient techniques for genome assembly with respect to quality, memory consumption and execution time. Our experiments prove that it is possible to generate draft assemblies of reasonable quality on conventional multi-purpose computers with very limited available memory by choosing suitable assembly methods. Our study reveals the minimum memory requirements for different assembly programs even when data volume exceeds memory capacity by orders of magnitude. By combining existing methodologies, we propose two general assembly strategies that can improve short-read assembly approaches and result in reduction of the memory footprint. Finally, we discuss the possibility of utilizing cloud infrastructures for genome assembly and we comment on some findings regarding suitable computational resources for assembly.
| S-EPMC3785575 | biostudies-literature
Project description:As genomes become more and more available, gene function prediction presents itself as one of the major hurdles in our quest to extract meaningful information on the biological processes genes participate in. In order to facilitate gene function prediction, we show how our user-friendly pipeline, the Large-Scale Transcriptomic Analysis Pipeline in Cloud (LSTrAP-Cloud), can be useful in helping biologists make a shortlist of genes involved in a biological process that they might be interested in, by using a single gene of interest as bait. The LSTrAP-Cloud is based on Google Colaboratory, and provides user-friendly tools that process quality-control RNA sequencing data streamed from the European Nucleotide Archive. The LSTRAP-Cloud outputs a gene coexpression network that can be used to identify functionally related genes for any organism with a sequenced genome and publicly available RNA sequencing data. Here, we used the biosynthesis pathway of Nicotiana tabacum as a case study to demonstrate how enzymes, transporters, and transcription factors involved in the synthesis, transport, and regulation of nicotine can be identified using our pipeline.
| S-EPMC7230309 | biostudies-literature
Project description:While the popular workflow manager Galaxy is currently made available through several publicly accessible servers, there are scenarios where users can be better served by full administrative control over a private Galaxy instance, including, but not limited to, concerns about data privacy, customisation needs, prioritisation of particular job types, tools development, and training activities. In such cases, a cloud-based Galaxy virtual instance represents an alternative that equips the user with complete control over the Galaxy instance itself without the burden of the hardware and software infrastructure involved in running and maintaining a Galaxy server. We present Laniakea, a complete software solution to set up a "Galaxy on-demand" platform as a service. Building on the INDIGO-DataCloud software stack, Laniakea can be deployed over common cloud architectures usually supported both by public and private e-infrastructures. The user interacts with a Laniakea-based service through a simple front-end that allows a general setup of a Galaxy instance, and then Laniakea takes care of the automatic deployment of the virtual hardware and the software components. At the end of the process, the user gains access with full administrative privileges to a private, production-grade, fully customisable, Galaxy virtual instance and to the underlying virtual machine (VM). Laniakea features deployment of single-server or cluster-backed Galaxy instances, sharing of reference data across multiple instances, data volume encryption, and support for VM image-based, Docker-based, and Ansible recipe-based Galaxy deployments. A Laniakea-based Galaxy on-demand service, named Laniakea@ReCaS, is currently hosted at the ELIXIR-IT ReCaS cloud facility. Laniakea offers to scientific e-infrastructures a complete and easy-to-use software solution to provide a Galaxy on-demand service to their users. Laniakea-based cloud services will help in making Galaxy more accessible to a broader user base by removing most of the burdens involved in deploying and running a Galaxy service. In turn, this will facilitate the adoption of Galaxy in scenarios where classic public instances do not represent an optimal solution. Finally, the implementation of Laniakea can be easily adapted and expanded to support different services and platforms beyond Galaxy.
| S-EPMC7136032 | biostudies-literature
Project description:Data-independent acquisition (DIA) has become an important approach in global, mass spectrometric proteomic studies because it provides in-depth insights into the molecular variety of biological systems. However, DIA data analysis remains challenging owing to the high complexity and large data and sample size, which require specialized software and vast computing infrastructures. Most available open-source DIA software necessitates basic programming skills and covers only a fraction of a complete DIA data analysis. In consequence, DIA data analysis often requires usage of multiple software tools and compatibility thereof, severely limiting the usability and reproducibility. To overcome this hurdle, we have integrated a suite of open-source DIA tools in the Galaxy framework for reproducible and version-controlled data processing. The DIA suite includes OpenSwath, PyProphet, diapysef, and swath2stats. We have compiled functional Galaxy pipelines for DIA processing, which provide a web-based graphical user interface to these pre-installed and pre-configured tools for their use on freely accessible, powerful computational resources of the Galaxy framework. This approach also enables seamless sharing workflows with full configuration in addition to sharing raw data and results. We demonstrate the usability of an all-in-one DIA pipeline in Galaxy by the analysis of a spike-in case study dataset. Additionally, extensive training material is provided to further increase access for the proteomics community. The integration of an open-source DIA analysis suite in the web-based and user-friendly Galaxy framework in combination with extensive training material empowers a broad community of researches to perform reproducible and transparent DIA data analysis.
| S-EPMC8848309 | biostudies-literature
Project description:BackgroundRNA-Seq is a well-established technology extensively used for transcriptome profiling, allowing the analysis of coding and non-coding RNA molecules. However, this technology produces a vast amount of data requiring sophisticated computational approaches for their analysis than other traditional technologies such as Real-Time PCR or microarrays, strongly discouraging non-expert users. For this reason, dozens of pipelines have been deployed for the analysis of RNA-Seq data. Although interesting, these present several limitations and their usage require a technical background, which may be uncommon in small research laboratories. Therefore, the application of these technologies in such contexts is still limited and causes a clear bottleneck in knowledge advancement.ResultsMotivated by these considerations, we have developed RNAdetector, a new free cross-platform and user-friendly RNA-Seq data analysis software that can be used locally or in cloud environments through an easy-to-use Graphical User Interface allowing the analysis of coding and non-coding RNAs from RNA-Seq datasets of any sequenced biological species.ConclusionsRNAdetector is a new software that fills an essential gap between the needs of biomedical and research labs to process RNA-Seq data and their common lack of technical background in performing such analysis, which usually relies on outsourcing such steps to third party bioinformatics facilities or using expensive commercial software.
| S-EPMC8173825 | biostudies-literature
Project description:User reliability is notably crucial for personalized cloud services. In cloud computing environments, large amounts of cloud services are provided for users. With the exponential increase in number of cloud services, it is difficult for users to select the appropriate services from equivalent or similar candidate services. The quality-of-service (QoS) has become an important criterion for selection, and the users can conduct personalized selection according to the observed QoS data of other users; however, it is difficult to ensure that the users are reliable. Actually, unreliable users may provide unreliable QoS data and have negative effects on the personalized cloud service selection. Therefore, how to determine reliable QoS data for personalized cloud service selection remains a significant problem. To measure the reliability for each user, we present a cloud service selection framework based on user reputation and propose a new user reputation calculation approach, which is named MeURep and includes L1-MeURep and L2-MeURep. Experiments are conducted, and the results confirm that MeURep has higher efficiency than previously proposed approaches.
| S-EPMC6588211 | biostudies-literature
Project description:The WRF-ACI model configuration is used to investigate the scale dependency of aerosol-cloud interactions (ACI) across the "grey zone" scales for grid and subgrid-scale clouds. The impacts of ACI on weather are examined across regions in the eastern and western U. S. at 36, 12, 4, and 1 km grid spacing for short-term periods during the summer of 2006. ACI impacts are determined by comparing simulations with current climatological aerosol levels to simulations with aerosol levels reduced by 90%. The aerosol-cloud lifetime effect is found to be the dominant process leading to suppressed precipitation in regions of the eastern U.S., while regions in the western U. S. experience offsetting impacts on precipitation from the cloud lifetime effect and other effects that enhance precipitation. Generally, the cloud lifetime effect weakens with decreasing grid spacing due to a decrease in relative importance of autoconversion compared to accretion. Subgrid-scale ACI are dominant at 36 km, while grid-scale ACI are dominant at 4 and 1 km. At 12 km grid spacing, grid-scale and subgrid-scale ACI processes are comparable in magnitude and spatial coverage, but random perturbations in grid-scale-ACI impacts make the overall grid-scale-ACI impact appear muted. This competing behavior of grid and subgrid-scale clouds complicate the understanding of ACI at 12 km within the current WRF modeling framework. The work implies including subgrid-scale-cloud microphysics and ice/mixed phase cloud ACI processes may be necessary in weather and climate models to study ACI effectively.
| S-EPMC7886284 | biostudies-literature
Project description:The effect of an increase in atmospheric aerosol concentrations on the distribution and radiative properties of Earth's clouds is the most uncertain component of the overall global radiative forcing from preindustrial time. General circulation models (GCMs) are the tool for predicting future climate, but the treatment of aerosols, clouds, and aerosol-cloud radiative effects carries large uncertainties that directly affect GCM predictions, such as climate sensitivity. Predictions are hampered by the large range of scales of interaction between various components that need to be captured. Observation systems (remote sensing, in situ) are increasingly being used to constrain predictions, but significant challenges exist, to some extent because of the large range of scales and the fact that the various measuring systems tend to address different scales. Fine-scale models represent clouds, aerosols, and aerosol-cloud interactions with high fidelity but do not include interactions with the larger scale and are therefore limited from a climatic point of view. We suggest strategies for improving estimates of aerosol-cloud relationships in climate models, for new remote sensing and in situ measurements, and for quantifying and reducing model uncertainty.
| S-EPMC4889348 | biostudies-other
Project description:MotivationBacterial metagenomics profiling for metagenomic whole sequencing (mWGS) usually starts by aligning sequencing reads to a collection of reference genomes. Current profiling tools are designed to work against a small representative collection of genomes, and do not scale very well to larger reference genome collections. However, large reference genome collections are capable of providing a more complete and accurate profile of the bacterial population in a metagenomics dataset. In this paper, we discuss a scalable, efficient and affordable approach to this problem, bringing big data solutions within the reach of laboratories with modest resources.ResultsWe developed Flint, a metagenomics profiling pipeline that is built on top of the Apache Spark framework, and is designed for fast real-time profiling of metagenomic samples against a large collection of reference genomes. Flint takes advantage of Spark's built-in parallelism and streaming engine architecture to quickly map reads against a large (170 GB) reference collection of 43 552 bacterial genomes from Ensembl. Flint runs on Amazon's Elastic MapReduce service, and is able to profile 1 million Illumina paired-end reads against over 40 K genomes on 64 machines in 67 s-an order of magnitude faster than the state of the art, while using a much larger reference collection. Streaming the sequencing reads allows this approach to sustain mapping rates of 55 million reads per hour, at an hourly cluster cost of $8.00 USD, while avoiding the necessity of storing large quantities of intermediate alignments.Availability and implementationFlint is open source software, available under the MIT License (MIT). Source code is available at https://github.com/camilo-v/flint.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC6612844 | biostudies-literature
Project description:Coincident multi-year measurements of aerosol, cloud, precipitation and radiation at near-global scales are analyzed to diagnose their apparent relationships as suggestive of interactions previously proposed based on theoretical, observational, and model constructs. Specifically, we examine whether differences in aerosol loading in separate observations go along with consistently different precipitation, cloud properties, and cloud radiative effects. Our analysis uses a cloud regime (CR) framework to dissect and sort the results. The CRs come from the Moderate Resolution Imaging Spectroradiometer (MODIS) sensor and are defined as distinct groups of cloud systems with similar co-variations of cloud top pressure and cloud optical thickness. Aerosol optical depth used as proxy for aerosol loading comes from two sources, MODIS observations, and the MERRA-2 re-analysis, and its variability is defined with respect to local seasonal climatologies. The choice of aerosol dataset impacts our results substantially. We also find that the responses of the marine and continental component of a CR are frequently quite disparate. Overall, CRs dominated by warm clouds tend to exhibit less ambiguous signals, but also have more uncertainty with regard to precipitation changes. Finally, we find weak, but occasionally systematic co-variations of select meteorological indicators and aerosol, which serves as a sober reminder that ascribing changes in cloud and cloud-affected variables solely to aerosol variations is precarious.
| S-EPMC5891332 | biostudies-literature