Browse
Submit Data
Databases
API
Help

Dataset Information

11 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

SparkBLAST: scalable BLAST processing using in-memory operations.

ABSTRACT: The demand for processing ever increasing amounts of genomic data has raised new challenges for the implementation of highly scalable and efficient computational systems. In this paper we propose SparkBLAST, a parallelization of a sequence alignment application (BLAST) that employs cloud computing for the provisioning of computational resources and Apache Spark as the coordination framework. As a proof of concept, some radionuclide-resistant bacterial genomes were selected for similarity analysis.Experiments in Google and Microsoft Azure clouds demonstrated that SparkBLAST outperforms an equivalent system implemented on Hadoop in terms of speedup and execution times.The superior performance of SparkBLAST is mainly due to the in-memory operations available through the Spark framework, consequently reducing the number of local I/O operations required for distributed BLAST processing.

SUBMITTER: de Castro MR

PROVIDER: S-EPMC5488373 | biostudies-literature | 2017 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

SparkBLAST: scalable BLAST processing using in-memory operations.

de Castro Marcelo Rodrigo MR Tostes Catherine Dos Santos CDS Dávila Alberto M R AMR Senger Hermes H da Silva Fabricio A B FAB

BMC bioinformatics 20170627 1

<h4>Background</h4>The demand for processing ever increasing amounts of genomic data has raised new challenges for the implementation of highly scalable and efficient computational systems. In this paper we propose SparkBLAST, a parallelization of a sequence alignment application (BLAST) that employs cloud computing for the provisioning of computational resources and Apache Spark as the coordination framework. As a proof of concept, some radionuclide-resistant bacterial genomes were selected for ...[more]

PMID: 28655296

Similar Datasets

Scalable in-memory processing of omics workflows.

Project description:We present a proof of concept implementation of the in-memory computing paradigm that we use to facilitate the analysis of metagenomic sequencing reads. In doing so we compare the performance of POSIX™file systems and key-value storage for omics data, and we show the potential for integrating high-performance computing (HPC) and cloud native technologies. We show that in-memory key-value storage offers possibilities for improved handling of omics data through more flexible and faster data processing. We envision fully containerized workflows and their deployment in portable micro-pipelines with multiple instances working concurrently with the same distributed in-memory storage. To highlight the potential usage of this technology for event driven and real-time data processing, we use a biological case study focused on the growing threat of antimicrobial resistance (AMR). We develop a workflow encompassing bioinformatics and explainable machine learning (ML) to predict life expectancy of a population based on the microbiome of its sewage while providing a description of AMR contribution to the prediction. We propose that in future, performing such analyses in 'real-time' would allow us to assess the potential risk to the population based on changes in the AMR profile of the community.

| S-EPMC9052061 | biostudies-literature

Mechanical memory operations in piezotransistive GaN microcantilevers using Au nanoparticle-enhanced photoacoustic excitation.

Project description:Nonlinear oscillations in micro- and nanoelectromechanical systems have emerged as an exciting research area in recent years due to their promise in realizing low-power, scalable, and reconfigurable mechanical memory and logic devices. Here, we report ultralow-power mechanical memory operations utilizing the nonlinear oscillation regime of GaN microcantilevers with embedded piezotransistive AlGaN/GaN heterostructure field effect transistors as highly sensitive deflection transducers. Switching between the high and low oscillatory states of the nonlinear oscillation regime was demonstrated using a novel phase-controlled opto-mechanical excitation setup, utilizing a piezo actuator and a pulsed laser as the primary and secondary excitation sources, respectively. Laser-based photoacoustic excitation was amplified through plasmonic absorption in Au nanoparticles deposited on a transistor. Thus, the minimum switching energy required for reliable memory operations was reduced to less than a picojoule (pJ), which translates to one of the lowest ever reported, when normalized for mass.

| S-EPMC8784537 | biostudies-literature

CloudProteoAnalyzer: scalable processing of big data from proteomics using cloud computing.

Project description:SummaryShotgun proteomics is widely used in many system biology studies to determine the global protein expression profiles of tissues, cultures, and microbiomes. Many non-distributed computer algorithms have been developed for users to process proteomics data on their local computers. However, the amount of data acquired in a typical proteomics study has grown rapidly in recent years, owing to the increasing throughput of mass spectrometry and the expanding scale of study designs. This presents a big data challenge for researchers to process proteomics data in a timely manner. To overcome this challenge, we developed a cloud-based parallel computing application to offer end-to-end proteomics data analysis software as a service (SaaS). A web interface was provided to users to upload mass spectrometry-based proteomics data, configure parameters, submit jobs, and monitor job status. The data processing was distributed across multiple nodes in a supercomputer to achieve scalability for large datasets. Our study demonstrated SaaS for proteomics as a viable solution for the community to scale up the data processing using cloud computing.Availability and implementationThis application is available online at https://sipros.oscer.ou.edu/ or https://sipros.unt.edu for free use. The source code is available at https://github.com/Biocomputing-Research-Group/CloudProteoAnalyzer under the GPL version 3.0 license.

| S-EPMC10942798 | biostudies-literature

What's behind a P600? Integration operations during irony processing.

Project description:The combined knowledge of word meanings and grammatical rules does not allow a listener to grasp the intended meaning of a speaker's utterance. Pragmatic inferences on the part of the listener are also required. The present work focuses on the processing of ironic utterances (imagine a slow day being described as "really productive") because these clearly require the listener to go beyond the linguistic code. Such utterances are advantageous experimentally because they can serve as their own controls in the form of literal sentences (now imagine an active day being described as "really productive") as we employ techniques from electrophysiology (EEG). Importantly, the results confirm previous ERP findings showing that irony processing elicits an enhancement of the P600 component (Regel et al., 2011). More original are the findings drawn from Time Frequency Analysis (TFA) and especially the increase of power in the gamma band in the 280-400 time-window, which points to an integration among different streams of information relatively early in the comprehension of an irony. This represents a departure from traditional accounts of language processing which generally view pragmatic inferences as late-arriving. We propose that these results indicate that unification operations between the linguistic code and contextual information play a critical role throughout the course of irony processing and earlier than previously thought.

| S-EPMC3691266 | biostudies-literature

In-Memory Logic Operations and Neuromorphic Computing in Non-Volatile Random Access Memory.

Project description:Recent progress in the development of artificial intelligence technologies, aided by deep learning algorithms, has led to an unprecedented revolution in neuromorphic circuits, bringing us ever closer to brain-like computers. However, the vast majority of advanced algorithms still have to run on conventional computers. Thus, their capacities are limited by what is known as the von-Neumann bottleneck, where the central processing unit for data computation and the main memory for data storage are separated. Emerging forms of non-volatile random access memory, such as ferroelectric random access memory, phase-change random access memory, magnetic random access memory, and resistive random access memory, are widely considered to offer the best prospect of circumventing the von-Neumann bottleneck. This is due to their ability to merge storage and computational operations, such as Boolean logic. This paper reviews the most common kinds of non-volatile random access memory and their physical principles, together with their relative pros and cons when compared with conventional CMOS-based circuits (Complementary Metal Oxide Semiconductor). Their potential application to Boolean logic computation is then considered in terms of their working mechanism, circuit design and performance metrics. The paper concludes by envisaging the prospects offered by non-volatile devices for future brain-inspired and neuromorphic computation.

| S-EPMC7475900 | biostudies-literature

Evidence for working memory storage operations in perceptual cortex.

Project description:Isolating the short-term storage component of working memory (WM) from the myriad of associated executive processes has been an enduring challenge. Recent efforts have identified patterns of activity in visual regions that contain information about items being held in WM. However, it remains unclear (1) whether these representations withstand intervening sensory input and (2) how communication between multimodal association cortex and the unimodal perceptual regions supporting WM representations is involved in WM storage. We present evidence that the features of a face held in WM are stored within face-processing regions, that these representations persist across subsequent sensory input, and that information about the match between sensory input and a memory representation is relayed forward from perceptual to prefrontal regions. Participants were presented with a series of probe faces and indicated whether each probe matched a target face held in WM. We parametrically varied the feature similarity between the probe and target faces. Activity within face-processing regions scaled linearly with the degree of feature similarity between the probe face and the features of the target face, suggesting that the features of the target face were stored in these regions. Furthermore, directed connectivity measures revealed that the direction of information flow that was optimal for performance was from sensory regions that stored the features of the target face to dorsal prefrontal regions, supporting the notion that sensory input is compared to representations stored within perceptual regions and is subsequently relayed forward. Together, these findings indicate that WM storage operations are carried out within perceptual cortex.

| S-EPMC4362687 | biostudies-literature

In-Memory Mathematical Operations with Spin-Orbit Torque Devices.

Project description:Analog arithmetic operations are the most fundamental mathematical operations used in image and signal processing as well as artificial intelligence (AI). In-memory computing (IMC) offers a high performance and energy-efficient computing paradigm. To date, in-memory analog arithmetic operations with emerging nonvolatile devices are usually implemented using discrete components, which limits the scalability and blocks large scale integration. Here, a prototypical implementation of in-memory analog arithmetic operations (summation, subtraction and multiplication) is experimentally demonstrated, based on in-memory electrical current sensing units using spin-orbit torque (SOT) devices. The proposed structures for analog arithmetic operations are smaller than the state-of-the-art complementary metal oxide semiconductor (CMOS) counterparts by several orders of magnitude. Moreover, data to be processed and computing results can be locally stored, or the analog computing can be done in the nonvolatile SOT devices, which are exploited to experimentally implement the image edge detection and signal amplitude modulation with a simple structure. Furthermore, an artificial neural network (ANN) with SOT devices based synapses is constructed to realize pattern recognition with high accuracy of ≈95%.

| S-EPMC9443454 | biostudies-literature

simpleaf: a simple, flexible, and scalable framework for single-cell data processing using alevin-fry.

Project description:SummaryThe alevin-fry ecosystem provides a robust and growing suite of programs for single-cell data processing. However, as new single-cell technologies are introduced, as the community continues to adjust best practices for data processing, and as the alevin-fry ecosystem itself expands and grows, it is becoming increasingly important to manage the complexity of alevin-fry's single-cell preprocessing workflows while retaining the performance and flexibility that make these tools enticing. We introduce simpleaf, a program that simplifies the processing of single-cell data using tools from the alevin-fry ecosystem, and adds new functionality and capabilities, while retaining the flexibility and performance of the underlying tools.Availability and implementationSimpleaf is written in Rust and released under a BSD 3-Clause license. It is freely available from its GitHub repository https://github.com/COMBINE-lab/simpleaf, and via bioconda. Documentation for simpleaf is available at https://simpleaf.readthedocs.io/en/latest/ and tutorials for simpleaf that have been developed can be accessed at https://combine-lab.github.io/alevin-fry-tutorials.

| S-EPMC10580267 | biostudies-literature

Distribution of agitation and related symptoms among hospitalized patients using a scalable natural language processing method.

Project description:BackgroundAgitation is a common feature of many neuropsychiatric disorders.ObjectiveUnderstanding the prevalence, implications, and characteristics of agitation among hospitalized populations can facilitate more precise recognition of disability arising from neuropsychiatric diseases.MethodsWe developed two agitation phenotypes using an expansion of expert curated term lists. These phenotypes were used to characterize five years of psychiatric admissions. The relationship of agitation symptoms and length of stay was examined.ResultsAmong 4548 psychiatric admissions, 1134 (24.9%) included documentation of agitation based on the primary agitation phenotype. These symptoms were greater among individuals with public insurance, and those with mania and psychosis compared to major depressive disorder. Greater symptoms were associated with longer hospital stay, with ~0.9 day increase in stay for every 10% increase in agitation phenotype.ConclusionAgitation was common at hospital admission and associated with diagnosis and longer length of stay. Characterizing agitation-related symptoms through natural language processing may provide new tools for understanding agitated behaviors and their relationship to delirium.

| S-EPMC7855889 | biostudies-literature

PyIR: a scalable wrapper for processing billions of immunoglobulin and T cell receptor sequences using IgBLAST.

Project description:BackgroundRecent advances in DNA sequencing technologies have enabled significant leaps in capacity to generate large volumes of DNA sequence data, which has spurred a rapid growth in the use of bioinformatics as a means of interrogating antibody variable gene repertoires. Common tools used for annotation of antibody sequences are often limited in functionality, modularity and usability.ResultsWe have developed PyIR, a Python wrapper and library for IgBLAST, which offers a minimal setup CLI and API, FASTQ support, file chunking for large sequence files, JSON and Python dictionary output, and built-in sequence filtering.ConclusionsPyIR offers improved processing speed over multithreaded IgBLAST (version 1.14) when spawning more than 16 processes on a single computer system. Its customizable filtering and data encapsulation allow it to be adapted to a wide range of computing environments. The API allows for IgBLAST to be used in customized bioinformatics workflows.

| S-EPMC7364545 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data