Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

PyBDA: a command line tool for automated analysis of big biological data sets.

ABSTRACT: BACKGROUND:Analysing large and high-dimensional biological data sets poses significant computational difficulties for bioinformaticians due to lack of accessible tools that scale to hundreds of millions of data points. RESULTS:We developed a novel machine learning command line tool called PyBDA for automated, distributed analysis of big biological data sets. By using Apache Spark in the backend, PyBDA scales to data sets beyond the size of current applications. It uses Snakemake in order to automatically schedule jobs to a high-performance computing cluster. We demonstrate the utility of the software by analyzing image-based RNA interference data of 150 million single cells. CONCLUSION:PyBDA allows automated, easy-to-use data analysis using common statistical methods and machine learning algorithms. It can be used with simple command line calls entirely making it accessible to a broad user base. PyBDA is available at https://pybda.rtfd.io.

SUBMITTER: Dirmeier S

PROVIDER: S-EPMC6849186 | biostudies-literature | 2019 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

PyBDA: a command line tool for automated analysis of big biological data sets.

Dirmeier Simon S Emmenlauer Mario M Dehio Christoph C Beerenwinkel Niko N

BMC bioinformatics 20191112 1

<h4>Background</h4>Analysing large and high-dimensional biological data sets poses significant computational difficulties for bioinformaticians due to lack of accessible tools that scale to hundreds of millions of data points.<h4>Results</h4>We developed a novel machine learning command line tool called PyBDA for automated, distributed analysis of big biological data sets. By using Apache Spark in the backend, PyBDA scales to data sets beyond the size of current applications. It uses Snakemake i ...[more]

PMID: 31718539

Similar Datasets

A new tool called DISSECT for analysing large genomic data sets using a Big Data approach.

Project description:Large-scale genetic and genomic data are increasingly available and the major bottleneck in their analysis is a lack of sufficiently scalable computational tools. To address this problem in the context of complex traits analysis, we present DISSECT. DISSECT is a new and freely available software that is able to exploit the distributed-memory parallel computational architectures of compute clusters, to perform a wide range of genomic and epidemiologic analyses, which currently can only be carried out on reduced sample sizes or under restricted conditions. We demonstrate the usefulness of our new tool by addressing the challenge of predicting phenotypes from genotype data in human populations using mixed-linear model analysis. We analyse simulated traits from 470,000 individuals genotyped for 590,004 SNPs in ?4?h using the combined computational power of 8,400 processor cores. We find that prediction accuracies in excess of 80% of the theoretical maximum could be achieved with large sample sizes.

| S-EPMC4682108 | biostudies-literature

ViralRecall-A Flexible Command-Line Tool for the Detection of Giant Virus Signatures in 'Omic Data.

Project description:Giant viruses are widespread in the biosphere and play important roles in biogeochemical cycling and host genome evolution. Also known as nucleo-cytoplasmic large DNA viruses (NCLDVs), these eukaryotic viruses harbor the largest and most complex viral genomes known. Studies have shown that NCLDVs are frequently abundant in metagenomic datasets, and that sequences derived from these viruses can also be found endogenized in diverse eukaryotic genomes. The accurate detection of sequences derived from NCLDVs is therefore of great importance, but this task is challenging owing to both the high level of sequence divergence between NCLDV families and the extraordinarily high diversity of genes encoded in their genomes, including some encoding for metabolic or translation-related functions that are typically found only in cellular lineages. Here, we present ViralRecall, a bioinformatic tool for the identification of NCLDV signatures in 'omic data. This tool leverages a library of giant virus orthologous groups (GVOGs) to identify sequences that bear signatures of NCLDVs. We demonstrate that this tool can effectively identify NCLDV sequences with high sensitivity and specificity. Moreover, we show that it can be useful both for removing contaminating sequences in metagenome-assembled viral genomes as well as the identification of eukaryotic genomic loci that derived from NCLDV. ViralRecall is written in Python 3.5 and is freely available on GitHub: https://github.com/faylward/viralrecall.

| S-EPMC7909515 | biostudies-literature

Citation.js: a format-independent, modular bibliography tool for the browser and command line.

Project description:BackgroundGiven the vast number of standards and formats for bibliographical data, any program working with bibliographies and citations has to be able to interpret such data. This paper describes the development of Citation.js (https://citation.js.org/), a tool to parse and format according to those standards. The program follows modern guidelines for software in general and JavaScript in specific, such as version control, source code analysis, integration testing and semantic versioning.ResultsThe result is an extensible tool that has already seen adaption in a variety of sources and use cases: as part of a server-side page generator of a publishing platform, as part of a local extensible document generator, and as part of an in-browser converter of extracted references. Use cases range from transforming a list of DOIs or Wikidata identifiers into a BibTeX file on the command line, to displaying RIS references on a webpage with added Altmetric badges to generating "How to cite this" sections on a blog. The accuracy of conversions is currently 27% for properties and 60% for types on average and a typical initialization takes 120 ms in browsers and 1 s with Node.js on the command line.ConclusionsCitation.js is a library supporting various formats of bibliographic information in a broad selection of use cases and environments. Given the support for plugins, more formats can be added with relative ease.

| S-EPMC7924481 | biostudies-literature

IonFlow: a galaxy tool for the analysis of ionomics data sets.

Project description:IntroductionInductively coupled plasma mass spectrometry (ICP-MS) experiments generate complex multi-dimensional data sets that require specialist data analysis tools.ObjectiveHere we describe tools to facilitate analysis of the ionome composed of high-throughput elemental profiling data.MethodsIonFlow is a Galaxy tool written in R for ionomics data analysis and is freely accessible at https://github.com/wanchanglin/ionflow . It is designed as a pipeline that can process raw data to enable exploration and interpretation using multivariate statistical techniques and network-based algorithms, including principal components analysis, hierarchical clustering, relevance network extraction and analysis, and gene set enrichment analysis.Results and conclusionThe pipeline is described and tested on two benchmark data sets of the haploid S. Cerevisiae ionome and of the human HeLa cell ionome.

| S-EPMC8464566 | biostudies-literature

gQuant, an Automated Tool for Quantitative Glycomic Data Analysis.

Project description:MALDI-MS-based glycan isotope labeling methods have been effectively and widely used for quantitative glycomics. However, interpretation of the data produced by MALDI-MS is inaccurate and tedious because the bioinformatic tools are inadequate. In this work, we present gQuant, an automated tool for MALDI-MS-based glycan isotope labeling data processing. gQuant was designed with a set of dedicated algorithms to improve the efficiency, accuracy and convenience of quantitation data processing. When tested on the reference data set, gQuant showed a fast processing speed, as it was able to search the glycan data of model glycoproteins in a few minutes and reported more results than the manual analysis did. The reported quantitation ratios matched well with the experimental glycan mixture ratios ranging from 1:10 to 10:1. In addition, gQuant is fully open-source and is coded in Python, which is supported by most operating systems, and it has a user-friendly interface. gQuant can be easily adapted by users for specific experimental designs, such as specific glycan databases, different derivatization types and relative quantitation designs and can thus facilitate fast glycomic quantitation for clinical sample analysis using MALDI-MS-based stable isotope labeling.

| S-EPMC8355585 | biostudies-literature

Reactome pathway analysis to enrich biological discovery in proteomics data sets.

Project description:Reactome (http://www.reactome.org) is an open-source, expert-authored, peer-reviewed, manually curated database of reactions, pathways and biological processes. We provide an intuitive web-based user interface to pathway knowledge and a suite of data analysis tools. The Pathway Browser is a Systems Biology Graphical Notation-like visualization system that supports manual navigation of pathways by zooming, scrolling and event highlighting, and that exploits PSI Common Query Interface web services to overlay pathways with molecular interaction data from the Reactome Functional Interaction Network and interaction databases such as IntAct, ChEMBL and BioGRID. Pathway and expression analysis tools employ web services to provide ID mapping, pathway assignment and over-representation analysis of user-supplied data sets. By applying Ensembl Compara to curated human proteins and reactions, Reactome generates pathway inferences for 20 other species. The Species Comparison tool provides a summary of results for each of these species as a table showing numbers of orthologous proteins found by pathway from which users can navigate to inferred details for specific proteins and reactions. Reactome's diverse pathway knowledge and suite of data analysis tools provide a platform for data mining, modeling and analysis of large-scale proteomics data sets. This Tutorial is part of the International Proteomics Tutorial Programme (IPTP 8).

| S-EPMC4617659 | biostudies-literature

Tbl2KnownGene: A command-line program to convert NCBI.tbl to UCSC knownGene.txt data file.

Project description:UnlabelledThe schema for UCSC Known Genes (knownGene.txt) has been widely adopted for use in both standard and custom downstream analysis tools/scripts. For many popular model organisms (e.g. Arabidopsis), sequence and annotation data tables (including "knownGene.txt") have not yet been made available to the public. Therefore, it is of interest to describe Tbl2KnownGene, a .tbl file parser that can process the contents of a NCBI .tbl file and produce a UCSC Known Genes annotation feature table. The algorithm is tested with chromosome datasets from Arabidopsis genome (TAIR10). The Tbl2KnownGene parser finds utility for data with other organisms having similar .tbl annotations.AvailabilityPerl scripts and required input files are available on the web at http://thoth.indstate.edu/~ybai2/Tbl2KnownGene/ index.html.

| S-EPMC4166776 | biostudies-literature

From command-line bioinformatics to bioGUI.

Project description:Bioinformatics is a highly interdisciplinary field providing (bioinformatics) applications for scientists from many disciplines. Installing and starting applications on the command-line (CL) is inconvenient and/or inefficient for many scientists. Nonetheless, most methods are implemented with a command-line interface only. Providing a graphical user interface (GUI) for bioinformatics applications is one step toward routinely making CL-only applications available to more scientists and, thus, toward a more effective interdisciplinary work. With our bioGUI framework we address two main problems of using CL bioinformatics applications: First, many tools work on UNIX-systems only, while many scientists use Microsoft Windows. Second, scientists refrain from using CL tools which, however, could well support them in their research. With bioGUI install modules and templates, installing and using CL tools is made possible for most scientists-even on Windows, due to bioGUI's support for Windows Subsystem for Linux. In addition, bioGUI templates can easily be created, making the bioGUI framework highly rewarding for developers. From the bioGUI repository it is possible to download, install and use bioinformatics tools with just a few clicks.

| S-EPMC6875409 | biostudies-literature

RadAA: A Command-line Tool for Identification of Radical Amino Acid Changes in Multiple Sequence Alignments.

Project description:High-throughput sequencing has revolutionised biology and medicine. Numerous genomes and transcriptome assemblies are now available, and these genomic data sets lend themselves to comparisons between species, strains, and other strata. Researchers often need to rapidly identify changes, in particular amino acid substitutions that could confer biological function in their system of interest. However, we are not aware of an easy-to-use tool that can be used to detect such changes, and researchers currently rely on idiosyncratic computer code. We present RadAA, a command-line tool which screens multiple sequence alignments for radical amino acid changes in a stratum/strata by classifying residues into groups by charge (with cysteine in its own group). RadAA is easy to use, even for researchers with little experience in computational biology. It can be run on most operating systems - including MacOS, Windows, and Linux - and integrated into high-performance computing environments. The RadAA source code and executable binaries are freely available at https://github.com/sciseim/RadAA.

| S-EPMC6585820 | biostudies-literature

Easy phylotyping of <i>Escherichia coli via</i> the EzClermont web app and command-line tool.

Project description:The Clermont PCR method for phylotyping Escherichia coli remains a useful classification scheme even though genome sequencing is now routine, and higher-resolution sequence typing schemes are now available. Relating present-day whole-genome E. coli classifications to legacy phylotyping is essential for harmonizing the historical literature and understanding of this important organism. Therefore, we present EzClermont - a novel in silico Clermont PCR phylotyping tool to enable ready application of this phylotyping scheme to whole-genome assemblies. We evaluate this tool against phylogenomic classifications, and an alternative software implementation of Clermont typing. EzClermont is available as a web app at www.ezclermont.org, and as a command-line tool at https://nickp60.github.io/EzClermont/.

| S-EPMC7656184 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data