Browse
Submit Data
Databases
API
Help

Dataset Information

14 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Unifying package managers, workflow engines, and containers: Computational reproducibility with BioNix.

ABSTRACT:

Motivation

A challenge for computational biologists is to make our analyses reproducible-i.e. to rerun, combine, and share, with the assurance that equivalent runs will generate identical results. Current best practice aims at this using a combination of package managers, workflow engines, and containers.

Results

We present BioNix, a lightweight library built on the Nix deployment system. BioNix manages software dependencies, computational environments, and workflow stages together using a single abstraction: pure functions. This lets users specify workflows in a clean, uniform way, with strong reproducibility guarantees.

Availability and implementation

BioNix is implemented in the Nix expression language and is released on GitHub under the 3-clause BSD license: https://github.com/PapenfussLab/bionix (biotools:BioNix) (BioNix, RRID:SCR_017662).

SUBMITTER: Bedo J

PROVIDER: S-EPMC7672450 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Unifying package managers, workflow engines, and containers: Computational reproducibility with BioNix.

Bedő Justin J Di Stefano Leon L Papenfuss Anthony T AT

GigaScience 20201101 11

<h4>Motivation</h4>A challenge for computational biologists is to make our analyses reproducible-i.e. to rerun, combine, and share, with the assurance that equivalent runs will generate identical results. Current best practice aims at this using a combination of package managers, workflow engines, and containers.<h4>Results</h4>We present BioNix, a lightweight library built on the Nix deployment system. BioNix manages software dependencies, computational environments, and workflow stages togethe ...[more]

PMID: 33205815

Similar Datasets

Competencies of military nurse managers: A scoping review and unifying framework.

Project description:AIM(S):To identify competencies of military nurse managers and develop a unifying framework of military nurse managers' competencies. BACKGROUND:Military nurse managers shoulder multiple responsibilities because of duality roles, and they should possess competencies that enable them to manage human and material resources during peacetime and wartime. Therefore, nursing management within military context is demanding, such that a comprehensive understanding of their competencies is needed for effective military nursing management. Although relevant studies have focused on different military branches and different levels of managers, there is no standard evaluation framework. EVALUATION:A scoping review of studies focusing on competencies of military nurse managers from seven databases was carried out. KEY ISSUES:Nine studies were included in this review, and a framework consisting of six domains of military nurse managers' competencies was identified: clinical expertise, role model, leadership competencies, human competencies, financial competencies and deployment competencies. CONCLUSION:Existing knowledge of competencies of military nurse managers is limited, and a comprehensive understanding of this topic can provide direction for future work. IMPLICATIONS FOR NURSING MANAGEMENT:Military nurse managers play substantial roles within the military nursing context. A unifying framework can facilitate personnel recruitment and competency measurement, as well as training protocol development.

| S-EPMC7540449 | biostudies-literature

Enhancing reproducibility in scientific computing: Metrics and registry for Singularity containers.

Project description:Here we present Singularity Hub, a framework to build and deploy Singularity containers for mobility of compute, and the singularity-python software with novel metrics for assessing reproducibility of such containers. Singularity containers make it possible for scientists and developers to package reproducible software, and Singularity Hub adds automation to this workflow by building, capturing metadata for, visualizing, and serving containers programmatically. Our novel metrics, based on custom filters of content hashes of container contents, allow for comparison of an entire container, including operating system, custom software, and metadata. First we will review Singularity Hub's primary use cases and how the infrastructure has been designed to support modern, common workflows. Next, we conduct three analyses to demonstrate build consistency, reproducibility metric and performance and interpretability, and potential for discovery. This is the first effort to demonstrate a rigorous assessment of measurable similarity between containers and operating systems. We provide these capabilities within Singularity Hub, as well as the source software singularity-python that provides the underlying functionality. Singularity Hub is available at https://singularity-hub.org, and we are excited to provide it as an openly available platform for building, and deploying scientific containers.

| S-EPMC5706697 | biostudies-literature

Increasing workflow development speed and reproducibility with Vectools.

Project description:Despite advances in bioinformatics, custom scripts remain a source of difficulty, slowing workflow development and hampering reproducibility. Here, we introduce Vectools, a command-line tool-suite to reduce reliance on custom scripts and improve reproducibility by offering a wide range of common easy-to-use functions for table and vector manipulation. Vectools also offers a number of vector related functions to speed up workflow development, such as simple machine learning and common statistics functions.

| S-EPMC6281013 | biostudies-literature

Skyline for Small Molecules: A Unifying Software Package for Quantitative Metabolomics.

Project description:Vendor-independent software tools for quantification of small molecules and metabolites are lacking, especially for targeted analysis workflows. Skyline is a freely available, open-source software tool for targeted quantitative mass spectrometry method development and data processing with a 10 year history supporting six major instrument vendors. Designed initially for proteomics analysis, we describe the expansion of Skyline to data for small molecule analysis, including selected reaction monitoring, high-resolution mass spectrometry, and calibrated quantification. This fundamental expansion of Skyline from a peptide-sequence-centric tool to a molecule-centric tool makes it agnostic to the source of the molecule while retaining Skyline features critical for workflows in both peptide and more general biomolecular research. The data visualization and interrogation features already available in Skyline, such as peak picking, chromatographic alignment, and transition selection, have been adapted to support small molecule data, including metabolomics. Herein, we explain the conceptual workflow for small molecule analysis using Skyline, demonstrate Skyline performance benchmarked against a comparable instrument vendor software tool, and present additional real-world applications. Further, we include step-by-step instructions on using Skyline for small molecule quantitative method development and data analysis on data acquired with a variety of mass spectrometers from multiple instrument vendors.

| S-EPMC7127945 | biostudies-literature

YAMP: a containerized workflow enabling reproducibility in metagenomics research.

Project description:YAMP ("Yet Another Metagenomics Pipeline") is a user-friendly workflow that enables the analysis of whole shotgun metagenomic data while using containerization to ensure computational reproducibility and facilitate collaborative research. YAMP can be executed on any UNIX-like system and offers seamless support for multiple job schedulers as well as for the Amazon AWS cloud. Although YAMP was developed to be ready to use by nonexperts, bioinformaticians will appreciate its flexibility, modularization, and simple customization.

| S-EPMC6047416 | biostudies-literature

Tools and techniques for computational reproducibility.

Project description:When reporting research findings, scientists document the steps they followed so that others can verify and build upon the research. When those steps have been described in sufficient detail that others can retrace the steps and obtain similar results, the research is said to be reproducible. Computers play a vital role in many research disciplines and present both opportunities and challenges for reproducibility. Computers can be programmed to execute analysis tasks, and those programs can be repeated and shared with others. The deterministic nature of most computer programs means that the same analysis tasks, applied to the same data, will often produce the same outputs. However, in practice, computational findings often cannot be reproduced because of complexities in how software is packaged, installed, and executed-and because of limitations associated with how scientists document analysis steps. Many tools and techniques are available to help overcome these challenges; here we describe seven such strategies. With a broad scientific audience in mind, we describe the strengths and limitations of each approach, as well as the circumstances under which each might be applied. No single strategy is sufficient for every scenario; thus we emphasize that it is often useful to combine approaches.

| S-EPMC4940747 | biostudies-other

A computational workflow for the analysis of 3’ Tag-Seq data

Project description:RNA-sequencing (RNA-seq) is a ubiquitous tool to profile genome-wide changes in gene expression. RNA-seq uses high-throughput sequencing technology to quantify the amount of RNA in a biological sample. With the increasing popularity of RNA-seq, many variations on the protocol have been proposed to extract unique and relevant information from biological samples. 3’ Tag-Seq (also called TagSeq, 3′ Tag-RNA-Seq, and Quant-Seq 3′ mRNA-Seq) is one RNA-seq variation, where the 3’ end of the transcript is selected and amplified to yield one copy of cDNA from each transcript in the biological sample.We present a simple, easy to use, and publicly available computational workflow to analyze 3’ Tag-Seq data. The workflow begins by trimming sequence adapters from raw FASTQ files. The trimmed sequence reads are checked for quality using FastQC, aligned to the reference genome, and read counts are obtained using STAR. Differential gene expression analysis is performed using DESeq2, based on differential analysis of gene count data. The outputs of this workflow are MA plots, tables of differentially expressed genes, and UpSet plots.This protocol is intended for users specifically interested in analyzing 3’ Tag-Seq data. As such, transcript length-based normalizations are not performed within the workflow. Future updates to this workflow could include custom analyses based on the gene counts table as well as data visualization enhancements.

2022-10-13 | GSE200778 | GEO

ROTS: An R package for reproducibility-optimized statistical testing.

Project description:Differential expression analysis is one of the most common types of analyses performed on various biological data (e.g. RNA-seq or mass spectrometry proteomics). It is the process that detects features, such as genes or proteins, showing statistically significant differences between the sample groups under comparison. A major challenge in the analysis is the choice of an appropriate test statistic, as different statistics have been shown to perform well in different datasets. To this end, the reproducibility-optimized test statistic (ROTS) adjusts a modified t-statistic according to the inherent properties of the data and provides a ranking of the features based on their statistical evidence for differential expression between two groups. ROTS has already been successfully applied in a range of different studies from transcriptomics to proteomics, showing competitive performance against other state-of-the-art methods. To promote its widespread use, we introduce here a Bioconductor R package for performing ROTS analysis conveniently on different types of omics data. To illustrate the benefits of ROTS in various applications, we present three case studies, involving proteomics and RNA-seq data from public repositories, including both bulk and single cell data. The package is freely available from Bioconductor (https://www.bioconductor.org/packages/ROTS).

| S-EPMC5470739 | biostudies-literature

Watchdog 2.0: New developments for reusability, reproducibility, and workflow execution.

Project description:Advances in high-throughput methods have brought new challenges for biological data analysis, often requiring many interdependent steps applied to a large number of samples. To address this challenge, workflow management systems, such as Watchdog, have been developed to support scientists in the (semi-)automated execution of large analysis workflows. Here, we present Watchdog 2.0, which implements new developments for module creation, reusability, and documentation and for reproducibility of analyses and workflow execution. Developments include a graphical user interface for semi-automatic module creation from software help pages, sharing repositories for modules and workflows, and a standardized module documentation format. The latter allows generation of a customized reference book of public and user-specific modules. Furthermore, extensive logging of workflow execution, module and software versions, and explicit support for package managers and container virtualization now ensures reproducibility of results. A step-by-step analysis protocol generated from the log file may, e.g., serve as a draft of a manuscript methods section. Finally, 2 new execution modes were implemented. One allows resuming workflow execution after interruption or modification without rerunning successfully executed tasks not affected by changes. The second one allows detaching and reattaching to workflow execution on a local computer while tasks continue running on computer clusters. Watchdog 2.0 provides several new developments that we believe to be of benefit for large-scale bioinformatics analysis and that are not completely covered by other competing workflow management systems. The software itself, module and workflow repositories, and comprehensive documentation are freely available at https://www.bio.ifi.lmu.de/watchdog.

| S-EPMC7298769 | biostudies-literature

A unifying computational framework for stability and flexibility of arousal.

Project description:Arousal and consciousness flexibly adjust to salient cues, but remain stable despite noise and disturbance. Diverse, highly interconnected neural networks govern the underlying transitions of behavioral state; these networks are robust but very complex. Frameworks from systems engineering provide powerful tools for understanding functional logic behind component complexity. From a general systems viewpoint, a minimum of three communicating control modules may enable flexibility and stability to coexist. Comparators would subtract current arousal from desired arousal, producing an error signal. Regulators would compute control signals from this error. Generators would convert control signals into arousal, which is fed back to comparators, to make the system noise-proof through self-correction. Can specific neurons correspond to these control elements? To explore this, here we consider the brain-wide orexin/hypocretin network, which is experimentally established to be vital for flexible and stable arousal. We discuss whether orexin neurons may act as comparators, and their target neurons as regulators and generators. Experiments are proposed for testing such predictions, based on computational simulations showing that comparators, regulators, and generators have distinct temporal signatures of activity. If some regulators integrate orexin-communicated errors, robust arousal control may be achieved via integral feedback (a basic engineering strategy for tracking a set-point despite noise). An integral feedback view also suggests functional roles for specific molecular aspects, such as differing life-spans of orexin peptides. The proposed framework offers a unifying logic for molecular, cellular, and network details of arousal systems, and provides insight into behavioral state transitions, complex behavior, and bases for disease.

| S-EPMC4202806 | biostudies-other

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data