VolPy: Automated and scalable analysis pipelines for voltage imaging datasets.
Ontology highlight
ABSTRACT:
SUBMITTER: Cai C
PROVIDER: S-EPMC8075204 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The processing of brain diffusion tensor imaging (DTI) data for large cohort studies requires fully automatic pipelines to perform quality control (QC) and artifact/outlier removal procedures on the raw DTI data prior to calculation of diffusion parameters. In this study, three automatic DTI processing pipelines, each complying with the general ENIGMA framework, were designed by uniquely combining multiple image processing software tools. Different QC procedures based on the RESTORE algorithm, the DTIPrep protocol, and a combination of both methods were compared using simulated ground truth and artifact containing DTI datasets modeling eddy current induced distortions, various levels of motion artifacts, and thermal noise. Variability was also examined in 20 DTI datasets acquired in subjects with vascular cognitive impairment (VCI) from the multi-site Ontario Neurodegenerative Disease Research Initiative (ONDRI). The mean fractional anisotropy (FA), mean diffusivity (MD), axial diffusivity (AD), and radial diffusivity (RD) were calculated in global brain grey matter (GM) and white matter (WM) regions. For the simulated DTI datasets, the measure used to evaluate the performance of the pipelines was the normalized difference between the mean DTI metrics measured in GM and WM regions and the corresponding ground truth DTI value. The performance of the proposed pipelines was very similar, particularly in FA measurements. However, the pipeline based on the RESTORE algorithm was the most accurate when analyzing the artifact containing DTI datasets. The pipeline that combined the DTIPrep protocol and the RESTORE algorithm produced the lowest standard deviation in FA measurements in normal appearing WM across subjects. We concluded that this pipeline was the most robust and is preferred for automated analysis of multisite brain DTI data.
| S-EPMC6924651 | biostudies-literature
Project description:Quantification of cellular antigens and their interactions via antibody-based detection methods are widely used in scientific research. Accurate high-throughput quantitation of these assays using general image analysis software can be time consuming and challenging, particularly when attempted by users with limited image processing and analysis knowledge. To overcome this, we have designed Andy's Algorithms, a series of automated image analysis pipelines for FIJI, that permits rapid, accurate and reproducible batch-processing of 3,3'-diaminobenzidine (DAB) immunohistochemistry, proximity ligation assays (PLAs) and other common assays. Andy's Algorithms incorporates a step-by-step tutorial and optimization pipeline to make batch image analysis simple for the untrained user and adaptable across laboratories. Andy's algorithms provide a simpler, faster, standardized work flow compared to existing programs, while offering equivalent performance and additional features, in a free to use open-source application of FIJI. Andy's Algorithms are available at GitHub, publicly accessed at https://github.com/andlaw1841/Andy-s-Algorithm .
| S-EPMC5691210 | biostudies-literature
Project description:BACKGROUND:With the growth of available sequenced datasets, analysis of heterogeneous processed data can answer increasingly relevant biological and clinical questions. Scientists are challenged in performing efficient and reproducible data extraction and analysis pipelines over heterogeneously processed datasets. Available software packages are suitable for analyzing experimental files from such datasets one by one, but do not scale to thousands of experiments. Moreover, they lack proper support for metadata manipulation. RESULTS:We present PyGMQL, a novel software for the manipulation of region-based genomic files and their relative metadata, built on top of the GMQL genomic big data management system. PyGMQL provides a set of expressive functions for the manipulation of region data and their metadata that can scale to arbitrary clusters and implicitly apply to thousands of files, producing millions of regions. PyGMQL provides data interoperability, distribution transparency and query outsourcing. The PyGMQL package integrates scalable data extraction over the Apache Spark engine underlying the GMQL implementation with native Python support for interactive data analysis and visualization. It supports data interoperability, solving the impedance mismatch between executing set-oriented queries and programming in Python. PyGMQL provides distribution transparency (the ability to address a remote dataset) and query outsourcing (the ability to assign processing to a remote service) in an orthogonal way. Outsourced processing can address cloud-based installations of the GMQL engine. CONCLUSIONS:PyGMQL is an effective and innovative tool for supporting tertiary data extraction and analysis pipelines. We demonstrate the expressiveness and performance of PyGMQL through a sequence of biological data analysis scenarios of increasing complexity, which highlight reproducibility, expressive power and scalability.
| S-EPMC6842186 | biostudies-literature
Project description:DamID is a powerful technique for identifying regions of the genome bound by a DNA-binding (or DNA-associated) protein. Currently no method exists for automatically processing next-generation sequencing DamID (DamID-seq) data, and the use of DamID-seq datasets with normalisation based on read-counts alone can lead to high background and the loss of bound signal. DamID-seq thus presents novel challenges in terms of normalisation and background minimisation. We describe here damidseq_pipeline, a software pipeline that performs automatic normalisation and background reduction on multiple DamID-seq FASTQ or BAM datasets. Single replicate profiling of pol II occupancy in 3rd instar larval neuroblasts of Drosophila
2015-06-26 | E-GEOD-69184 | biostudies-arrayexpress
Project description:Understanding the role of short-interfering RNA (siRNA) in diverse biological processes is of current interest and often approached through small RNA sequencing. However, analysis of these datasets is difficult due to the complexity of biological RNA processing pathways, which differ between species. Several properties like strand specificity, length distribution, and distribution of soft-clipped bases are few parameters known to guide researchers in understanding the role of siRNAs. We present RAPID, a generic eukaryotic siRNA analysis pipeline, which captures information inherent in the datasets and automatically produces numerous visualizations as user-friendly HTML reports, covering multiple categories required for siRNA analysis. RAPID also facilitates an automated comparison of multiple datasets, with one of the normalization techniques dedicated for siRNA knockdown analysis, and integrates differential expression analysis using DESeq2.Availability and implementationRAPID is available under MIT license at https://github.com/SchulzLab/RAPID. We recommend using it as a conda environment available from https://anaconda.org/bioconda/rapid.
| S-EPMC6462184 | biostudies-literature
Project description:The preprocessing of functional magnetic resonance imaging (fMRI) data is necessary to remove unwanted artifacts and transform the data into a standard format. There are several neuroimaging data processing tools that are widely used, such as SPM, AFNI, FSL, FreeSurfer, Workbench, and fMRIPrep. Different data preprocessing pipelines yield differing results, which might reduce the reproducibility of neuroimaging studies. Here, we developed a preprocessing pipeline for T1-weighted structural MRI and fMRI data by combining components of well-known software packages to fully incorporate recent developments in MRI preprocessing into a single coherent software package. The developed software, called FuNP (Fusion of Neuroimaging Preprocessing) pipelines, is fully automatic and provides both volume- and surface-based preprocessing pipelines with a user-friendly graphical interface. The reliability of the software was assessed by comparing resting-state networks (RSNs) obtained using FuNP with pre-defined RSNs using open research data (n = 90). The obtained RSNs were well-matched with the pre-defined RSNs, suggesting that the pipelines in FuNP are reliable. In addition, image quality metrics (IQMs) were calculated from the results of three different software packages (i.e., FuNP, FSL, and fMRIPrep) to compare the quality of the preprocessed data. We found that our FuNP outperformed other software in terms of temporal characteristics and artifacts removal. We validated our pipeline with independent local data (n = 28) in terms of IQMs. The IQMs of our local data were similar to those obtained from the open research data. The codes for FuNP are available online to help researchers.
| S-EPMC6378808 | biostudies-literature
Project description:For the last two decades, researchers have worked independently to automate protein model building, and four widely used software pipelines have been developed for this purpose: ARP/wARP, Buccaneer, Phenix AutoBuild and SHELXE. Here, the usefulness of combining these pipelines to improve the built protein structures by running them in pairwise combinations is examined. The results show that integrating these pipelines can lead to significant improvements in structure completeness and Rfree. In particular, running Phenix AutoBuild after Buccaneer improved structure completeness for 29% and 75% of the data sets that were examined at the original resolution and at a simulated lower resolution, respectively, compared with running Phenix AutoBuild on its own. In contrast, Phenix AutoBuild alone produced better structure completeness than the two pipelines combined for only 7% and 3% of these data sets.
| S-EPMC7466752 | biostudies-literature
Project description:SUMMARY:We introduce Tibanna, an open-source software tool for automated execution of bioinformatics pipelines on Amazon Web Services (AWS). Tibanna accepts reproducible and portable pipeline standards including Common Workflow Language (CWL), Workflow Description Language (WDL) and Docker. It adopts a strategy of isolation and optimization of individual executions, combined with a serverless scheduling approach. Pipelines are executed and monitored using local commands or the Python Application Programming Interface (API) and cloud configuration is automatically handled. Tibanna is well suited for projects with a range of computational requirements, including those with large and widely fluctuating loads. Notably, it has been used to process terabytes of data for the 4D Nucleome (4DN) Network. AVAILABILITY AND IMPLEMENTATION:Source code is available on GitHub at https://github.com/4dn-dcic/tibanna. SUPPLEMENTARY INFORMATION:Supplementary data are available at Bioinformatics online.
| S-EPMC6931271 | biostudies-literature
Project description:Proteins are macromolecules that perform essential biological functions which depend on their three-dimensional structure. Determining this structure involves complex laboratory and computational work. For the computational work, multiple software pipelines have been developed to build models of the protein structure from crystallographic data. Each of these pipelines performs differently depending on the characteristics of the electron-density map received as input. Identifying the best pipeline to use for a protein structure is difficult, as the pipeline performance differs significantly from one protein structure to another. As such, researchers often select pipelines that do not produce the best possible protein models from the available data. Here, a software tool is introduced which predicts key quality measures of the protein structures that a range of pipelines would generate if supplied with a given crystallographic data set. These measures are crystallographic quality-of-fit indicators based on included and withheld observations, and structure completeness. Extensive experiments carried out using over 2500 data sets show that the tool yields accurate predictions for both experimental phasing data sets (at resolutions between 1.2 and 4.0 Å) and molecular-replacement data sets (at resolutions between 1.0 and 3.5 Å). The tool can therefore provide a recommendation to the user concerning the pipelines that should be run in order to proceed most efficiently to a depositable model.
| S-EPMC8647178 | biostudies-literature
Project description:A multistage clustering and data processing method, SWIFT (detailed in a companion manuscript), has been developed to detect rare subpopulations in large, high-dimensional flow cytometry datasets. An iterative sampling procedure initially fits the data to multidimensional Gaussian distributions, then splitting and merging stages use a criterion of unimodality to optimize the detection of rare subpopulations, to converge on a consistent cluster number, and to describe non-Gaussian distributions. Probabilistic assignment of cells to clusters, visualization, and manipulation of clusters by their cluster medians, facilitate application of expert knowledge using standard flow cytometry programs. The dual problems of rigorously comparing similar complex samples, and enumerating absent or very rare cell subpopulations in negative controls, were solved by assigning cells in multiple samples to a cluster template derived from a single or combined sample. Comparison of antigen-stimulated and control human peripheral blood cell samples demonstrated that SWIFT could identify biologically significant subpopulations, such as rare cytokine-producing influenza-specific T cells. A sensitivity of better than one part per million was attained in very large samples. Results were highly consistent on biological replicates, yet the analysis was sensitive enough to show that multiple samples from the same subject were more similar than samples from different subjects. A companion manuscript (Part 1) details the algorithmic development of SWIFT.
| S-EPMC4238823 | biostudies-other