Speeding up ecological and evolutionary computations in R; essentials of high performance computing for biologists.
Ontology highlight
ABSTRACT:
SUBMITTER: Visser MD
PROVIDER: S-EPMC4374948 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The function of a newly sequenced gene can be discovered by determining its sequence homology with known proteins. BLAST is the most extensively used sequence analysis program for sequence similarity search in large databases of sequences. With the advent of next generation sequencing technologies it has now become possible to study genes and their expression at a genome-wide scale through RNA-seq and metagenome sequencing experiments. Functional annotation of all the genes is done by sequence similarity search against multiple protein databases. This annotation task is computationally very intensive and can take days to obtain complete results. The program mpiBLAST, an open-source parallelization of BLAST that achieves superlinear speedup, can be used to accelerate large-scale annotation by using supercomputers and high performance computing (HPC) clusters. Although many parallel bioinformatics applications using the Message Passing Interface (MPI) are available in the public domain, researchers are reluctant to use them due to lack of expertise in the Linux command line and relevant programming experience. With these limitations, it becomes difficult for biologists to use mpiBLAST for accelerating annotation. No web interface is available in the open-source domain for mpiBLAST. We have developed WImpiBLAST, a user-friendly open-source web interface for parallel BLAST searches. It is implemented in Struts 1.3 using a Java backbone and runs atop the open-source Apache Tomcat Server. WImpiBLAST supports script creation and job submission features and also provides a robust job management interface for system administrators. It combines script creation and modification features with job monitoring and management through the Torque resource manager on a Linux-based HPC cluster. Use case information highlights the acceleration of annotation analysis achieved by using WImpiBLAST. Here, we describe the WImpiBLAST web interface features and architecture, explain design decisions, describe workflows and provide a detailed analysis.
| S-EPMC4076281 | biostudies-literature
Project description:In this paper we propose a vectorized implementation of the non-parametric bootstrap for statistics based on sample moments. Basically, we adopt the multinomial sampling formulation of the non-parametric bootstrap, and compute bootstrap replications of sample moment statistics by simply weighting the observed data according to multinomial counts instead of evaluating the statistic on a resampled version of the observed data. Using this formulation we can generate a matrix of bootstrap weights and compute the entire vector of bootstrap replications with a few matrix multiplications. Vectorization is particularly important for matrix-oriented programming languages such as R, where matrix/vector calculations tend to be faster than scalar operations implemented in a loop. We illustrate the application of the vectorized implementation in real and simulated data sets, when bootstrapping Pearson's sample correlation coefficient, and compared its performance against two state-of-the-art R implementations of the non-parametric bootstrap, as well as a straightforward one based on a for loop. Our investigations spanned varying sample sizes and number of bootstrap replications. The vectorized bootstrap compared favorably against the state-of-the-art implementations in all cases tested, and was remarkably/considerably faster for small/moderate sample sizes. The same results were observed in the comparison with the straightforward implementation, except for large sample sizes, where the vectorized bootstrap was slightly slower than the straightforward implementation due to increased time expenditures in the generation of weight matrices via multinomial sampling.
| S-EPMC4488363 | biostudies-literature
Project description:MOTIVATION: Biologists and chemists are facing problems of high computational complexity that require the use of several computers organized in clusters or in specialized grids. Examples of such problems can be found in molecular dynamics (MD), in silico screening, and genome analysis. Grid Computing and Cloud Computing are becoming prevalent mainly because of their competitive performance/cost ratio. Regrettably, the diffusion of Grid Computing is strongly limited because two main limitations: it is confined to scientists with strong Computer Science background and the analyses of the large amount of data produced can be cumbersome it. We have developed a package named GRIMD to provide an easy and flexible implementation of distributed computing for the Bioinformatics community. GRIMD is very easy to install and maintain, and it does not require any specific Computer Science skill. Moreover, permits preliminary analysis on the distributed machines to reduce the amount of data to transfer. GRIMD is very flexible because it shields the typical computational biologist from the need to write specific code for tasks such as molecular dynamics or docking calculations. Furthermore, it permits an efficient use of GPU cards whenever is possible. GRIMD calculations scale almost linearly and, therefore, permits to exploit efficiently each machine in the network. Here, we provide few examples of grid computing in computational biology (MD and docking) and bioinformatics (proteome analysis). AVAILABILITY: GRIMD is available for free for noncommercial research at www.yadamp.unisa.it/grimd. SUPPLEMENTARY INFORMATION: www.yadamp.unisa.it/grimd/howto.aspx.
| S-EPMC3916819 | biostudies-other
Project description:In this paper we will describe two new optimisations implemented in MadGraph5_aMC@NLO, both of which are designed to speed-up the computation of leading-order processes (for any model). First we implement a new method to evaluate the squared matrix element, dubbed helicity recycling, which results in factor of two speed-up. Second, we have modified the multi-channel handling of the phase-space integrator providing tremendous speed-up for VBF-like processes (up to thousands times faster).Supplementary informationThe online version contains supplementary material available at 10.1140/epjc/s10052-021-09204-7.
| S-EPMC8136271 | biostudies-literature
Project description:Next-generation sequencing (NGS) technologies have led to a huge amount of genomic data that need to be analyzed and interpreted. This fact has a huge impact on the DNA sequence alignment process, which nowadays requires the mapping of billions of small DNA sequences onto a reference genome. In this way, sequence alignment remains the most time-consuming stage in the sequence analysis workflow. To deal with this issue, state of the art aligners take advantage of parallelization strategies. However, the existent solutions show limited scalability and have a complex implementation. In this work we introduce SparkBWA, a new tool that exploits the capabilities of a big data technology as Spark to boost the performance of one of the most widely adopted aligner, the Burrows-Wheeler Aligner (BWA). The design of SparkBWA uses two independent software layers in such a way that no modifications to the original BWA source code are required, which assures its compatibility with any BWA version (future or legacy). SparkBWA is evaluated in different scenarios showing noticeable results in terms of performance and scalability. A comparison to other parallel BWA-based aligners validates the benefits of our approach. Finally, an intuitive and flexible API is provided to NGS professionals in order to facilitate the acceptance and adoption of the new tool. The source code of the software described in this paper is publicly available at https://github.com/citiususc/SparkBWA, with a GPL3 license.
| S-EPMC4868289 | biostudies-literature
Project description:In the last decade, bioinformatics has become an indispensable branch of modern science research, experiencing an explosion in financial support, developed applications and data collection. The growth of the datasets that are emerging from research laboratories, industry, the health sector, etc., are increasingly raising the levels of demand in computing power and storage. Processing biological data, in the large scales of these datasets, often requires the use of High Performance Computing (HPC) resources, especially when dealing with certain types of omics data, such as genomic and metagenomic data. Such computational resources not only require substantial investments, but they also involve high maintenance costs. More importantly, in order to keep good returns from the investments, specific training needs to be put in place to ensure that wasting is minimized. Furthermore, given that bioinformatics is a highly interdisciplinary field where several other domains intersect (such as biology, chemistry, physics and computer science), researchers from these areas also require bioinformatics-specific training in HPC, in order to fully take advantage of supercomputing centers. In this document, we describe our experience in training researchers from several different disciplines in HPC, as applied to bioinformatics under the framework of the leading European bioinformatics platform ELIXIR, and analyze both the content and outcomes of the course.
| S-EPMC6299036 | biostudies-literature
Project description:Technological advances in the past decade, hardware and software alike, have made access to high-performance computing (HPC) easier than ever. We review these advances from a statistical computing perspective. Cloud computing makes access to supercomputers affordable. Deep learning software libraries make programming statistical algorithms easy and enable users to write code once and run it anywhere-from a laptop to a workstation with multiple graphics processing units (GPUs) or a supercomputer in a cloud. Highlighting how these developments benefit statisticians, we review recent optimization algorithms that are useful for high-dimensional models and can harness the power of HPC. Code snippets are provided to demonstrate the ease of programming. We also provide an easy-to-use distributed matrix data structure suitable for HPC. Employing this data structure, we illustrate various statistical applications including large-scale positron emission tomography and ℓ1-regularized Cox regression. Our examples easily scale up to an 8-GPU workstation and a 720-CPU-core cluster in a cloud. As a case in point, we analyze the onset of type-2 diabetes from the UK Biobank with 200,000 subjects and about 500,000 single nucleotide polymorphisms using the HPC ℓ1-regularized Cox regression. Fitting this half-million-variate model takes less than 45 minutes and reconfirms known associations. To our knowledge, this is the first demonstration of the feasibility of penalized regression of survival outcomes at this scale.
| S-EPMC10168006 | biostudies-literature
Project description:BackgroundGait speed is an important outcome that relates to mobility, function, and mortality, and is altered in people with Parkinson's disease (PwPD). However, changes in gait speed may not reflect changes in other important aspects of gait.ObjectiveTo characterize which outcomes change concomitantly with walking speed in PwPD. This information can inform the choice of outcome variables for characterizing and tracking gait performance in this population.Methods67 PwPD and 40 neurotypical adults completed 2-minute overground walking bouts at comfortable and fast self-selected speeds. Eight inertial sensors were used to characterize gait and turning. We identified a subset of participants (38 per group) where the PD participant's "fast" walk was similar speed to neurotypical participants "comfortable" walk, facilitating an across-group gait comparison controlling for gait speed.ResultsWalking at fast gait speed compared to comfortable lead to significant changes in stride length, cadence, and stride time variability, but not in steps to turn, trunk ROM, and trunk and lumbar stability in PwPD. Sub-group analyses showed that despite walking at a similar speed as neurotypical adults, PwPD exhibit altered turning outcomes, lumbar stability, and stride length/cadence.ConclusionsGait speed is a critical outcome for characterizing mobility. However, in PwPD, several important outcomes do not exhibit a uniform relationship with gait speed, and remain altered compared to neurotypical adults despite "normalizing" walking speed. Given the complex relationship between gait speed and other gait quality measures, care should be taken when choosing outcome measures to characterize the breadth of gait abnormality in PwPD.
| S-EPMC7304052 | biostudies-literature
Project description: Not available
| S-EPMC3504542 | biostudies-literature
Project description:Numerous biological processes involve association of a protein with its binding partner, an event that is preceded by a diffusion-mediated search bringing the two partners together. Often hindered by crowding in biologically relevant environments, three-dimensional diffusion can be slow and result in long bimolecular association times. Similarly, the initial association step between two binding partners often represents a rate-limiting step in biotechnologically relevant reactions. We demonstrate the practical use of an 11-a.a. DNA-interacting peptide derived from adenovirus to reduce the dimensionality of diffusional search processes and speed up associations between biological macromolecules. We functionalise binding partners with the peptide and demonstrate that the ability of the peptide to one-dimensionally diffuse along DNA results in a 20-fold reduction in reaction time. We also show that modifying PCR primers with the peptide sled enables significant acceleration of standard PCR reactions.
| S-EPMC4762599 | biostudies-literature