Browse
Submit Data
Databases
API
Help

Dataset Information

25 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

SeqKit: A Cross-Platform and Ultrafast Toolkit for FASTA/Q File Manipulation.

ABSTRACT: FASTA and FASTQ are basic and ubiquitous formats for storing nucleotide and protein sequences. Common manipulations of FASTA/Q file include converting, searching, filtering, deduplication, splitting, shuffling, and sampling. Existing tools only implement some of these manipulations, and not particularly efficiently, and some are only available for certain operating systems. Furthermore, the complicated installation process of required packages and running environments can render these programs less user friendly. This paper describes a cross-platform ultrafast comprehensive toolkit for FASTA/Q processing. SeqKit provides executable binary files for all major operating systems, including Windows, Linux, and Mac OSX, and can be directly used without any dependencies or pre-configurations. SeqKit demonstrates competitive performance in execution time and memory usage compared to similar tools. The efficiency and usability of SeqKit enable researchers to rapidly accomplish common FASTA/Q file manipulations. SeqKit is open source and available on Github at https://github.com/shenwei356/seqkit.

SUBMITTER: Shen W

PROVIDER: S-EPMC5051824 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

FASTAFS: file system virtualisation of random access compressed FASTA files.

Project description:BackgroundThe FASTA file format, used to store polymeric sequence data, has become a bioinformatics file standard used for decades. The relatively large files require additional files, beyond the scope of the original format, to identify sequences and to provide random access. Multiple compressors have been developed to archive FASTA files back and forth, but these lack direct access to targeted content or metadata of the archive. Moreover, these solutions are not directly backwards compatible to FASTA files, resulting in limited software integration.ResultsWe designed a linux based toolkit that virtualises the content of DNA, RNA and protein FASTA archives into the filesystem by using filesystem in userspace. This guarantees in-sync virtualised metadata files and offers fast random-access decompression using bit encodings plus Zstandard (zstd). The toolkit, FASTAFS, can track all its system-wide running instances, allows file integrity verification and can provide, instantly, scriptable access to sequence files and is easy to use and deploy. The file compression ratios were comparable but not superior to other state of the art archival tools, despite the innovative random access feature implemented in FASTAFS.ConclusionsFASTAFS is a user-friendly and easy to deploy backwards compatible generic purpose solution to store and access compressed FASTA files, since it offers file system access to FASTA files as well as in-sync metadata files through file virtualisation. Using virtual filesystems as in-between layer offers format conversion without the need to rewrite code into different programming languages while preserving compatibility.

| S-EPMC8558547 | biostudies-literature

ThermoRawFileParser: Modular, Scalable, and Cross-Platform RAW File Conversion.

Project description:The field of computational proteomics is approaching the big data age, driven both by a continuous growth in the number of samples analyzed per experiment as well as by the growing amount of data obtained in each analytical run. In order to process these large amounts of data, it is increasingly necessary to use elastic compute resources such as Linux-based cluster environments and cloud infrastructures. Unfortunately, the vast majority of cross-platform proteomics tools are not able to operate directly on the proprietary formats generated by the diverse mass spectrometers. Here, we present ThermoRawFileParser, an open-source, cross-platform tool that converts Thermo RAW files into open file formats such as MGF and the HUPO-PSI standard file format mzML. To ensure the broadest possible availability and to increase integration capabilities with popular workflow systems such as Galaxy or Nextflow, we have also built Conda package and BioContainers container around ThermoRawFileParser. In addition, we implemented a user-friendly interface (ThermoRawFileParserGUI) for those users not familiar with command-line tools. Finally, we performed a benchmark of ThermoRawFileParser and msconvert to verify that the converted mzML files contain reliable quantitative results.

| S-EPMC7116465 | biostudies-literature

BigSeqKit: a parallel Big Data toolkit to process FASTA and FASTQ files at scale.

Project description:BackgroundHigh-throughput sequencing technologies have led to an unprecedented explosion in the amounts of sequencing data available, which are typically stored using FASTA and FASTQ files. We can find in the literature several tools to process and manipulate those type of files with the aim of transforming sequence data into biological knowledge. However, none of them are well fitted for processing efficiently very large files, likely in the order of terabytes in the following years, since they are based on sequential processing. Only some routines of the well-known seqkit tool are partly parallelized. In any case, its scalability is limited to use few threads on a single computing node.ResultsOur approach, BigSeqKit, takes advantage of a high-performance computing-Big Data framework to parallelize and optimize the commands included in seqkit with the aim of speeding up the manipulation of FASTA/FASTQ files. In this way, in most cases, it is from tens to hundreds of times faster than several state-of-the-art tools. At the same time, our toolkit is easy to use and install on any kind of hardware platform (local server or cluster), and its routines can be used as a bioinformatics library or from the command line.ConclusionsBigSeqKit is a very complete and ultra-fast toolkit to process and manipulate large FASTA and FASTQ files. It is publicly available at https://github.com/citiususc/BigSeqKit.

| S-EPMC10388699 | biostudies-literature

XPAT: a toolkit to conduct cross-platform association studies with heterogeneous sequencing datasets.

Project description:High-throughput sequencing data are increasingly being made available to the research community for secondary analyses, providing new opportunities for large-scale association studies. However, heterogeneity in target capture and sequencing technologies often introduce strong technological stratification biases that overwhelm subtle signals of association in studies of complex traits. Here, we introduce the Cross-Platform Association Toolkit, XPAT, which provides a suite of tools designed to support and conduct large-scale association studies with heterogeneous sequencing datasets. XPAT includes tools to support cross-platform aware variant calling, quality control filtering, gene-based association testing and rare variant effect size estimation. To evaluate the performance of XPAT, we conducted case-control association studies for three diseases, including 783 breast cancer cases, 272 ovarian cancer cases, 205 Crohn disease cases and 3507 shared controls (including 1722 females) using sequencing data from multiple sources. XPAT greatly reduced Type I error inflation in the case-control analyses, while replicating many previously identified disease-gene associations. We also show that association tests conducted with XPAT using cross-platform data have comparable performance to tests using matched platform data. XPAT enables new association studies that combine existing sequencing datasets to identify genetic loci associated with common diseases and other complex traits.

| S-EPMC5888834 | biostudies-other

1:1 FASTA update: Using the power of E-values in FASTA to detect potential allergen cross-reactivity.

Project description:In the context of regulatory assessment of transgenic proteins for potential allergenicity, a previous investigation demonstrated that a 1:1 FASTA comparison using an E-value of 1.0E-09 as a criterion is superior to the conventional FASTA search (using the whole sequence as a query) for >35% identity over 80 amino acids, but with improved specificity. A further study, using groups of known cross-reactive peanut allergens, indicates the sensitivity of this approach is superior to the conventional FASTA search and equivalent to 80-mer sliding window FASTA search recommended by WHO/FAO. Specifically, the 1:1 FASTA approach eliminated the technical issues resulting from lack of identification of short query sequences with high identity to known allergens, or high identity over short amino acid stretches, and different E-value settings when searching for >35% identity over 80aa. Based on the performance of this simple application of existing bioinformatics tools, and its ease of implementation and interpretation in the context of a regulatory assessment, we advocate that adoption of this 1:1 FASTA approach as a supplement to the FAO/WHO/ CODEX criterion (>35% identity over 80aa) formulated 13 years ago. Adoption of this approach eliminates many biologically irrelevant homology hits generated by the FAO/WHO/CODEX criterion and improves the safety assessment of GM crops.

| S-EPMC5598423 | biostudies-literature

fqtools: an efficient software suite for modern FASTQ file manipulation.

Project description:UnlabelledMany Next Generation Sequencing analyses involve the basic manipulation of input sequence data before downstream processing (e.g. searching for specific sequences, format conversion or basic file statistics). The rapidly increasing data volumes involved in NGS make any dataset manipulation a time-consuming and error-prone process. I have developed fqtools; a fast and reliable FASTQ file manipulation suite that can process the full set of valid FASTQ files, including those with multi-line sequences, whilst identifying invalid files. Fqtools is faster than similar tools, and is designed for use in automatic processing pipelines.Availability and implementationfqtools is open source and is available at: https://github.com/alastair-droop/fqtoolsSupplementary informationSupplementary data are available at Bioinformatics online.Contacta.p.droop@leeds.ac.uk.

| S-EPMC4908325 | biostudies-literature

PlasmaDNA: a free, cross-platform plasmid manipulation program for molecular biology laboratories.

Project description:BackgroundMost molecular biology experiments, and the techniques associated with this field of study, involve a great deal of engineering in the form of molecular cloning. Like all forms of engineering, perfect information about the starting material is crucial for successful completion of design and strategies.ResultsWe have generated a program that allows complete in silico simulation of the cloning experiment. Starting with a primary DNA sequence, PlasmaDNA looks for restriction sites, open reading frames, primer annealing sequences, and various common domains. The databases are easily expandable by the user to fit his most common cloning needs. PlasmaDNA can manage and graphically represent multiple sequences at the same time, and keeps in memory the overhangs at the end of the sequences if any. This means that it is possible to virtually digest fragments, to add the digestion products to the project, and to ligate together fragments with compatible ends to generate the new sequences. Polymerase Chain Reaction (PCR) fragments can also be virtually generated using the primer database, automatically adding to the fragments any 5' extra sequences present in the primers.ConclusionPlasmaDNA is a program available both on Windows and Apple operating systems, designed to facilitate molecular cloning experiments by building a visual map of the DNA. It then allows the complete planning and simulation of the cloning experiment. It also automatically updates the new sequences generated in the process, which is an important help in practice. The capacity to maintain multiple sequences in the same file can also be used to archive the various steps and strategies involved in the cloning of each construct. The program is freely available for download without charge or restriction.

| S-EPMC2075515 | biostudies-literature

Cross-Platform Comparison

Project description:Cross-Platform Comparison

| PRJNA888143 | ENA

Cross-platform study

Project description:Cross-platform study

| PRJNA95853 | ENA

Cross-platform study

Project description:Gene expression microarrays have made a profound impact in biomedical research. The diversity of platforms and analytical methods has made comparison of data from multiple platforms very challenging. In this study, we describe a framework for comparisons across platforms and laboratories. We have attempted to include nearly all the available commercial and in house platforms. Using probe sequences matched at the exon level improved consistency of measurements across the different microarray platforms compared to annotation-based matches. Generally, consistency was good for highly expressed genes, and variable for genes with lower expression values as confirmed by QRT-PCR. Concordance of measurements was higher between laboratories on the same platform than across platforms. We demonstrate that, after stringent pre-processing, commercial arrays were more consistent than in-house arrays, and by most measures, one-dye platforms were more consistent than two-dye platforms. This SuperSeries is composed of the SubSeries listed below.

2006-06-01 | GSE4854 | GEO

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data