Performance comparison and evaluation of software tools for microRNA deep-sequencing data analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Li Y
PROVIDER: S-EPMC3378883 | biostudies-literature | 2012 May
REPOSITORIES: biostudies-literature
Publications
Performance comparison and evaluation of software tools for microRNA deep-sequencing data analysis.
Li Yue Y Zhang Zhuo Z Liu Feng F Vongsangnak Wanwipa W Jing Qing Q Shen Bairong B
Nucleic acids research 20120128 10
With the development of next-generation sequencing (NGS) techniques, many software tools have emerged for the discovery of novel microRNAs (miRNAs) and for analyzing the miRNAs expression profiles. An overall evaluation of these diverse software tools is lacking. In this study, we evaluated eight software tools based on their common feature and key algorithms. Three deep-sequencing data sets were collected from different species and used to assess the computational time, sensitivity and accuracy ...[more]
Similar Datasets
Project description:Background:For evaluating the fate of xenobiotics in the environment, a variety of degradation or environmental metabolism experiments are routinely conducted. The data generated in such experiments are evaluated by optimizing the parameters of kinetic models in a way that the model simulation fits the data. No comparison of the main software tools currently in use has been published to date. This article shows a comparison of numerical results as well as an overall, somewhat subjective comparison based on a scoring system using a set of criteria. The scoring was separately performed for two types of uses. Uses of type I are routine evaluations involving standard kinetic models and up to three metabolites in a single compartment. Evaluations involving non-standard model components, more than three metabolites or more than a single compartment belong to use type II. For use type I, usability is most important, while the flexibility of the model definition is most important for use type II. Results:Test datasets were assembled that can be used to compare the numerical results for different software tools. These datasets can also be used to ensure that no unintended or erroneous behaviour is introduced in newer versions. In the comparison of numerical results, good agreement between the parameter estimates was observed for datasets with up to three metabolites. For the now unmaintained reference software DegKinManager/ModelMaker, and for OpenModel which is still under development, user options were identified that should be taken care of in order to obtain results that are as reliable as possible. Based on the scoring system mentioned above, the software tools gmkin, KinGUII and CAKE received the best scores for use type I. Out of the 15 software packages compared with respect to use type II, again gmkin and KinGUII were the first two, followed by the script based tool mkin, which is the technical basis for gmkin, and by OpenModel. Conclusions:Based on the evaluation using the system of criteria mentioned above and the comparison of numerical results for the suite of test datasets, the software tools gmkin, KinGUII and CAKE are recommended for use type I, and gmkin and KinGUII for use type II. For users that prefer to work with scripts instead of graphical user interfaces, mkin is recommended. For future software evaluations, it is recommended to include a measure for the total time that a typical user needs for a kinetic evaluation into the scoring scheme. It is the hope of the authors that the publication of test data, source code and overall rankings foster the evolution of useful and reliable software in the field.
| S-EPMC5960009 | biostudies-literature
Project description: Not available
| S-EPMC8497909 | biostudies-literature
Project description:MicroRNAs (miRNAs) regulate many genes critical for tumorigenesis. We profiled miRNAs from 11 normal breast tissues, 17 non-invasive, 151 invasive breast carcinomas, and 6 cell lines by in-house-developed barcoded Solexa sequencing. miRNAs were organized in genomic clusters representing promoter-controlled miRNA expression and sequence families representing seed-sequence-dependent miRNA-target regulation. Unsupervised clustering of samples by miRNA sequence families best reflected the clustering based on mRNA expression available for this sample set. Clustering and comparative analysis of miRNA read frequencies showed that normal breast samples were separated from most non-invasive ductal carcinoma in situ and invasive carcinomas by increased miR-21 (the most abundant miRNA in carcinomas) and multiple decreased miRNA families (including mir-98/let-7), with most miRNA changes apparent already in the non-invasive carcinomas. In addition, patients that went on to develop metastasis demonstrated increased expression of mir-423, and triple negative breast carcinomas were most distinct from other tumor subtypes due to up-regulation of the mir-17~92 cluster. However, absolute miRNA levels between normal breast and carcinomas did not reveal any significant differences. We also discovered two polymorphic nucleotide variations among the more abundant miRNAs miR-181a (T19G) and miR-185 (T16G), but we did not identify nucleotide variations expected for classical tumor suppressor function associated with miRNAs. The differentiation of tumor subtypes and prediction of metastasis based on miRNA levels is statistically possible, but is not driven by deregulation of abundant miRNAs, implicating far fewer miRNAs in tumorigenic processes than previously suggested. 21 barcoded sequencing runs, including 185 unique samples and 54 samples in replicate (6 in triplicate and the remaining in duplicate). The details of each sample can be found in Supplementary Tables S1 and S2.
| S-ECPF-GEOD-29173 | biostudies-other
Project description:BACKGROUND:Bioinformatic tools for the enrichment of 'omics' datasets facilitate interpretation and understanding of data. To date few are suitable for metabolomics datasets. The main objective of this work is to give a critical overview, for the first time, of the performance of these tools. To that aim, datasets from metabolomic repositories were selected and enriched data were created. Both types of data were analysed with these tools and outputs were thoroughly examined. RESULTS:An exploratory multivariate analysis of the most used tools for the enrichment of metabolite sets, based on a non-metric multidimensional scaling (NMDS) of Jaccard's distances, was performed and mirrored their diversity. Codes (identifiers) of the metabolites of the datasets were searched in different metabolite databases (HMDB, KEGG, PubChem, ChEBI, BioCyc/HumanCyc, LipidMAPS, ChemSpider, METLIN and Recon2). The databases that presented more identifiers of the metabolites of the dataset were PubChem, followed by METLIN and ChEBI. However, these databases had duplicated entries and might present false positives. The performance of over-representation analysis (ORA) tools, including BioCyc/HumanCyc, ConsensusPathDB, IMPaLA, MBRole, MetaboAnalyst, Metabox, MetExplore, MPEA, PathVisio and Reactome and the mapping tool KEGGREST, was examined. Results were mostly consistent among tools and between real and enriched data despite the variability of the tools. Nevertheless, a few controversial results such as differences in the total number of metabolites were also found. Disease-based enrichment analyses were also assessed, but they were not found to be accurate probably due to the fact that metabolite disease sets are not up-to-date and the difficulty of predicting diseases from a list of metabolites. CONCLUSIONS:We have extensively reviewed the state-of-the-art of the available range of tools for metabolomic datasets, the completeness of metabolite databases, the performance of ORA methods and disease-based analyses. Despite the variability of the tools, they provided consistent results independent of their analytic approach. However, more work on the completeness of metabolite and pathway databases is required, which strongly affects the accuracy of enrichment analyses. Improvements will be translated into more accurate and global insights of the metabolome.
| S-EPMC5749025 | biostudies-literature
Project description:MicroRNAs (miRNAs) regulate many genes critical for tumorigenesis. We profiled miRNAs from 11 normal breast tissues, 17 non-invasive, 151 invasive breast carcinomas, and 6 cell lines by in-house-developed barcoded Solexa sequencing. miRNAs were organized in genomic clusters representing promoter-controlled miRNA expression and sequence families representing seed-sequence-dependent miRNA-target regulation. Unsupervised clustering of samples by miRNA sequence families best reflected the clustering based on mRNA expression available for this sample set. Clustering and comparative analysis of miRNA read frequencies showed that normal breast samples were separated from most non-invasive ductal carcinoma in situ and invasive carcinomas by increased miR-21 (the most abundant miRNA in carcinomas) and multiple decreased miRNA families (including mir-98/let-7), with most miRNA changes apparent already in the non-invasive carcinomas. In addition, patients that went on to develop metastasis demonstrated increased expression of mir-423, and triple negative breast carcinomas were most distinct from other tumor subtypes due to up-regulation of the mir-17~92 cluster. However, absolute miRNA levels between normal breast and carcinomas did not reveal any significant differences. We also discovered two polymorphic nucleotide variations among the more abundant miRNAs miR-181a (T19G) and miR-185 (T16G), but we did not identify nucleotide variations expected for classical tumor suppressor function associated with miRNAs. The differentiation of tumor subtypes and prediction of metastasis based on miRNA levels is statistically possible, but is not driven by deregulation of abundant miRNAs, implicating far fewer miRNAs in tumorigenic processes than previously suggested. mRNA profiles for 161 normal breast and invasive breast carcinomas using NKI Operon V3 arrays; These samples have matched entries for miRNA profiles also included in this GEO submission.
| S-ECPF-GEOD-29174 | biostudies-other
Project description:Pharmacogenomics (PGx) investigates the influence of genetics on drug responses, enabling tailored treatments for personalized healthcare. This study assessed the accuracy of genotyping six genes using whole genome sequencing with four different computational tools and various sequencing depths. The effects of using different reference genomes (GRCh38 and GRCh37) and sequence aligners (BWA-MEM and Bowtie2) were also explored. The results showed generally minor variations in tool performance across most genes; however, more notable discrepancies were observed in the analysis of the complex CYP2D6 gene. Cyrius, a CYP2D6-specific tool, demonstrated the most robust performance, achieving the highest concordance rates for CYP2D6 in all instances, comparable to the consensus approach in most cases. There were rather small differences between the samples with 20× coverage depth and those with higher depth, but the decreased performance was more evident at lower depths, particularly at 5×. Additionally, variations in CYP2D6 results were observed when samples were aligned to different reference genomes using the same method, or to the same genome using different aligners, which led to reporting incorrect rare star alleles in several cases. These findings inform the selection of optimal PGx tools and methodologies as well as suggest that employing a consensus approach with two or more tools might be preferable for certain genes and tool combinations, especially at lower sequencing depths, to ensure accurate results. Additionally, we show how the upstream alignment can affect the performance of tools, an important factor to take into account.
| S-EPMC11315677 | biostudies-literature
Project description:The advent of next-generation sequencing technologies is accompanied with the development of many whole-genome sequence assembly methods and software, especially for de novo fragment assembly. Due to the poor knowledge about the applicability and performance of these software tools, choosing a befitting assembler becomes a tough task. Here, we provide the information of adaptivity for each program, then above all, compare the performance of eight distinct tools against eight groups of simulated datasets from Solexa sequencing platform. Considering the computational time, maximum random access memory (RAM) occupancy, assembly accuracy and integrity, our study indicate that string-based assemblers, overlap-layout-consensus (OLC) assemblers are well-suited for very short reads and longer reads of small genomes respectively. For large datasets of more than hundred millions of short reads, De Bruijn graph-based assemblers would be more appropriate. In terms of software implementation, string-based assemblers are superior to graph-based ones, of which SOAPdenovo is complex for the creation of configuration file. Our comparison study will assist researchers in selecting a well-suited assembler and offer essential information for the improvement of existing assemblers or the developing of novel assemblers.
| S-EPMC3056720 | biostudies-literature
Project description:miRNAs are important regulators of gene expression that play a key role in many biological processes. High-throughput techniques allow researchers to discover and characterize large sets of miRNAs, and enrichment analysis tools are becoming increasingly important in decoding which miRNAs are implicated in biological processes. Enrichment analysis of miRNA targets is the standard technique for functional analysis, but this approach carries limitations and bias; alternatives are currently being proposed, based on direct and curated annotations. In this review, we describe the two workflows of miRNAs enrichment analysis, based on target gene or miRNA annotations, highlighting statistical tests, software tools, up-to-date databases, and functional annotations resources in the study of metazoan miRNAs.
| S-EPMC7563698 | biostudies-literature
Project description:BackgroundLong non-coding RNAs (lncRNAs) are typically expressed at low levels and are inherently highly variable. This is a fundamental challenge for differential expression (DE) analysis. In this study, the performance of 25 pipelines for testing DE in RNA-seq data is comprehensively evaluated, with a particular focus on lncRNAs and low-abundance mRNAs. Fifteen performance metrics are used to evaluate DE tools and normalization methods using simulations and analyses of six diverse RNA-seq datasets.ResultsGene expression data are simulated using non-parametric procedures in such a way that realistic levels of expression and variability are preserved in the simulated data. Throughout the assessment, results for mRNA and lncRNA were tracked separately. All the pipelines exhibit inferior performance for lncRNAs compared to mRNAs across all simulated scenarios and benchmark RNA-seq datasets. The substandard performance of DE tools for lncRNAs applies also to low-abundance mRNAs. No single tool uniformly outperformed the others. Variability, number of samples, and fraction of DE genes markedly influenced DE tool performance.ConclusionsOverall, linear modeling with empirical Bayes moderation (limma) and a non-parametric approach (SAMSeq) showed good control of the false discovery rate and reasonable sensitivity. Of note, for achieving a sensitivity of at least 50%, more than 80 samples are required when studying expression levels in realistic settings such as in clinical cancer research. About half of the methods showed a substantial excess of false discoveries, making these methods unreliable for DE analysis and jeopardizing reproducible science. The detailed results of our study can be consulted through a user-friendly web application, giving guidance on selection of the optimal DE tool ( http://statapps.ugent.be/tools/AppDGE/ ).
| S-EPMC6058388 | biostudies-literature
Project description:BackgroundRecent developments in deep (next-generation) sequencing technologies are significantly impacting medical research. The global analysis of protein coding regions in genomes of interest by whole exome sequencing is a widely used application. Many technologies for exome capture are commercially available; here we compare the performance of four of them: NimbleGen's SeqCap EZ v3.0, Agilent's SureSelect v4.0, Illumina's TruSeq Exome, and Illumina's Nextera Exome, all applied to the same human tumor DNA sample.ResultsEach capture technology was evaluated for its coverage of different exome databases, target coverage efficiency, GC bias, sensitivity in single nucleotide variant detection, sensitivity in small indel detection, and technical reproducibility. In general, all technologies performed well; however, our data demonstrated small, but consistent differences between the four capture technologies. Illumina technologies cover more bases in coding and untranslated regions. Furthermore, whereas most of the technologies provide reduced coverage in regions with low or high GC content, the Nextera technology tends to bias towards target regions with high GC content.ConclusionsWe show key differences in performance between the four technologies. Our data should help researchers who are planning exome sequencing to select appropriate exome capture technology for their particular application.
| S-EPMC4092227 | biostudies-literature