Dataset Information

Stability of methods for differential expression analysis of RNA-seq data.

ABSTRACT:

Background

As RNA-seq becomes the assay of choice for measuring gene expression levels, differential expression analysis has received extensive attentions of researchers. To date, for the evaluation of DE methods, most attention has been paid on validity. Yet another important aspect of DE methods, stability, is overlooked and has not been studied to the best of our knowledge.

Results

In this study, we empirically show the need of assessing stability of DE methods and propose a stability metric, called Area Under the Correlation curve (AUCOR), that generates the perturbed datasets by a mixture distribution and combines the information of similarities between sets of selected features from these perturbed datasets and the original dataset.

Conclusion

Empirical results support that AUCOR can effectively rank the DE methods in terms of stability for given RNA-seq datasets. In addition, we explore how biological or technical factors from experiments and data analysis affect the stability of DE methods. AUCOR is implemented in the open-source R package AUCOR, with source code freely available at https://github.com/linbingqing/stableDE .

SUBMITTER: Lin B

PROVIDER: S-EPMC6330476 | biostudies-literature | 2019 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Stability of methods for differential expression analysis of RNA-seq data.

Lin Bingqing B Pang Zhen Z

BMC genomics 20190111 1

<h4>Background</h4>As RNA-seq becomes the assay of choice for measuring gene expression levels, differential expression analysis has received extensive attentions of researchers. To date, for the evaluation of DE methods, most attention has been paid on validity. Yet another important aspect of DE methods, stability, is overlooked and has not been studied to the best of our knowledge.<h4>Results</h4>In this study, we empirically show the need of assessing stability of DE methods and propose a st ...[more]

PMID: 30634899

Similar Datasets

Project description:BackgroundSingle-cell RNA Sequencing is gaining popularity in recent years. Compared to bulk RNA-Seq, single-cell RNA Sequencing allows the gene expression being measured within individual cells instead of mean gene expression levels across all cells in the sample. Thus, cell-to-cell variation of gene expressions could be examined. Gene differential expression analysis remains the major purpose in most single-cell RNA sequencing experiments and many methods have been developed in recent years to conduct gene differential expression analysis for single-cell RNA sequencing data.ResultsThrough simulation studies and real data examples, we evaluated the performance of five open-source popular methods used for gene differential expression analysis in single-cell RNA sequencing data. The five methods included DEsingle (Zero-inflated negative binomial model), Linnorm (Empirical Bayes method on transformed count data using the limma package), monocle (An approximate Chi-Square likelihood ratio test), MAST (A generalized linear hurdle model), and DESeq2 (A generalized linear model with empirical Bayes approach and also commonly used for bulk RNA sequencing differential express analyses). We assessed the false discovery rate (FDR) control, sensitivity, specificity, accuracy, and area under the receiver operating characteristics (AUROC) curve for all five methods under different sample sizes, distribution assumptions, and proportions of zeros in the data.ConclusionsWe found the MAST method performed the best among the five methods compared with the largest AUROC values across all tested sample sizes and different proportion of truly differential expressed genes, when the data followed negative binomial distributions. When the sample size increased to 100 in each group, the MAST method showed the best performance with the highest AUROC regardless of the data distributions. If the excess zeros were first filtered out before the gene differential analyses, the DESingle, Linnorm, and DESeq2 performed relatively better than the MAST and the monocle methods with higher AUROC values.

Project description:RNA-seq is a high-throughput sequencing technology widely used for gene transcript discovery and quantification under different biological or biomedical conditions. A fundamental research question in most RNA-seq experiments is the identification of differentially expressed genes among experimental conditions or sample groups. Numerous statistical methods for RNA-seq differential analysis have been proposed since the emergence of the RNA-seq assay. To evaluate popular differential analysis methods used in the open source R and Bioconductor packages, we conducted multiple simulation studies to compare the performance of eight RNA-seq differential analysis methods used in RNA-seq data analysis (edgeR, DESeq, DESeq2, baySeq, EBSeq, NOISeq, SAMSeq, Voom). The comparisons were across different scenarios with either equal or unequal library sizes, different distribution assumptions and sample sizes. We measured performance using false discovery rate (FDR) control, power, and stability. No significant differences were observed for FDR control, power, or stability across methods, whether with equal or unequal library sizes. For RNA-seq count data with negative binomial distribution, when sample size is 3 in each group, EBSeq performed better than the other methods as indicated by FDR control, power, and stability. When sample sizes increase to 6 or 12 in each group, DESeq2 performed slightly better than other methods. All methods have improved performance when sample size increases to 12 in each group except DESeq. For RNA-seq count data with log-normal distribution, both DESeq and DESeq2 methods performed better than other methods in terms of FDR control, power, and stability across all sample sizes. Real RNA-seq experimental data were also used to compare the total number of discoveries and stability of discoveries for each method. For RNA-seq data analysis, the EBSeq method is recommended for studies with sample size as small as 3 in each group, and the DESeq2 method is recommended for sample size of 6 or higher in each group when the data follow the negative binomial distribution. Both DESeq and DESeq2 methods are recommended when the data follow the log-normal distribution.

Dataset Information

Stability of methods for differential expression analysis of RNA-seq data.

Background

Results

Conclusion

Publications

Stability of methods for differential expression analysis of RNA-seq data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets