Dataset Information

Effects of sample size on differential gene expression, rank order and prediction accuracy of a gene signature.

ABSTRACT: Top differentially expressed gene lists are often inconsistent between studies and it has been suggested that small sample sizes contribute to lack of reproducibility and poor prediction accuracy in discriminative models. We considered sex differences (69?, 65 ?) in 134 human skeletal muscle biopsies using DNA microarray. The full dataset and subsamples (n = 10 (5 ?, 5 ?) to n = 120 (60 ?, 60 ?)) thereof were used to assess the effect of sample size on the differential expression of single genes, gene rank order and prediction accuracy. Using our full dataset (n = 134), we identified 717 differentially expressed transcripts (p<0.0001) and we were able predict sex with ~90% accuracy, both within our dataset and on external datasets. Both p-values and rank order of top differentially expressed genes became more variable using smaller subsamples. For example, at n = 10 (5 ?, 5 ?), no gene was considered differentially expressed at p<0.0001 and prediction accuracy was ~50% (no better than chance). We found that sample size clearly affects microarray analysis results; small sample sizes result in unstable gene lists and poor prediction accuracy. We anticipate this will apply to other phenotypes, in addition to sex.

SUBMITTER: Stretch C

PROVIDER: S-EPMC3670871 | biostudies-literature | 2013

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Effects of sample size on differential gene expression, rank order and prediction accuracy of a gene signature.

Stretch Cynthia C Khan Sheehan S Asgarian Nasimeh N Eisner Roman R Vaisipour Saman S Damaraju Sambasivarao S Graham Kathryn K Bathe Oliver F OF Steed Helen H Greiner Russell R Baracos Vickie E VE

PloS one 20130603 6

Top differentially expressed gene lists are often inconsistent between studies and it has been suggested that small sample sizes contribute to lack of reproducibility and poor prediction accuracy in discriminative models. We considered sex differences (69♂, 65 ♀) in 134 human skeletal muscle biopsies using DNA microarray. The full dataset and subsamples (n = 10 (5 ♂, 5 ♀) to n = 120 (60 ♂, 60 ♀)) thereof were used to assess the effect of sample size on the differential expression of single genes ...[more]

PMID: 23755224

Similar Datasets

Project description:Background. National valuation studies are costly, with ∼1000 face-to-face interviews recommended, and some countries may deem such studies infeasible. Building on previous studies exploring sample size, we determined the effect of sample size and alternative model specifications on prediction accuracy of modeled coefficients in EQ-5D-5L value set generating regression analyses. Methods. Data sets (n = 50 to ∼1000) were simulated from 3 valuation studies, resampled at the respondent level and randomly drawn 1000 times with replacement. We estimated utilities for each subsample with leave-one-out at the block level using regression models (8 or 20 parameter; with or without a random intercept; time tradeoff [TTO] data only or TTO + discrete choice experiment [DCE] data). Prediction accuracy, root mean square error (RMSE), was calculated by comparing to censored mean predicted values to the left-out block in the full data set. Linear regression was used to estimate the relative effect of changes in sample size and each model specification. Results. Results showed that doubling the sample size decreased RMSE by on average 0.012. Effects of other model specifications were smaller but can when combined compensate for loss in prediction accuracy from a small sample size. For models using TTO data only, 8-parameter models clearly outperformed 20-parameter models. Adding a random intercept, or including DCE responses, also improved mean RMSE, most prominently for variants of the 20-parameter models. Conclusions. The prediction accuracy impact of further increases in sample size after 300 to 500 were smaller than the impact of combining alternative modeling choices. Hybrid modeling, use of constrained models, and inclusion of random intercepts all substantially improve the expected prediction accuracy. Beyond a minimum of 300 to 500 respondents, the sample size may be better informed by other considerations, such as legitimacy and representativeness, than by the technical prediction accuracy achievable.HighlightsIncreases in sample size beyond a minimum in the range of 300 to 500 respondents provide smaller gains in expected prediction accuracy than alternative modeling approaches.Constrained, nonlinear models; time tradeoff + discrete choice experiment hybrid modeling; and including a random intercept all improved the prediction accuracy of models estimating values for the EQ-5D-5L based on data from 3 different valuation studies.The tested modeling choices can compensate for smaller sample sizes.

Dataset Information

Effects of sample size on differential gene expression, rank order and prediction accuracy of a gene signature.

Publications

Effects of sample size on differential gene expression, rank order and prediction accuracy of a gene signature.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets