Dataset Information

Effects of sample size on robustness and prediction accuracy of a prognostic gene signature.

ABSTRACT:

Background

Few overlap between independently developed gene signatures and poor inter-study applicability of gene signatures are two of major concerns raised in the development of microarray-based prognostic gene signatures. One recent study suggested that thousands of samples are needed to generate a robust prognostic gene signature.

Results

A data set of 1,372 samples was generated by combining eight breast cancer gene expression data sets produced using the same microarray platform and, using the data set, effects of varying samples sizes on a few performances of a prognostic gene signature were investigated. The overlap between independently developed gene signatures was increased linearly with more samples, attaining an average overlap of 16.56% with 600 samples. The concordance between predicted outcomes by different gene signatures also was increased with more samples up to 94.61% with 300 samples. The accuracy of outcome prediction also increased with more samples. Finally, analysis using only Estrogen Receptor-positive (ER+) patients attained higher prediction accuracy than using both patients, suggesting that sub-type specific analysis can lead to the development of better prognostic gene signatures

Conclusion

Increasing sample sizes generated a gene signature with better stability, better concordance in outcome prediction, and better prediction accuracy. However, the degree of performance improvement by the increased sample size was different between the degree of overlap and the degree of concordance in outcome prediction, suggesting that the sample size required for a study should be determined according to the specific aims of the study.

SUBMITTER: Kim SY

PROVIDER: S-EPMC2689196 | biostudies-literature | 2009 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Effects of sample size on robustness and prediction accuracy of a prognostic gene signature.

Kim Seon-Young SY

BMC bioinformatics 20090516

<h4>Background</h4>Few overlap between independently developed gene signatures and poor inter-study applicability of gene signatures are two of major concerns raised in the development of microarray-based prognostic gene signatures. One recent study suggested that thousands of samples are needed to generate a robust prognostic gene signature.<h4>Results</h4>A data set of 1,372 samples was generated by combining eight breast cancer gene expression data sets produced using the same microarray plat ...[more]

PMID: 19445687

Similar Datasets

Project description:Background. National valuation studies are costly, with ∼1000 face-to-face interviews recommended, and some countries may deem such studies infeasible. Building on previous studies exploring sample size, we determined the effect of sample size and alternative model specifications on prediction accuracy of modeled coefficients in EQ-5D-5L value set generating regression analyses. Methods. Data sets (n = 50 to ∼1000) were simulated from 3 valuation studies, resampled at the respondent level and randomly drawn 1000 times with replacement. We estimated utilities for each subsample with leave-one-out at the block level using regression models (8 or 20 parameter; with or without a random intercept; time tradeoff [TTO] data only or TTO + discrete choice experiment [DCE] data). Prediction accuracy, root mean square error (RMSE), was calculated by comparing to censored mean predicted values to the left-out block in the full data set. Linear regression was used to estimate the relative effect of changes in sample size and each model specification. Results. Results showed that doubling the sample size decreased RMSE by on average 0.012. Effects of other model specifications were smaller but can when combined compensate for loss in prediction accuracy from a small sample size. For models using TTO data only, 8-parameter models clearly outperformed 20-parameter models. Adding a random intercept, or including DCE responses, also improved mean RMSE, most prominently for variants of the 20-parameter models. Conclusions. The prediction accuracy impact of further increases in sample size after 300 to 500 were smaller than the impact of combining alternative modeling choices. Hybrid modeling, use of constrained models, and inclusion of random intercepts all substantially improve the expected prediction accuracy. Beyond a minimum of 300 to 500 respondents, the sample size may be better informed by other considerations, such as legitimacy and representativeness, than by the technical prediction accuracy achievable.HighlightsIncreases in sample size beyond a minimum in the range of 300 to 500 respondents provide smaller gains in expected prediction accuracy than alternative modeling approaches.Constrained, nonlinear models; time tradeoff + discrete choice experiment hybrid modeling; and including a random intercept all improved the prediction accuracy of models estimating values for the EQ-5D-5L based on data from 3 different valuation studies.The tested modeling choices can compensate for smaller sample sizes.

Project description:The aim of the present study was to examine the molecular factors associated with the prognosis of colon cancer. Gene expression datasets were downloaded from The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus databases to screen differentially expressed genes (DEGs) between colon cancer samples and normal samples. Survival?related genes were selected from the DEGs using the Cox regression method. A co?expression network of survival?related genes was then constructed, and functional clusters were extracted from this network. The significantly enriched functions and pathways of the genes in the network were identified. Using Bayesian discriminant analysis, a prognostic prediction system was established to distinguish the positive from negative prognostic samples. The discrimination efficacy of the system was validated in the GSE17538 dataset using Kaplan?Meier survival analysis. A total of 636 and 1,892 DEGs between the colon cancer samples and normal samples were screened from the TCGA and GSE44861 dataset, respectively. There were 155 survival?related genes selected. The co?expression network of survival?related genes included 138 genes, 534 lines (connections) and five functional clusters, including the signaling pathway, cellular response to cAMP, and immune system process functional clusters. The molecular function, cellular components and biological processes were the significantly enriched functions. The peroxisome proliferator?activated receptor signaling pathway, Wnt signaling pathway, B cell receptor signaling pathway, and cytokine?cytokine receptor interactions were the significant pathways. A prognostic prediction system based on a 65?gene signature was established using this co?expression network. Its discriminatory effect was validated in the TCGA dataset (P=3.56e?12) and the GSE17538 dataset (P=1.67e?6). The 65?gene signature included kallikrein?related peptidase 6 (KLK6), collagen type XI ?1 (COL11A1), cartilage oligomeric matrix protein, wingless?type MMTV integration site family member 2 (WNT2) and keratin 6B. In conclusion, a 65?gene signature was screened in the present study, which showed a prognostic prediction effect in colon adenocarcinoma. KLK6, COL11A1, and WNT2 may be suitable prognostic predictors for colon adenocarcinoma.

Project description:BackgroundGrowing evidence has revealed the crucial roles of stromal cells in the microenvironment of various malignant tumors. However, efficient prognostic signatures based on stromal characteristics in colon cancer have not been well-established yet. The present study aimed to construct a stromal score-based multigene prognostic prediction model for colon cancer.MethodsStromal scores were calculated based on the expression profiles of a colon cancer cohort from TCGA database applying the ESTIMATE algorithm. Linear models were used to identify differentially expressed genes between low-score and high-score groups by limma R package. Univariate, LASSO, and multivariate Cox regression models were used successively to select the prognostic gene signature. Two independent datasets from GEO were used as external validation cohorts.ResultsLow stromal score was demonstrated to be a favorable factor to the overall survival of colon cancer patients in TCGA cohort (p = 0.0046). Three hundred and seven stromal score-related differentially expressed genes were identified. Through univariate, LASSO, and multivariate Cox regression analyses, a gene signature consisting of LEP, NOG, and SYT3 was recognized to build a prognostic prediction model. Based on the predictive values estimated by the established integrated model, patients were divided into two groups with significantly different overall survival outcomes (p < 0.0001). Time-dependent Receiver operating characteristic curve analyses suggested the satisfactory predictive efficacy for the 5-year overall survival of the model (AUC value = 0.733). A nomogram with great predictive performance combining the multigene prediction model and clinicopathological factors was developed. The established model was validated in an external cohort (AUC value = 0.728). In another independent cohort, the model was verified to be of significant prognostic value for different subgroups, which was demonstrated to be especially accurate for young patients (AUC value = 0.763).ConclusionThe well-established model based on stromal score-related gene signature might serve as a promising tool for the prognostic prediction of colon cancer.

Dataset Information

Effects of sample size on robustness and prediction accuracy of a prognostic gene signature.

Background

Results

Conclusion

Publications

Effects of sample size on robustness and prediction accuracy of a prognostic gene signature.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets