Dataset Information

Characterization of uncertainty in the classification of multivariate assays: application to PAM50 centroid-based genomic predictors for breast cancer treatment plans.

ABSTRACT:

Background

Multivariate assays (MVAs) for assisting clinical decisions are becoming commonly available, but due to complexity, are often considered a high-risk approach. A key concern is that uncertainty on the assay's final results is not well understood. This study focuses on developing a process to characterize error introduced in the MVA's results from the intrinsic error in the laboratory process: sample preparation and measurement of the contributing factors, such as gene expression.

Methods

Using the PAM50 Breast Cancer Intrinsic Classifier, we show how to characterize error within an MVA, and how these errors may affect results reported to clinicians. First we estimated the error distribution for measured factors within the PAM50 assay by performing repeated measures on four archetypal samples representative of the major breast cancer tumor subtypes. Then, using the error distributions and the original archetypal sample data, we used Monte Carlo simulations to generate a sufficient number of simulated samples. The effect of these errors on the PAM50 tumor subtype classification was estimated by measuring subtype reproducibility after classifying all simulated samples. Subtype reproducibility was measured as the percentage of simulated samples classified identically to the parent sample. The simulation was thereafter repeated on a large, independent data set of samples from the GEICAM 9906 clinical trial. Simulated samples from the GEICAM sample set were used to explore a more realistic scenario where, unlike archetypal samples, many samples are not easily classified.

Results

All simulated samples derived from the archetypal samples were classified identically to the parent sample. Subtypes for simulated samples from the GEICAM set were also highly reproducible, but there were a non-negligible number of samples that exhibit significant variability in their classification.

Conclusions

We have developed a general methodology to estimate the effects of intrinsic errors within MVAs. We have applied the method to the PAM50 assay, showing that the PAM50 results are resilient to intrinsic errors within the assay, but also finding that in non-archetypal samples, experimental errors can lead to quite different classification of a tumor. Finally we propose a way to provide the uncertainty information in a usable way for clinicians.

SUBMITTER: Ebbert MT

PROVIDER: S-EPMC3275466 | biostudies-literature | 2011 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Characterization of uncertainty in the classification of multivariate assays: application to PAM50 centroid-based genomic predictors for breast cancer treatment plans.

Ebbert Mark Tw MT Bastien Roy Rl RR Boucher Kenneth M KM Martín Miguel M Carrasco Eva E Caballero Rosalía R Stijleman Inge J IJ Bernard Philip S PS Facelli Julio C JC

Journal of clinical bioinformatics 20111223

<h4>Background</h4>Multivariate assays (MVAs) for assisting clinical decisions are becoming commonly available, but due to complexity, are often considered a high-risk approach. A key concern is that uncertainty on the assay's final results is not well understood. This study focuses on developing a process to characterize error introduced in the MVA's results from the intrinsic error in the laboratory process: sample preparation and measurement of the contributing factors, such as gene expressio ...[more]

PMID: 22196354

Similar Datasets

Project description:MotivationGene expression-based multiclass prediction, such as tumor subtyping, is a non-trivial bioinformatic problem. Most classifier methods operate by comparing expression levels relative to other samples. Methods that base predictions on the expression pattern within a sample have been proposed as an alternative. As these methods are invariant to the cohort composition and can be applied to a sample in isolation, they can collectively be termed single sample predictors (SSP). Such predictors could potentially be used for preprocessing-free classification of new samples and be built to function across different expression platforms where proper batch and dataset normalization is challenging. Here we evaluate the behavior of several multiclass single sample predictors based on binary gene-pair rules (k-Top Scoring Pairs, Absolute Intrinsic Molecular Subtyping, and a new Random Forest approach) and compare them to centroids built with centered or raw expression values, with the criteria that an optimal predictor should have high accuracy, overcome differences in tumor purity, be robust across expression platforms, and provide an informative prediction output score.ResultsWe found that gene-pair based SSPs showed excellent performance on many expression-based classification tasks. The three methods differed in prediction score output, handling of tied scores, and behavior in low purity samples. The k-Top Scoring Pairs and Random Forest approach both achieved high classification accuracy while providing an informative prediction score. Although gene-pair-based SSPs have been touted as being cross-platform compatible (through training on mixed platform data), out-of-the-box compatibility with a new dataset remains a potential issue that warrants cohort-to-cohort verification.AvailabilityOur R package 'multiclassPairs' (https://cran.r-project.org/package=multiclassPairs) (https://doi.org/10.1093/bioinformatics/btab088) is freely available and enables easy training, prediction, and visualization using the gene-pair rule-based Random Forest SSP method and provides additional multiclass functionalities to the switchBox k-Top-Scoring Pairs package.Supplementary informationSupplementary data are available at Bioinformatics online.

Project description:Purpose: Despite advances in radical surgery and chemotherapy delivery, ovarian cancer is the most lethal gynecologic malignancy. Most of these patients are treated with platinum-based chemotherapies, but there is no biomarker model to guide their responses to these therapeutic agents. We have developed and independently tested our novel multivariate molecular predictors for forecasting patients' responses to individual drugs on a cohort of 58 ovarian cancer patients. Experimental Design: We adapted and applied the previously-published COXEN algorithm to develop molecular predictors for therapeutic responses of patients' tumors based on expression signatures derived from the NCI-60 in vitro drug activities and genomic expression data. Genome-wide candidate biomarkers were first triaged by examining expression patterns of frozen and formalin-fixed paraffin embedded (FFPE) tissue samples. We then identify initial drug sensitivity biomarkers for carboplatin and paclitaxel, respectively. These biomarkers were further narrowed by examining concordant expression patterns between cell lines and a historical set of ovarian cancer patients. Multivariate predictors were obtained from the NCI-60 cell lines and refined using historical patient cohorts. To independent validate these molecular predictors, we performed genome-wide profiling on FFPE samples of 58 ovarian cancer patients obtained prior to adjuvant chemotherapy. Results: Carboplatin predictor significantly stratified platinum sensitive and resistant patients (p = 0.019) with sensitivity = 93%, specificity = 33%, PPV = 65%, and NPV = 78%. Paclitaxel predictor also significantly stratified patients' responses (p = 0.033) with sensitivity = 96%, specificity = 26%, PPV = 61%, and NPV = 86%. The combination predictor for platinum-taxane combination demonstrated a significant survival difference between the predicted responders and nonresponders with median survival of 12.9 months vs. 8.1 months (p = 0.045). Conclusions: COXEN predictors successfully stratified platinum resistance and taxane response in this retrospective cohort, especially based on their FFPE tumor samples. Accurate prediction of chemotherapeutic response, especially to platinum agents is highly clinically relevant and could alter primary management of ovarian cancer.

Project description:BackgroundTwo prostate cancer (PC) classification methods based on transcriptome profiles, a de novo method referred to as the "Prostate Cancer Classification System" (PCS) and a variation of the established PAM50 breast cancer algorithm, were recently proposed. Both studies concluded that most human PC can be assigned to one of three tumor subtypes, two categorized as luminal and one as basal, suggesting the two methods reflect consistency in underlying biology. Despite the similarity, differences and commonalities between the two classification methods have not yet been reported.MethodsHere, we describe a comparison of the PCS and PAM50 classification systems. PCS and PAM50 signatures consisting of 37 (PCS37) and 50 genes, respectively, were used to categorize 9,947 PC patients into PCS and PAM50 classes. Enrichment of hallmark gene sets and luminal and basal marker gene expression were assessed in the same datasets. Finally, survival analysis was performed to compare PCS and PAM50 subtypes in terms of clinical outcomes.ResultsPCS and PAM50 subtypes show clear differential expression of PCS37 and PAM50 genes. While only three genes are shared in common between the two systems, there is some consensus between three subtype pairs (PCS1 versus Luminal B, PCS2 versus Luminal A, and PCS3 versus Basal) with respect to gene expression, cellular processes, and clinical outcomes. PCS categories displayed better separation of cellular processes and luminal and basal marker gene expression compared to PAM50. Although both PCS1 and Luminal B tumors exhibited the worst clinical outcomes, outcomes between aggressive and less aggressive subtypes were better defined in the PCS system, based on larger hazard ratios observed.ConclusionThe PCS and PAM50 classification systems are similar in terms of molecular profiles and clinical outcomes. However, the PCS system exhibits greater separation in multiple clinical outcomes and provides better separation of prostate luminal and basal characteristics.

Project description:BackgroundTaxonomic profiles of vaginal microbial communities can be sorted into a discrete number of categories termed community state types (CSTs). This approach is advantageous because collapsing a hyper-dimensional taxonomic profile into a single categorical variable enables efforts such as data exploration, epidemiological studies, and statistical modeling. Vaginal communities are typically assigned to CSTs based on the results of hierarchical clustering of the pairwise distances between samples. However, this approach is problematic because it complicates between-study comparisons and because the results are entirely dependent on the particular set of samples that were analyzed. We sought to standardize and advance the assignment of samples to CSTs.ResultsWe developed VALENCIA (VAginaL community state typE Nearest CentroId clAssifier), a nearest centroid-based tool which classifies samples based on their similarity to a set of reference centroids. The references were defined using a comprehensive set of 13,160 taxonomic profiles from 1975 women in the USA. This large dataset allowed us to comprehensively identify, define, and characterize vaginal CSTs common to reproductive age women and expand upon the CSTs that had been defined in previous studies. We validated the broad applicability of VALENCIA for the classification of vaginal microbial communities by using it to classify three test datasets which included reproductive age eastern and southern African women, adolescent girls, and a racially/ethnically and geographically diverse sample of postmenopausal women. VALENCIA performed well on all three datasets despite the substantial variations in sequencing strategies and bioinformatics pipelines, indicating its broad application to vaginal microbiota. We further describe the relationships between community characteristics (vaginal pH, Nugent score) and participant demographics (race, age) and the CSTs defined by VALENCIA.ConclusionVALENCIA provides a much-needed solution for the robust and reproducible assignment of vaginal community state types. This will allow unbiased analysis of both small and large vaginal microbiota datasets, comparisons between datasets and meta-analyses that combine multiple datasets. Video abstract.

Dataset Information

Characterization of uncertainty in the classification of multivariate assays: application to PAM50 centroid-based genomic predictors for breast cancer treatment plans.

Background

Methods

Results

Conclusions

Publications

Characterization of uncertainty in the classification of multivariate assays: application to PAM50 centroid-based genomic predictors for breast cancer treatment plans.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets