Dataset Information

A comparison framework and guideline of clustering methods for mass cytometry data.

ABSTRACT:

Background

With the expanding applications of mass cytometry in medical research, a wide variety of clustering methods, both semi-supervised and unsupervised, have been developed for data analysis. Selecting the optimal clustering method can accelerate the identification of meaningful cell populations.

Result

To address this issue, we compared three classes of performance measures, "precision" as external evaluation, "coherence" as internal evaluation, and stability, of nine methods based on six independent benchmark datasets. Seven unsupervised methods (Accense, Xshift, PhenoGraph, FlowSOM, flowMeans, DEPECHE, and kmeans) and two semi-supervised methods (Automated Cell-type Discovery and Classification and linear discriminant analysis (LDA)) are tested on six mass cytometry datasets. We compute and compare all defined performance measures against random subsampling, varying sample sizes, and the number of clusters for each method. LDA reproduces the manual labels most precisely but does not rank top in internal evaluation. PhenoGraph and FlowSOM perform better than other unsupervised tools in precision, coherence, and stability. PhenoGraph and Xshift are more robust when detecting refined sub-clusters, whereas DEPECHE and FlowSOM tend to group similar clusters into meta-clusters. The performances of PhenoGraph, Xshift, and flowMeans are impacted by increased sample size, but FlowSOM is relatively stable as sample size increases.

Conclusion

All the evaluations including precision, coherence, stability, and clustering resolution should be taken into synthetic consideration when choosing an appropriate tool for cytometry data analysis. Thus, we provide decision guidelines based on these characteristics for the general reader to more easily choose the most suitable clustering tools.

SUBMITTER: Liu X

PROVIDER: S-EPMC6929440 | biostudies-literature | 2019 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A comparison framework and guideline of clustering methods for mass cytometry data.

Liu Xiao X Song Weichen W Wong Brandon Y BY Zhang Ting T Yu Shunying S Lin Guan Ning GN Ding Xianting X

Genome biology 20191223 1

<h4>Background</h4>With the expanding applications of mass cytometry in medical research, a wide variety of clustering methods, both semi-supervised and unsupervised, have been developed for data analysis. Selecting the optimal clustering method can accelerate the identification of meaningful cell populations.<h4>Result</h4>To address this issue, we compared three classes of performance measures, "precision" as external evaluation, "coherence" as internal evaluation, and stability, of nine metho ...[more]

PMID: 31870419

Similar Datasets

Project description:BackgroundDNA microarrays, which determine the expression levels of tens of thousands of genes from a sample, are an important research tool. However, the volume of data they produce can be an obstacle to interpretation of the results. Clustering the genes on the basis of similarity of their expression profiles can simplify the data, and potentially provides an important source of biological inference, but these methods have not been tested systematically on datasets from complex human tissues. In this paper, four clustering methods, CRC, k-means, ISA and memISA, are used upon three brain expression datasets. The results are compared on speed, gene coverage and GO enrichment. The effects of combining the clusters produced by each method are also assessed.Resultsk-means outperforms the other methods, with 100% gene coverage and GO enrichments only slightly exceeded by memISA and ISA. Those two methods produce greater GO enrichments on the datasets used, but at the cost of much lower gene coverage, fewer clusters produced, and speed. The clusters they find are largely different to those produced by k-means. Combining clusters produced by k-means and memISA or ISA leads to increased GO enrichment and number of clusters produced (compared to k-means alone), without negatively impacting gene coverage. memISA can also find potentially disease-related clusters. In two independent dorsolateral prefrontal cortex datasets, it finds three overlapping clusters that are either enriched for genes associated with schizophrenia, genes differentially expressed in schizophrenia, or both. Two of these clusters are enriched for genes of the MAP kinase pathway, suggesting a possible role for this pathway in the aetiology of schizophrenia.ConclusionConsidered alone, k-means clustering is the most effective of the four methods on typical microarray brain expression datasets. However, memISA and ISA can add extra high-quality clusters to the set produced by k-means, so combining these three methods is the method of choice.

Project description:Analysis of imaging mass cytometry (IMC) data and other low-resolution multiplexed tissue imaging technologies is often confounded by poor single-cell segmentation and suboptimal approaches for data visualization and exploration. This can lead to inaccurate identification of cell phenotypes, states, or spatial relationships compared to reference data from single-cell suspension technologies. To this end we have developed the "OPTimized Imaging Mass cytometry AnaLysis (OPTIMAL)" framework to benchmark any approaches for cell segmentation, parameter transformation, batch effect correction, data visualization/clustering, and spatial neighborhood analysis. Using a panel of 27 metal-tagged antibodies recognizing well-characterized phenotypic and functional markers to stain the same Formalin-Fixed Paraffin Embedded (FFPE) human tonsil sample tissue microarray over 12 temporally distinct batches we tested several cell segmentation models, a range of different arcsinh cofactor parameter transformation values, 5 different dimensionality reduction algorithms, and 2 clustering methods. Finally, we assessed the optimal approach for performing neighborhood analysis. We found that single-cell segmentation was improved by the use of an Ilastik-derived probability map but that issues with poor segmentation were only really evident after clustering and cell type/state identification and not always evident when using "classical" bivariate data display techniques. The optimal arcsinh cofactor for parameter transformation was 1 as it maximized the statistical separation between negative and positive signal distributions and a simple Z-score normalization step after arcsinh transformation eliminated batch effects. Of the five different dimensionality reduction approaches tested, PacMap gave the best data structure with FLOWSOM clustering out-performing phenograph in terms of cell type identification. We also found that neighborhood analysis was influenced by the method used for finding neighboring cells with a "disc" pixel expansion outperforming a "bounding box" approach combined with the need for filtering objects based on size and image-edge location. Importantly, OPTIMAL can be used to assess and integrate with any existing approach to IMC data analysis and, as it creates .FCS files from the segmentation output and allows for single-cell exploration to be conducted using a wide variety of accessible software and algorithms familiar to conventional flow cytometrists.

Project description:The choice of the most appropriate unsupervised machine-learning method for "heterogeneous" or "mixed" data, i.e. with both continuous and categorical variables, can be challenging. Our aim was to examine the performance of various clustering strategies for mixed data using both simulated and real-life data. We conducted a benchmark analysis of "ready-to-use" tools in R comparing 4 model-based (Kamila algorithm, Latent Class Analysis, Latent Class Model [LCM] and Clustering by Mixture Modeling) and 5 distance/dissimilarity-based (Gower distance or Unsupervised Extra Trees dissimilarity followed by hierarchical clustering or Partitioning Around Medoids, K-prototypes) clustering methods. Clustering performances were assessed by Adjusted Rand Index (ARI) on 1000 generated virtual populations consisting of mixed variables using 7 scenarios with varying population sizes, number of clusters, number of continuous and categorical variables, proportions of relevant (non-noisy) variables and degree of variable relevance (low, mild, high). Clustering methods were then applied on the EPHESUS randomized clinical trial data (a heart failure trial evaluating the effect of eplerenone) allowing to illustrate the differences between different clustering techniques. The simulations revealed the dominance of K-prototypes, Kamila and LCM models over all other methods. Overall, methods using dissimilarity matrices in classical algorithms such as Partitioning Around Medoids and Hierarchical Clustering had a lower ARI compared to model-based methods in all scenarios. When applying clustering methods to a real-life clinical dataset, LCM showed promising results with regard to differences in (1) clinical profiles across clusters, (2) prognostic performance (highest C-index) and (3) identification of patient subgroups with substantial treatment benefit. The present findings suggest key differences in clustering performance between the tested algorithms (limited to tools readily available in R). In most of the tested scenarios, model-based methods (in particular the Kamila and LCM packages) and K-prototypes typically performed best in the setting of heterogeneous data.

Dataset Information

A comparison framework and guideline of clustering methods for mass cytometry data.

Background

Result

Conclusion

Publications

A comparison framework and guideline of clustering methods for mass cytometry data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets