Dataset Information

Improving deconvolution methods in biology through open innovation competitions: an application to the connectivity map.

ABSTRACT:

Motivation

Do machine learning methods improve standard deconvolution techniques for gene expression data? This article uses a unique new dataset combined with an open innovation competition to evaluate a wide range of approaches developed by 294 competitors from 20 countries. The competition's objective was to address a deconvolution problem critical to analyzing genetic perturbations from the Connectivity Map. The issue consists of separating gene expression of individual genes from raw measurements obtained from gene pairs. We evaluated the outcomes using ground-truth data (direct measurements for single genes) obtained from the same samples.

Results

We find that the top-ranked algorithm, based on random forest regression, beat the other methods in accuracy and reproducibility; more traditional gaussian-mixture methods performed well and tended to be faster, and the best deep learning approach yielded outcomes slightly inferior to the above methods. We anticipate researchers in the field will find the dataset and algorithms developed in this study to be a powerful research tool for benchmarking their deconvolution methods and a resource useful for multiple applications. Availability and implementation: The data is freely available at clue.io/data (section Contests) and the software is on GitHub at https://github.com/cmap/gene_deconvolution_challenge.

Supplementary information

Supplementary data are available at Bioinformatics online.

SUBMITTER: Blasco A

PROVIDER: S-EPMC8479655 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundDigital innovations in health care have traditionally followed a top-down pathway, with manufacturers leading the design and production of technology-enabled solutions and those living with chronic conditions involved only as passive recipients of the end product. However, user-driven open-source initiatives in health care are becoming increasingly popular. An example is the growing movement of people with diabetes, who create their own "Do-It-Yourself Artificial Pancreas Systems" (DIYAPS).ObjectiveThe overall aim of this study is to establish the empirical evidence base for the clinical effectiveness and quality-of-life benefits of DIYAPS and identify the challenges and possible solutions to enable their wider diffusion.MethodsA research program comprising 5 work packages will examine the outcomes and potential for scaling up DIYAPS solutions. Quantitative and qualitative methodologies will be used to examine clinical and self-reported outcome measures of DIYAPS users. The majority of members of the research team live with type 1 diabetes and are active DIYAPS users, making Outcomes of Patients' Evidence With Novel, Do-It-Yourself Artificial Pancreas Technology (OPEN) a unique, user-driven research project.ResultsThis project has received funding from the European Commission's Horizon 2020 Research and Innovation Program, under the Marie Skłodowska-Curie Action Research and Innovation Staff Exchange. Researchers with both academic and nonacademic backgrounds have been recruited to formulate research questions, drive the research process, and disseminate ongoing findings back to the DIYAPS community and other stakeholders.ConclusionsThe OPEN project is unique in that it is a truly patient- and user-led research project, which brings together an international, interdisciplinary, and intersectoral research group, comprising health care professionals, technical developers, biomedical and social scientists, the majority of whom are also living with diabetes. Thus, it directly addresses the core research and user needs of the DIYAPS movement. As a new model of cooperation, it will highlight how researchers in academia, industry, and the patient community can create patient-centric innovation and reduce disease burden together.International registered report identifier (irrid)PRR1-10.2196/15368.

Project description:BackgroundConfounding due to cellular heterogeneity represents one of the foremost challenges currently facing Epigenome-Wide Association Studies (EWAS). Statistical methods leveraging the tissue-specificity of DNA methylation for deconvoluting the cellular mixture of heterogenous biospecimens offer a promising solution, however the performance of such methods depends entirely on the library of methylation markers being used for deconvolution. Here, we introduce a novel algorithm for Identifying Optimal Libraries (IDOL) that dynamically scans a candidate set of cell-specific methylation markers to find libraries that optimize the accuracy of cell fraction estimates obtained from cell mixture deconvolution.ResultsApplication of IDOL to training set consisting of samples with both whole-blood DNA methylation data (Illumina HumanMethylation450 BeadArray (HM450)) and flow cytometry measurements of cell composition revealed an optimized library comprised of 300 CpG sites. When compared existing libraries, the library identified by IDOL demonstrated significantly better overall discrimination of the entire immune cell landscape (p = 0.038), and resulted in improved discrimination of 14 out of the 15 pairs of leukocyte subtypes. Estimates of cell composition across the samples in the training set using the IDOL library were highly correlated with their respective flow cytometry measurements, with all cell-specific R (2)>0.99 and root mean square errors (RMSEs) ranging from [0.97 % to 1.33 %] across leukocyte subtypes. Independent validation of the optimized IDOL library using two additional HM450 data sets showed similarly strong prediction performance, with all cell-specific R (2)>0.90 and R M S E<4.00 %. In simulation studies, adjustments for cell composition using the IDOL library resulted in uniformly lower false positive rates compared to competing libraries, while also demonstrating an improved capacity to explain epigenome-wide variation in DNA methylation within two large publicly available HM450 data sets.ConclusionsDespite consisting of half as many CpGs compared to existing libraries for whole blood mixture deconvolution, the optimized IDOL library identified herein resulted in outstanding prediction performance across all considered data sets and demonstrated potential to improve the operating characteristics of EWAS involving adjustments for cell distribution. In addition to providing the EWAS community with an optimized library for whole blood mixture deconvolution, our work establishes a systematic and generalizable framework for the assembly of libraries that improve the accuracy of cell mixture deconvolution.