Dataset Information

Evaluation of connectivity map shows limited reproducibility in drug repositioning.

ABSTRACT: The Connectivity Map (CMap) is a popular resource designed for data-driven drug repositioning using a large transcriptomic compendium. However, evaluations of its performance are limited. We used two iterations of CMap (CMap 1 and 2) to assess their comparability and reliability. We queried CMap 2 with CMap 1-derived signatures, expecting CMap 2 would highly prioritize the queried compounds; the success rate was 17%. Analysis of previously published prioritizations yielded similar results. Low recall is caused by low differential expression (DE) reproducibility both between CMaps and within each CMap. DE strength was predictive of reproducibility, and is influenced by compound concentration and cell-line responsiveness. Reproducibility of CMap 2 sample expression levels was also lower than expected. We attempted to identify the "better" CMap by comparison with a third dataset, but they were mutually discordant. Our findings have implications for CMap usage and we suggest steps for investigators to limit false positives.

SUBMITTER: Lim N

PROVIDER: S-EPMC8413422 | biostudies-literature | 2021 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluation of connectivity map shows limited reproducibility in drug repositioning.

Lim Nathaniel N Pavlidis Paul P

Scientific reports 20210902 1

The Connectivity Map (CMap) is a popular resource designed for data-driven drug repositioning using a large transcriptomic compendium. However, evaluations of its performance are limited. We used two iterations of CMap (CMap 1 and 2) to assess their comparability and reliability. We queried CMap 2 with CMap 1-derived signatures, expecting CMap 2 would highly prioritize the queried compounds; the success rate was 17%. Analysis of previously published prioritizations yielded similar results. Low r ...[more]

PMID: 34475469

Similar Datasets

Project description:BackgroundDrug repositioning is a cost-efficient and time-saving process to drug development compared to traditional techniques. A systematic method to drug repositioning is to identify candidate drug's gene expression profiles on target disease models and determine how similar these profiles are to approved drugs. Databases such as the CMAP have been developed recently to help with systematic drug repositioning.MethodsTo overcome the limitation of connectivity maps on data coverage, we constructed a comprehensive in silico drug-protein connectivity map called DMAP, which contains directed drug-to-protein effects and effect scores. The drug-to-protein effect scores are compiled from all database entries between the drug and protein have been previously observed and provide a confidence measure on the quality of such drug-to-protein effects.ResultsIn DMAP, we have compiled the direct effects between 24,121 PubChem Compound ID (CID), which were mapped from 289,571 chemical entities recognized from public literature, and 5,196 reviewed Uniprot proteins. DMAP compiles a total of 438,004 chemical-to-protein effect relationships. Compared to CMAP, DMAP shows an increase of 221 folds in the number of chemicals and 1.92 fold in the number of ATC codes. Furthermore, by overlapping DMAP chemicals with the approved drugs with known indications from the TTD database and literature, we obtained 982 drugs and 622 diseases; meanwhile, we only obtained 394 drugs with known indication from CMAP. To validate the feasibility of applying new DMAP for systematic drug repositioning, we compared the performance of DMAP and the well-known CMAP database on two popular computational techniques: drug-drug-similarity-based method with leave-one-out validation and Kolmogorov-Smirnov scoring based method. In drug-drug-similarity-based method, the drug repositioning prediction using DMAP achieved an Area-Under-Curve (AUC) score of 0.82, compared with that using CMAP, AUC = 0.64. For Kolmogorov-Smirnov scoring based method, with DMAP, we were able to retrieve several drug indications which could not be retrieved using CMAP. DMAP data can be queried using the existing C2MAP server or downloaded freely at: http://bio.informatics.iupui.edu/cmapsConclusionsReliable measurements of how drug affect disease-related proteins are critical to ongoing drug development in the genome medicine era. We demonstrated that DMAP can help drug development professionals assess drug-to-protein relationship data and improve chances of success for systematic drug repositioning efforts.

Dataset Information

Evaluation of connectivity map shows limited reproducibility in drug repositioning.

Publications

Evaluation of connectivity map shows limited reproducibility in drug repositioning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets