CavBench: A benchmark for protein cavity detection methods.
Ontology highlight
ABSTRACT:
SUBMITTER: Dias S
PROVIDER: S-EPMC6791542 | biostudies-literature | 2019
REPOSITORIES: biostudies-literature
Publications
CavBench: A benchmark for protein cavity detection methods.
Dias Sérgio S Simões Tiago T Fernandes Francisco F Martins Ana Mafalda AM Ferreira Alfredo A Jorge Joaquim J Gomes Abel J P AJP
PloS one 20191014 10
Extensive research has been applied to discover new techniques and methods to model protein-ligand interactions. In particular, considerable efforts focused on identifying candidate binding sites, which quite often are active sites that correspond to protein pockets or cavities. Thus, these cavities play an important role in molecular docking. However, there is no established benchmark to assess the accuracy of new cavity detection methods. In practice, each new technique is evaluated using a sm ...[more]
Similar Datasets
Project description:With the development of many computational methods that predict the structural models of protein-protein complexes, there is a pressing need to benchmark their performance. As was the case for protein monomers, assessing the quality of models of protein complexes is not straightforward. An effective scoring scheme should be able to detect substructure similarity and estimate its statistical significance. Here, we focus on characterizing the similarity of the interfaces of the complex and introduce two scoring functions. The first, the interfacial Template Modeling score (iTM-score), measures the geometric distance between the interfaces, while the second, the Interface Similarity score (IS-score), evaluates their residue-residue contact similarity in addition to their geometric similarity. We first demonstrate that the IS-score is more suitable for assessing docking models than the iTM-score. The IS-score is then validated in a large-scale benchmark test on 1562 dimeric complexes. Finally, the scoring function is applied to evaluate docking models submitted to the Critical Assessment of Prediction of Interactions (CAPRI) experiments. While the results according to the new scoring scheme are generally consistent with the original CAPRI assessment, the IS-score identifies models whose significance was previously underestimated.
| S-EPMC3076516 | biostudies-literature
Project description:Horizontal gene transfer (HGT) has appeared to be of importance for prokaryotic species evolution. As a consequence numerous parametric methods, using only the information embedded in the genomes, have been designed to detect HGTs. Numerous reports of incongruencies in results of the different methods applied to the same genomes were published. The use of artificial genomes in which all HGT parameters are controlled allows testing different methods in the same conditions. The results of this benchmark concerning 16 representative parametric methods showed a great variety of efficiencies. Some methods work very poorly whatever the type of HGTs and some depend on the conditions or on the metrics used. The best methods in terms of total errors were those using tetranucleotides as criterion for the window methods or those using codon usage for gene based methods and the Kullback-Leibler divergence metric. Window methods are very sensitive but less specific and detect badly lone isolated gene. On the other hand gene based methods are often very specific but lack of sensitivity. We propose using two methods in combination to get the best of each category, a gene based one for specificity and a window based one for sensitivity.
| S-EPMC2848678 | biostudies-literature
Project description:BackgroundProtein sequence alignment analyses have become a crucial step for many bioinformatics studies during the past decades. Multiple sequence alignment (MSA) and pair-wise sequence alignment (PSA) are two major approaches in sequence alignment. Former benchmark studies revealed drawbacks of MSA methods on nucleotide sequence alignments. To test whether similar drawbacks also influence protein sequence alignment analyses, we propose a new benchmark framework for protein clustering based on cluster validity. This new framework directly reflects the biological ground truth of the application scenarios that adopt sequence alignments, and evaluates the alignment quality according to the achievement of the biological goal, rather than the comparison on sequence level only, which averts the biases introduced by alignment scores or manual alignment templates. Compared with former studies, we calculate the cluster validity score based on sequence distances instead of clustering results. This strategy could avoid the influence brought by different clustering methods thus make results more dependable.ResultsResults showed that PSA methods performed better than MSA methods on most of the BAliBASE benchmark datasets. Analyses on the 80 re-sampled benchmark datasets constructed by randomly choosing 90% of each dataset 10 times showed similar results.ConclusionsThese results validated that the drawbacks of MSA methods revealed in nucleotide level also existed in protein sequence alignment analyses and affect the accuracy of results.
| S-EPMC6311937 | biostudies-literature
Project description:The most important way of conveying new findings in biomedical research is scientific publication. Extraction of protein-protein interactions (PPIs) reported in scientific publications is one of the core topics of text mining in the life sciences. Recently, a new class of such methods has been proposed - convolution kernels that identify PPIs using deep parses of sentences. However, comparing published results of different PPI extraction methods is impossible due to the use of different evaluation corpora, different evaluation metrics, different tuning procedures, etc. In this paper, we study whether the reported performance metrics are robust across different corpora and learning settings and whether the use of deep parsing actually leads to an increase in extraction quality. Our ultimate goal is to identify the one method that performs best in real-life scenarios, where information extraction is performed on unseen text and not on specifically prepared evaluation data. We performed a comprehensive benchmarking of nine different methods for PPI extraction that use convolution kernels on rich linguistic information. Methods were evaluated on five different public corpora using cross-validation, cross-learning, and cross-corpus evaluation. Our study confirms that kernels using dependency trees generally outperform kernels based on syntax trees. However, our study also shows that only the best kernel methods can compete with a simple rule-based approach when the evaluation prevents information leakage between training and test corpora. Our results further reveal that the F-score of many approaches drops significantly if no corpus-specific parameter optimization is applied and that methods reaching a good AUC score often perform much worse in terms of F-score. We conclude that for most kernels no sensible estimation of PPI extraction performance on new text is possible, given the current heterogeneity in evaluation data. Nevertheless, our study shows that three kernels are clearly superior to the other methods.
| S-EPMC2895635 | biostudies-literature
Project description:Genome-wide identification of transcription factor (TF) binding sites is pivotal to our understanding of gene expression regulation. Although much progress has been made in the determination of potential binding regions of proteins by chromatin immunoprecipitation (ChIP), this method has some inherent limitations regarding DNA enrichment efficiency and antibody necessity. Here, we report an alternative strategy for assaying in vivo TF-DNA binding in Arabidopsis thaliana cells by tandem chromatin affinity purification (TChAP). Evaluation of TChAP using the E2Fa TF and comparison with traditional ChIP and single chromatin affinity purification illustrates the suitability of TChAP and provides a resource for exploring the E2Fa transcriptional network. Integration with transcriptome, cis-regulatory element, functional enrichment, and co-expression network analyses demonstrates the quality of the E2Fa TChAP-seq data and validates the identification of new direct E2Fa targets. TChAP enhances both TF target mapping throughput, by circumventing issues related to antibody availability, and output, by improving DNA enrichment efficiency. Illumina Seq analysis of E2Fa bound DNA elements isolated using different chromatin isolation methods. BioProject PRJNA172013; SRA study ID SRP014713
2014-02-04 | E-GEOD-53422 | biostudies-arrayexpress
Project description:Internal water molecules are important to protein structure and function, but positional disorder and low occupancies can obscure their detection by X-ray crystallography. Here, we show that water can be detected within the distal cavities of myoglobin mutants by subtle changes in the absorbance spectrum of pentacoordinate heme, even when the presence of solvent is not readily observed in the corresponding crystal structures. A well-defined, noncoordinated water molecule hydrogen bonded to the distal histidine (His64) is seen within the distal heme pocket in the crystal structure of wild type (wt) deoxymyoglobin. Displacement of this water decreases the rate of ligand entry into wt Mb, and we have shown previously that the entry of this water is readily detected optically after laser photolysis of MbCO complexes. However, for L29F and V68L Mb no discrete positions for solvent molecules are seen in the electron density maps of the crystal structures even though His64 is still present and slow rates of ligand binding indicative of internal water are observed. In contrast, time-resolved perturbations of the visible absorption bands of L29F and V68L deoxyMb generated after laser photolysis detect the entry and significant occupancy of water within the distal pockets of these variants. Thus, the spectral perturbation of pentacoordinate heme offers a potentially robust system for measuring nonspecific hydration of the active sites of heme proteins.
| S-EPMC2774273 | biostudies-literature
Project description:With the increasing importance and complexity of data pipelines, data quality became one of the key challenges in modern software applications. The importance of data quality has been recognized beyond the field of data engineering and database management systems (DBMSs). Also, for machine learning (ML) applications, high data quality standards are crucial to ensure robust predictive performance and responsible usage of automated decision making. One of the most frequent data quality problems is missing values. Incomplete datasets can break data pipelines and can have a devastating impact on downstream ML applications when not detected. While statisticians and, more recently, ML researchers have introduced a variety of approaches to impute missing values, comprehensive benchmarks comparing classical and modern imputation approaches under fair and realistic conditions are underrepresented. Here, we aim to fill this gap. We conduct a comprehensive suite of experiments on a large number of datasets with heterogeneous data and realistic missingness conditions, comparing both novel deep learning approaches and classical ML imputation methods when either only test or train and test data are affected by missing data. Each imputation method is evaluated regarding the imputation quality and the impact imputation has on a downstream ML task. Our results provide valuable insights into the performance of a variety of imputation methods under realistic conditions. We hope that our results help researchers and engineers to guide their data preprocessing method selection for automated data quality improvement.
| S-EPMC8297389 | biostudies-literature
Project description:G protein-coupled receptors (GPCR) are important drug discovery targets. Despite progress, many GPCR structures have not yet been solved. For these targets, comparative modeling is used in virtual ligand screening to prioritize experimental efforts. However, the structure of extracellular loop 2 (ECL2) is often poorly predicted. This is significant due to involvement of ECL2 in ligand binding for many Class A GPCR. Here we examine the performance of loop modeling protocols available in the Rosetta (cyclic coordinate descent [CCD], KIC with fragments [KICF] and next generation KIC [NGK]) and Molecular Operating Environment (MOE) software suites (de novo search). ECL2 from GPCR crystal structures served as the structure prediction targets and were divided into four sets depending on loop length. Results suggest that KICF and NGK sampled and scored more loop models with sub-angstrom and near-atomic accuracy than CCD or de novo search for loops of 24 or fewer residues. None of the methods were able to sample loop conformations with near-atomic accuracy for the longest targets ranging from 25 to 32 residues based on 1000 models generated. For these long loop targets, increased conformational sampling is necessary. The strongly conserved disulfide bond between Cys3.25 and Cys45.50 in ECL2 proved an effective filter. Setting an upper limit of 5.1 Å on the S-S distance improved the lowest RMSD model included in the top 10 scored structures in Groups 1-4 on average between 0.33 and 1.27 Å. Disulfide bond formation and geometry optimization of ECL2 provided an additional incremental benefit in structure quality.
| S-EPMC6628340 | biostudies-literature
Project description:The data reported here are a comparison among four different methods for the detection of carbonylated proteins, a validated biomarker of oxidative stress. The reference samples were heart and kidney extracts of Guinea pigs transfused with hemoglobin-based oxygen carriers (Alomari et al. FRBM, [11]). We measured the carbonyl content of organ extracts by using i) the Levine spectrophotometric method, which takes advantage of the chromogenic reaction of carbonyl groups with 2,4-dinitrophenylhydrazine (DNPH), ii) a commercially available ELISA assay based on an anti-DNPH antibodies, iii) a commercially available Western blot method based on anti-DNPH antibodies and iv) an in-gel detection approach with the fluorophoric reagent fluorescein-5-thiosemicarbazide. The former two methods measure total protein carbonylation of a sample, whereas the latter two require an electrophoretic separation and therefore potentially allow for the identification of specific carbonylated proteins.
| S-EPMC6141388 | biostudies-literature
Project description:In order to maximize the use of results from high-throughput experimental studies, e.g. GWAS, for identification and diagnostics of new disease-associated genes, it is important to have properly analyzed and benchmarked gene prioritization tools. While prospective benchmarks are underpowered to provide statistically significant results in their attempt to differentiate the performance of gene prioritization tools, a strategy for retrospective benchmarking has been missing, and new tools usually only provide internal validations. The Gene Ontology(GO) contains genes clustered around annotation terms. This intrinsic property of GO can be utilized in construction of robust benchmarks, objective to the problem domain. We demonstrate how this can be achieved for network-based gene prioritization tools, utilizing the FunCoup network. We use cross-validation and a set of appropriate performance measures to compare state-of-the-art gene prioritization algorithms: three based on network diffusion, NetRank and two implementations of Random Walk with Restart, and MaxLink that utilizes network neighborhood. Our benchmark suite provides a systematic and objective way to compare the multitude of available and future gene prioritization tools, enabling researchers to select the best gene prioritization tool for the task at hand, and helping to guide the development of more accurate methods.
| S-EPMC5399445 | biostudies-literature