Dataset Information

Benchmarking the next generation of homology inference tools.

ABSTRACT:

Motivation

Over the last decades, vast numbers of sequences were deposited in public databases. Bioinformatics tools allow homology and consequently functional inference for these sequences. New profile-based homology search tools have been introduced, allowing reliable detection of remote homologs, but have not been systematically benchmarked. To provide such a comparison, which can guide bioinformatics workflows, we extend and apply our previously developed benchmark approach to evaluate the 'next generation' of profile-based approaches, including CS-BLAST, HHSEARCH and PHMMER, in comparison with the non-profile based search tools NCBI-BLAST, USEARCH, UBLAST and FASTA.

Method

We generated challenging benchmark datasets based on protein domain architectures within either the PFAM + Clan, SCOP/Superfamily or CATH/Gene3D domain definition schemes. From each dataset, homologous and non-homologous protein pairs were aligned using each tool, and standard performance metrics calculated. We further measured congruence of domain architecture assignments in the three domain databases.

Results

CSBLAST and PHMMER had overall highest accuracy. FASTA, UBLAST and USEARCH showed large trade-offs of accuracy for speed optimization.

Conclusion

Profile methods are superior at inferring remote homologs but the difference in accuracy between methods is relatively small. PHMMER and CSBLAST stand out with the highest accuracy, yet still at a reasonable computational cost. Additionally, we show that less than 0.1% of Swiss-Prot protein pairs considered homologous by one database are considered non-homologous by another, implying that these classifications represent equivalent underlying biological phenomena, differing mostly in coverage and granularity.

Availability and implementation

Benchmark datasets and all scripts are placed at (http://sonnhammer.org/download/Homology_benchmark).

Contact

forslund@embl.de

Supplementary information

Supplementary data are available at Bioinformatics online.

SUBMITTER: Saripella GV

PROVIDER: S-EPMC5013910 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:G protein-coupled receptors (GPCR) comprise the largest family of membrane proteins and are of considerable interest as targets for drug development. However, many GPCR structures remain unsolved. To address the structural ambiguity of these receptors, computational tools such as homology modeling and loop modeling are often employed to generate predictive receptor structures. Here we combined both methods to benchmark a protocol incorporating homology modeling based on a locally selected template and extracellular loop modeling that additionally evaluates the presence of template ligands during these modeling steps. Ligands were also docked using three docking methods and two pose selection methods to elucidate an optimal ligand pose selection method. Results suggest that local template-based homology models followed by loop modeling produce more accurate and predictive receptor models than models produced without loop modeling, with decreases in average receptor and ligand RMSD of 0.54 Å and 2.91 Å, respectively. Ligand docking results showcased the ability of MOE induced fit docking to produce ligand poses with atom root-mean-square deviation (RMSD) values at least 0.20 Å lower (on average) than the other two methods benchmarked in this study. In addition, pose selection methods (software-based scoring, ligand complementation) selected lower RMSD poses with MOE induced fit docking than either of the other methods (averaging at least 1.57 Å lower), indicating that MOE induced fit docking is most suited for docking into GPCR homology models in our hands. In addition, target receptor models produced with a template ligand present throughout the modeling process most often produced target ligand poses with RMSD values ≤ 4.5 Å and Tanimoto coefficients > 0.6 after selection based on ligand complementation than target receptor models produced in the absence of template ligands. Overall, the findings produced by this study support the use of local template homology modeling in combination with de novo ECL2 modeling in the presence of a ligand from the template crystal structure to generate GPCR models intended to study ligand binding interactions.