Dataset Information

An expanded evaluation of protein function prediction methods shows an improvement in accuracy.

ABSTRACT: BACKGROUND:A major bottleneck in our understanding of the molecular underpinnings of life is the assignment of function to proteins. While molecular experiments provide the most reliable annotation of proteins, their relatively low throughput and restricted purview have led to an increasing role for computational function prediction. However, assessing methods for protein function prediction and tracking progress in the field remain challenging. RESULTS:We conducted the second critical assessment of functional annotation (CAFA), a timed challenge to assess computational methods that automatically assign protein function. We evaluated 126 methods from 56 research groups for their ability to predict biological functions using Gene Ontology and gene-disease associations using Human Phenotype Ontology on a set of 3681 proteins from 18 species. CAFA2 featured expanded analysis compared with CAFA1, with regards to data set size, variety, and assessment metrics. To review progress in the field, the analysis compared the best methods from CAFA1 to those of CAFA2. CONCLUSIONS:The top-performing methods in CAFA2 outperformed those from CAFA1. This increased accuracy can be attributed to a combination of the growing number of experimental annotations and improved methods for function prediction. The assessment also revealed that the definition of top-performing algorithms is ontology specific, that different performance metrics can be used to probe the nature of accurate predictions, and the relative diversity of predictions in the biological process and human phenotype ontologies. While there was methodological improvement between CAFA1 and CAFA2, the interpretation of results and usefulness of individual methods remain context-dependent.

SUBMITTER: Jiang Y

PROVIDER: S-EPMC5015320 | biostudies-literature | 2016 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An expanded evaluation of protein function prediction methods shows an improvement in accuracy.

Jiang Yuxiang Y Oron Tal Ronnen TR Clark Wyatt T WT Bankapur Asma R AR D'Andrea Daniel D Lepore Rosalba R Funk Christopher S CS Kahanda Indika I Verspoor Karin M KM Ben-Hur Asa A Koo Da Chen Emily da CE Penfold-Brown Duncan D Shasha Dennis D Youngs Noah N Bonneau Richard R Lin Alexandra A Sahraeian Sayed M E SM Martelli Pier Luigi PL Profiti Giuseppe G Casadio Rita R Cao Renzhi R Zhong Zhaolong Z Cheng Jianlin J Altenhoff Adrian A Skunca Nives N Dessimoz Christophe C Dogan Tunca T Hakala Kai K Kaewphan Suwisa S Mehryary Farrokh F Salakoski Tapio T Ginter Filip F Fang Hai H Smithers Ben B Oates Matt M Gough Julian J Törönen Petri P Koskinen Patrik P Holm Liisa L Chen Ching-Tai CT Hsu Wen-Lian WL Bryson Kevin K Cozzetto Domenico D Minneci Federico F Jones David T DT Chapman Samuel S Bkc Dukka D Khan Ishita K IK Kihara Daisuke D Ofer Dan D Rappoport Nadav N Stern Amos A Cibrian-Uhalte Elena E Denny Paul P Foulger Rebecca E RE Hieta Reija R Legge Duncan D Lovering Ruth C RC Magrane Michele M Melidoni Anna N AN Mutowo-Meullenet Prudence P Pichler Klemens K Shypitsyna Aleksandra A Li Biao B Zakeri Pooya P ElShal Sarah S Tranchevent Léon-Charles LC Das Sayoni S Dawson Natalie L NL Lee David D Lees Jonathan G JG Sillitoe Ian I Bhat Prajwal P Nepusz Tamás T Romero Alfonso E AE Sasidharan Rajkumar R Yang Haixuan H Paccanaro Alberto A Gillis Jesse J Sedeño-Cortés Adriana E AE Pavlidis Paul P Feng Shou S Cejuela Juan M JM Goldberg Tatyana T Hamp Tobias T Richter Lothar L Salamov Asaf A Gabaldon Toni T Marcet-Houben Marina M Supek Fran F Gong Qingtian Q Ning Wei W Zhou Yuanpeng Y Tian Weidong W Falda Marco M Fontana Paolo P Lavezzo Enrico E Toppo Stefano S Ferrari Carlo C Giollo Manuel M Piovesan Damiano D Tosatto Silvio C E SC Del Pozo Angela A Fernández José M JM Maietta Paolo P Valencia Alfonso A Tress Michael L ML Benso Alfredo A Di Carlo Stefano S Politano Gianfranco G Savino Alessandro A Rehman Hafeez Ur HU Re Matteo M Mesiti Marco M Valentini Giorgio G Bargsten Joachim W JW van Dijk Aalt D J AD Gemovic Branislava B Glisic Sanja S Perovic Vladmir V Veljkovic Veljko V Veljkovic Nevena N Almeida-E-Silva Danillo C DC Vencio Ricardo Z N RZ Sharan Malvika M Vogel Jörg J Kansakar Lakesh L Zhang Shanshan S Vucetic Slobodan S Wang Zheng Z Sternberg Michael J E MJ Wass Mark N MN Huntley Rachael P RP Martin Maria J MJ O'Donovan Claire C Robinson Peter N PN Moreau Yves Y Tramontano Anna A Babbitt Patricia C PC Brenner Steven E SE Linial Michal M Orengo Christine A CA Rost Burkhard B Greene Casey S CS Mooney Sean D SD Friedberg Iddo I Radivojac Predrag P

Genome biology 20160907 1

<h4>Background</h4>A major bottleneck in our understanding of the molecular underpinnings of life is the assignment of function to proteins. While molecular experiments provide the most reliable annotation of proteins, their relatively low throughput and restricted purview have led to an increasing role for computational function prediction. However, assessing methods for protein function prediction and tracking progress in the field remain challenging.<h4>Results</h4>We conducted the second cri ...[more]

PMID: 27604469

Similar Datasets

Project description:BackgroundDetermining the function of uncharacterized proteins is a major challenge in the post-genomic era due to the problem's complexity and scale. Identifying a protein's function contributes to an understanding of its role in the involved pathways, its suitability as a drug target, and its potential for protein modifications. Several graph-theoretic approaches predict unidentified functions of proteins by using the functional annotations of better-characterized proteins in protein-protein interaction networks. We systematically consider the use of literature co-occurrence data, introduce a new method for quantifying the reliability of co-occurrence and test how performance differs across species. We also quantify changes in performance as the prediction algorithms annotate with increased specificity.ResultsWe find that including information on the co-occurrence of proteins within an abstract greatly boosts performance in the Functional Flow graph-theoretic function prediction algorithm in yeast, fly and worm. This increase in performance is not simply due to the presence of additional edges since supplementing protein-protein interactions with co-occurrence data outperforms supplementing with a comparably-sized genetic interaction dataset. Through the combination of protein-protein interactions and co-occurrence data, the neighborhood around unknown proteins is quickly connected to well-characterized nodes which global prediction algorithms can exploit. Our method for quantifying co-occurrence reliability shows superior performance to the other methods, particularly at threshold values around 10% which yield the best trade off between coverage and accuracy. In contrast, the traditional way of asserting co-occurrence when at least one abstract mentions both proteins proves to be the worst method for generating co-occurrence data, introducing too many false positives. Annotating the functions with greater specificity is harder, but co-occurrence data still proves beneficial.ConclusionCo-occurrence data is a valuable supplemental source for graph-theoretic function prediction algorithms. A rapidly growing literature corpus ensures that co-occurrence data is a readily-available resource for nearly every studied organism, particularly those with small protein interaction databases. Though arguably biased toward known genes, co-occurrence data provides critical additional links to well-studied regions in the interaction network that graph-theoretic function prediction algorithms can exploit.

Project description:BackgroundCellular activities are governed by the physical and the functional interactions among several proteins involved in various biological pathways. With the availability of sequenced genomes and high-throughput experimental data one can identify genome-wide protein-protein interactions using various computational techniques. Comparative assessments of these techniques in predicting protein interactions have been frequently reported in the literature but not their ability to elucidate a particular biological pathway.MethodsTowards the goal of understanding the prediction capabilities of interactions among the specific biological pathway proteins, we report the analyses of 14 biological pathways of Escherichia coli catalogued in KEGG database using five protein-protein functional linkage prediction methods. These methods are phylogenetic profiling, gene neighborhood, co-presence of orthologous genes in the same gene clusters, a mirrortree variant, and expression similarity.ConclusionsOur results reveal that the prediction of metabolic pathway protein interactions continues to be a challenging task for all methods which possibly reflect flexible/independent evolutionary histories of these proteins. These methods have predicted functional associations of proteins involved in amino acids, nucleotide, glycans and vitamins & co-factors pathways slightly better than the random performance on carbohydrate, lipid and energy metabolism. We also make similar observations for interactions involved among the environmental information processing proteins. On the contrary, genetic information processing or specialized processes such as motility related protein-protein linkages that occur in the subset of organisms are predicted with comparable accuracy. Metabolic pathways are best predicted by using neighborhood of orthologous genes whereas phyletic pattern is good enough to reconstruct central dogma pathway protein interactions. We have also shown that the effective use of a particular prediction method depends on the pathway under investigation. In case one is not focused on specific pathway, gene expression similarity method is the best option.

Dataset Information

An expanded evaluation of protein function prediction methods shows an improvement in accuracy.

Publications

An expanded evaluation of protein function prediction methods shows an improvement in accuracy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets