Dataset Information

Evaluation of model quality predictions in CASP9.

ABSTRACT: CASP has been assessing the state of the art in the a priori estimation of accuracy of protein structure prediction since 2006. The inclusion of model quality assessment category in CASP contributed to a rapid development of methods in this area. In the last experiment, 46 quality assessment groups tested their approaches to estimate the accuracy of protein models as a whole and/or on a per-residue basis. We assessed the performance of these methods predominantly on the basis of the correlation between the predicted and observed quality of the models on both global and local scales. The ability of the methods to identify the models closest to the best one, to differentiate between good and bad models, and to identify well modeled regions was also analyzed. Our evaluations demonstrate that even though global quality assessment methods seem to approach perfection point (weighted average per-target Pearson's correlation coefficients are as high as 0.97 for the best groups), there is still room for improvement. First, all top-performing methods use consensus approaches to generate quality estimates, and this strategy has its own limitations. Second, the methods that are based on the analysis of individual models lag far behind clustering techniques and need a boost in performance. The methods for estimating per-residue accuracy of models are less accurate than global quality assessment methods, with an average weighted per-model correlation coefficient in the range of 0.63-0.72 for the best 10 groups.

SUBMITTER: Kryshtafovych A

PROVIDER: S-EPMC3226935 | biostudies-literature | 2011

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluation of model quality predictions in CASP9.

Kryshtafovych Andriy A Fidelis Krzysztof K Tramontano Anna A

Proteins 20111014

CASP has been assessing the state of the art in the a priori estimation of accuracy of protein structure prediction since 2006. The inclusion of model quality assessment category in CASP contributed to a rapid development of methods in this area. In the last experiment, 46 quality assessment groups tested their approaches to estimate the accuracy of protein models as a whole and/or on a per-residue basis. We assessed the performance of these methods predominantly on the basis of the correlation ...[more]

PMID: 21997462

Similar Datasets

Project description:BackgroundSkeeter Buster is a stochastic, spatially explicit simulation model of Aedes aegypti populations, designed to predict the outcome of vector population control methods. In this study, we apply the model to two specific locations, the cities of Iquitos, Peru, and Buenos Aires, Argentina. These two sites differ in the amount of field data that is available for location-specific customization. By comparing output from Skeeter Buster to field observations in these two cases we evaluate population dynamics predictions by Skeeter Buster with varying degrees of customization.Methodology/principal findingsSkeeter Buster was customized to the Iquitos location by simulating the layout of houses and the associated distribution of water-holding containers, based on extensive surveys of Ae. aegypti populations and larval habitats that have been conducted in Iquitos for over 10 years. The model is calibrated by adjusting the food input into various types of containers to match their observed pupal productivity in the field. We contrast the output of this customized model to the data collected from the natural population, comparing pupal numbers and spatial distribution of pupae in the population. Our results show that Skeeter Buster replicates specific population dynamics and spatial structure of Ae. aegypti in Iquitos. We then show how Skeeter Buster can be customized for Buenos Aires, where we only had Ae. aegypti abundance data that was averaged across all locations. In the Argentina case Skeeter Buster provides a satisfactory simulation of temporal population dynamics across seasons.ConclusionsThis model can provide a faithful description of Ae. aegypti populations, through a process of location-specific customization that is contingent on the amount of data available from field collections. We discuss limitations presented by some specific components of the model such as the description of food dynamics and challenges that these limitations bring to model evaluation.

Dataset Information

Evaluation of model quality predictions in CASP9.

Publications

Evaluation of model quality predictions in CASP9.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets