Dataset Information

A comprehensive system for evaluation of remote sequence similarity detection.

ABSTRACT: Accurate and sensitive performance evaluation is crucial for both effective development of better structure prediction methods based on sequence similarity, and for the comparative analysis of existing methods. Up to date, there has been no satisfactory comprehensive evaluation method that (i) is based on a large and statistically unbiased set of proteins with clearly defined relationships; and (ii) covers all performance aspects of sequence-based structure predictors, such as sensitivity and specificity, alignment accuracy and coverage, and structure template quality.With the aim of designing such a method, we (i) select a statistically balanced set of divergent protein domains from SCOP, and define similarity relationships for the majority of these domains by complementing the best of information available in SCOP with a rigorous SVM-based algorithm; and (ii) develop protocols for the assessment of similarity detection and alignment quality from several complementary perspectives. The evaluation of similarity detection is based on ROC-like curves and includes several complementary approaches to the definition of true/false positives. Reference-dependent approaches use the 'gold standard' of pre-defined domain relationships and structure-based alignments. Reference-independent approaches assess the quality of structural match predicted by the sequence alignment, with respect to the whole domain length (global mode) or to the aligned region only (local mode). Similarly, the evaluation of alignment quality includes several reference-dependent and -independent measures, in global and local modes. As an illustration, we use our benchmark to compare the performance of several methods for the detection of remote sequence similarities, and show that different aspects of evaluation reveal different properties of the evaluated methods, highlighting their advantages, weaknesses, and potential for further development.The presented benchmark provides a new tool for a statistically unbiased assessment of methods for remote sequence similarity detection, from various complementary perspectives. This tool should be useful both for users choosing the best method for a given purpose, and for developers designing new, more powerful methods. The benchmark set, reference alignments, and evaluation codes can be downloaded from ftp://iole.swmed.edu/pub/evaluation/.

SUBMITTER: Qi Y

PROVIDER: S-EPMC2031906 | biostudies-literature | 2007 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A comprehensive system for evaluation of remote sequence similarity detection.

Qi Yuan Y Sadreyev Ruslan I RI Wang Yong Y Kim Bong-Hyun BH Grishin Nick V NV

BMC bioinformatics 20070828

<h4>Background</h4>Accurate and sensitive performance evaluation is crucial for both effective development of better structure prediction methods based on sequence similarity, and for the comparative analysis of existing methods. Up to date, there has been no satisfactory comprehensive evaluation method that (i) is based on a large and statistically unbiased set of proteins with clearly defined relationships; and (ii) covers all performance aspects of sequence-based structure predictors, such as ...[more]

PMID: 17725841

Similar Datasets

Project description:BackgroundThe smoking relapse rate during the first 12 months after pregnancy is around 80% in the United States. Delivering remote smoking cessation interventions to women in the postpartum period can reduce the burden associated with frequent office visits and can enable remote communication and support. Developing reliable, remote, smoking measuring instruments is a crucial step in achieving this vision.ObjectiveThe study presents the evaluation of the MoMba Live Long system, a smartphone-based breath carbon monoxide (CO) meter and a custom iOS smartphone app. We report on how our smoking detection system worked in a controlled office environment and in an out-of-office environment to examine its potential to deliver a remote contingency management intervention.MethodsIn-office breath tests were completed using both the MoMba Live Long system and a commercial monitor, the piCO+ Smokerlyzer. In addition, each participant provided a urine test for smoking status validation through cotinine. We used in-office test data to verify the validity of the MoMba Live Long smoking detection system. We also collected out-of-office tests to assess how the system worked remotely and enabled user verification. Pregnant adult women in their second or third trimester participated in the study for a period of 12 weeks. This study was carried out in the United States.ResultsAnalyses of in-office tests included 143 breath tests contributed from 10 participants. CO readings between the MoMba Live Long system and the piCO+ were highly correlated (r=.94). In addition, the MoMba Live Long system accurately distinguished smokers from nonsmokers with a sensitivity of 0.91 and a specificity of 0.94 when the piCO+ was used as a gold standard, and a sensitivity of 0.81 and specificity of 1.0 when cotinine in urine was used to confirm smoking status. All participants indicated that the system was easy to use.ConclusionsRelatively inexpensive portable and internet-connected CO monitors can enable remote smoking status detection in a wide variety of nonclinical settings with reliable and valid measures comparable to a commercially available CO monitor.Trial registrationClinicalTrials.gov NCT02237898; https://clinicaltrials.gov/ct2/show/NCT02237898.

Project description:Similarity measure, also called information measure, is a concept used to distinguish different objects. It has been studied from different contexts by employing mathematical, psychological, and fuzzy approaches. Image steganography is the art of hiding secret data into an image in such a way that it cannot be detected by an intruder. In image steganography, hiding secret data in the plain or non-edge regions of the image is significant due to the high similarity and redundancy of the pixels in their neighborhood. However, the similarity measure of the neighboring pixels, i.e., their proximity in color space, is perceptual rather than mathematical. Thus, this paper proposes an interval type-2 fuzzy logic system (IT2 FLS) to determine the similarity between the neighboring pixels by involving an instinctive human perception through a rule-based approach. The pixels of the image having high similarity values, calculated using the proposed IT2 FLS similarity measure, are selected for embedding via the least significant bit (LSB) method. We term the proposed procedure of steganography as 'IT2 FLS-LSB method'. Moreover, we have developed two more methods, namely, type-1 fuzzy logic system based least significant bits (T1FLS-LSB) and Euclidean distance based similarity measures for least significant bit (SM-LSB) steganographic methods. Experimental simulations were conducted for a collection of images and quality index metrics, such as PSNR (peak signal-to-noise ratio), UQI (universal quality index), and SSIM (structural similarity measure) are used. All the three steganographic methods are applied on dataset and the quality metrics are calculated. The obtained stego images and results are shown and thoroughly compared to determine the efficacy of the IT2 FLS-LSB method. We have also demonstrated the high payload capacity of our proposed method. Finally, we have done a comparative analysis of the proposed approach with the existing well-known steganographic methods to show the effectiveness of our proposed steganographic method.

Dataset Information

A comprehensive system for evaluation of remote sequence similarity detection.

Publications

A comprehensive system for evaluation of remote sequence similarity detection.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets