A publicly available benchmark for biomedical dataset retrieval: the reference standard for the 2016 bioCADDIE dataset retrieval challenge.
Ontology highlight
ABSTRACT:
SUBMITTER: Cohen T
PROVIDER: S-EPMC5737202 | biostudies-literature | 2017 Jan
REPOSITORIES: biostudies-literature
Publications
A publicly available benchmark for biomedical dataset retrieval: the reference standard for the 2016 bioCADDIE dataset retrieval challenge.
Cohen Trevor T Roberts Kirk K Gururaj Anupama E AE Chen Xiaoling X Pournejati Saeid S Alter George G Hersh William R WR Demner-Fushman Dina D Ohno-Machado Lucila L Xu Hua H
Database : the journal of biological databases and curation 20170101
<h4>Database url</h4>https://biocaddie.org/benchmark-data. ...[more]
Similar Datasets
Project description:Electrocardiography (ECG) is a key non-invasive diagnostic tool for cardiovascular diseases which is increasingly supported by algorithms based on machine learning. Major obstacles for the development of automatic ECG interpretation algorithms are both the lack of public datasets and well-defined benchmarking procedures to allow comparison s of different algorithms. To address these issues, we put forward PTB-XL, the to-date largest freely accessible clinical 12-lead ECG-waveform dataset comprising 21837 records from 18885 patients of 10 seconds length. The ECG-waveform data was annotated by up to two cardiologists as a multi-label dataset, where diagnostic labels were further aggregated into super and subclasses. The dataset covers a broad range of diagnostic classes including, in particular, a large fraction of healthy records. The combination with additional metadata on demographics, additional diagnostic statements, diagnosis likelihoods, manually annotated signal properties as well as suggested folds for splitting training and test sets turns the dataset into a rich resource for the development and the evaluation of automatic ECG interpretation algorithms.
| S-EPMC7248071 | biostudies-literature
Project description:This Dataset is no actual new study but the Yeast Gold Standard benchmark dataset used in the PIA manuscript.
2015-05-08 | PXD000792 | Pride
Project description:https://github.com/w2wei/dataset_retrieval_pipeline.
| S-EPMC5861401 | biostudies-literature
Project description:INTRODUCTION AND OBJECTIVE: Social media has been suggested as a source for safety information, supplementing existing safety surveillance data sources. This article summarises the activities undertaken, and the associated challenges, to create a benchmark reference dataset that can be used to evaluate the performance of automated methods and systems for adverse event recognition. METHODS:A retrospective analysis of public English-language Twitter posts (Tweets) was performed. We sampled 57,473 Tweets out of 5,645,336 Tweets created between 1 March, 2012 and 1 March, 2015 that mentioned at least one of six medicinal products of interest (insulin glargine, levetiracetam, methylphenidate, sorafenib, terbinafine, zolpidem). Products, adverse events, indications, product-event combinations, and product-indication combinations were extracted and coded by two independent teams of safety reviewers. RESULTS:The benchmark reference dataset consisted of 1056 positive controls ("adverse event Tweets") and 56,417 negative controls ("non-adverse event Tweets"). The 1056 adverse event Tweets contained 1396 product-event combinations referring to personal adverse event experiences, comprising 292 different MedDRA® Preferred Terms. The 1171 product-event combinations (83.9%) were confined to four MedDRA® System Organ Classes. The 195 Tweets (18.5%) contained indication information, comprising 25 different Preferred Terms. CONCLUSIONS:A manually curated benchmark reference dataset based on Twitter data has been created and is made available to the research community to evaluate the performance of automated methods and systems for adverse event recognition in unstructured free-text information.
| S-EPMC7165158 | biostudies-literature
Project description:Genome-wide association (GWA) studies have proved extremely successful in identifying novel genetic loci contributing effects to complex human diseases. In doing so, they have highlighted the fact that many potential loci of modest effect remain undetected, partly due to the need for samples consisting of many thousands of individuals. Large-scale international initiatives, such as the Wellcome Trust Case Control Consortium, the Genetic Association Information Network, and the database of genetic and phenotypic information, aim to facilitate discovery of modest-effect genes by making genome-wide data publicly available, allowing information to be combined for the purpose of pooled analysis. In principle, disease or control samples from these studies could be used to increase the power of any GWA study via judicious use as "genetically matched controls" for other traits. Here, we present the biological motivation for the problem and the theoretical potential for expanding the control group with publicly available disease or reference samples. We demonstrate that a naïve application of this strategy can greatly inflate the false-positive error rate in the presence of population structure. As a remedy, we make use of genome-wide data and model selection techniques to identify "axes" of genetic variation which are associated with disease. These axes are then included as covariates in association analysis to correct for population structure, which can result in increases in power over standard analysis of genetic information from the samples in the original GWA study.
| S-EPMC2962805 | biostudies-other
Project description:MotivationBiomedical entity linking (BEL) is the task of grounding entity mentions to a knowledge base (KB). It plays a vital role in information extraction pipelines for the life sciences literature. We review recent work in the field and find that, as the task is absent from existing benchmarks for biomedical text mining, different studies adopt different experimental setups making comparisons based on published numbers problematic. Furthermore, neural systems are tested primarily on instances linked to the broad coverage KB UMLS, leaving their performance to more specialized ones, e.g. genes or variants, understudied.ResultsWe therefore developed BELB, a biomedical entity linking benchmark, providing access in a unified format to 11 corpora linked to 7 KBs and spanning six entity types: gene, disease, chemical, species, cell line, and variant. BELB greatly reduces preprocessing overhead in testing BEL systems on multiple corpora offering a standardized testbed for reproducible experiments. Using BELB, we perform an extensive evaluation of six rule-based entity-specific systems and three recent neural approaches leveraging pre-trained language models. Our results reveal a mixed picture showing that neural approaches fail to perform consistently across entity types, highlighting the need of further studies towards entity-agnostic models.Availability and implementationThe source code of BELB is available at: https://github.com/sg-wbi/belb. The code to reproduce our experiments can be found at: https://github.com/sg-wbi/belb-exp.
| S-EPMC10681865 | biostudies-literature
Project description:The Pyrenees is a mountain range in south-western Europe that supports a rich diversity of bird species. In 1981, point count surveys of breeding birds (passerines and picidae) were carried out in the Vanera valley, a valley in the eastern Pyrenees, resulting in a data set of 228 counts. These data provide valuable information on the distribution and abundance of bird populations on 5350 ha of heterogeneous land (including cropland, grasslands, heaths and river banks, and pine forest) at altitudes ranging from 1100 to 2600 meters. Additional point count surveys were carried out in the pine forest from 1982 to 1985, resulting in a data set of 144 counts. Habitat descriptors (percentage of herbaceous and ligneous plant cover at different heights - 0-1m, 1-4m and more than 8m; vegetation type) and altitude were assessed around each point count. This dataset provides a complete picture of the breeding bird community in a typical valley of eastern Pyrenees in the early 1980s, which could be compared with future censuses to contribute to a variety of research questions, such as quantifying changes in birds between the early 1980s and now in mountain areas, understanding the effects of climate change on bird populations, examining the effects of habitat fragmentation and land-use change, and identifying priority areas for conservation and management. These data could inspire new research and contribute to our collective understanding of bird ecology in the Pyrenees.
| S-EPMC10562142 | biostudies-literature
Project description:BackgroundCellular deconvolution of bulk RNA-sequencing (RNA-seq) data using single cell or nuclei RNA-seq (sc/snRNA-seq) reference data is an important strategy for estimating cell type composition in heterogeneous tissues, such as human brain. Several deconvolution methods have been developed and they have been previously benchmarked against simulated data, pseudobulked sc/snRNA-seq data, or cell type proportions derived from immunohistochemistry reference data. A major limitation preventing the improvement of deconvolution algorithms has been the lack of highly integrated datasets with orthogonal measurements of gene expression and estimates of cell type proportions on the same tissue block. The performance of existing deconvolution algorithms has not yet been explored across different RNA extraction methods (e.g. cytosolic, nuclear, or whole cell RNA), different library preparation types (e.g. mRNA enrichment vs. ribosomal RNA depletion), or with matched single cell reference datasets.ResultsA rich multi-assay dataset was generated in postmortem human dorsolateral prefrontal cortex (DLPFC) from 22 tissue blocks. Assays included spatially-resolved transcriptomics, snRNA-seq, bulk RNA-seq across six RNA extraction and RNA-seq library combinations, and orthogonal cell type measurements via RNAScope/Immunofluorescence (RNAScope/IF). The Mean Ratio method was developed for selecting cell type marker genes for deconvolution and is implemented in the DeconvoBuddies R package. Five extensively benchmarked computational deconvolution algorithms were evaluated in DLPFC across six RNA-seq combinations and predicted cell type proportions were compared to those measured by RNAScope/IF.ConclusionsWe show that Bisque and hspe are the top performing methods with performance dependent on the RNA-seq library preparation conditions. We provide a multi-assay resource for the development and evaluation of deconvolution algorithms.
| S-EPMC10888823 | biostudies-literature
Project description:High-throughput genomic and molecular profiling of tumors is emerging as an important clinical approach. Molecular profiling is increasingly being used to guide cancer patient care, especially in advanced and incurable cancers. However, navigating the scientific literature to make evidence-based clinical decisions based on molecular profiling results is overwhelming for many oncology clinicians and researchers. The Personalized Cancer Therapy website (www.personalizedcancertherapy.org) was created to provide an online resource for clinicians and researchers to facilitate navigation of available data. Specifically, this resource can be used to help identify potential therapy options for patients harboring oncogenic genomic alterations. Herein, we describe how content on www.personalizedcancertherapy.org is generated and maintained. We end with case scenarios to illustrate the clinical utility of the website. The goal of this publicly available resource is to provide easily accessible information to a broad oncology audience, as this may help ease the information retrieval burden facing participants in the precision oncology field. Cancer Res; 77(21); e123-6. ©2017 AACR.
| S-EPMC5774617 | biostudies-other
Project description:Metabolomic data processing pipelines have been improving in recent years, allowing for greater feature extraction and identification. Lately, machine learning and robust statistical techniques to control false discoveries are being incorporated into metabolomic data analysis. In this paper, we introduce one such recently developed technique called aggregate knockoff filtering to untargeted metabolomic analysis. When applied to a publicly available dataset, aggregate knockoff filtering combined with typical p-value filtering improves the number of significantly changing metabolites by 25% when compared to conventional untargeted metabolomic data processing. By using this method, features that would normally not be extracted under standard processing would be brought to researchers' attention for further analysis.
| S-EPMC8320926 | biostudies-literature