Evaluating active learning methods for annotating semantic predications.
Ontology highlight
ABSTRACT:
SUBMITTER: Vasilakes J
PROVIDER: S-EPMC6367018 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:It is common practice for data providers to include text descriptions for each column when publishing datasets in the form of data dictionaries. While these documents are useful in helping an end-user properly interpret the meaning of a column in a dataset, existing data dictionaries typically are not machine-readable and do not follow a common specification standard. We introduce the Semantic Data Dictionary, a specification that formalizes the assignment of a semantic representation of data, enabling standardization and harmonization across diverse datasets. In this paper, we present our Semantic Data Dictionary work in the context of our work with biomedical data; however, the approach can and has been used in a wide range of domains. The rendition of data in this form helps promote improved discovery, interoperability, reuse, traceability, and reproducibility. We present the associated research and describe how the Semantic Data Dictionary can help address existing limitations in the related literature. We discuss our approach, present an example by annotating portions of the publicly available National Health and Nutrition Examination Survey dataset, present modeling challenges, and describe the use of this approach in sponsored research, including our work on a large NIH-funded exposure and health data portal and in the RPI-IBM collaborative Health Empowerment by Analytics, Learning, and Semantics project. We evaluate this work in comparison with traditional data dictionaries, mapping languages, and data integration tools.
| S-EPMC7583433 | biostudies-literature
Project description:BackgroundNepal has achieved and sustained the elimination of leprosy as a public health problem since 2009, but 17 districts and 3 provinces with 41% (10,907,128) of Nepal's population have yet to eliminate the disease. Pediatric cases and grade-2 disabilities (G2D) indicate recent transmission and late diagnosis, respectively, which necessitate active and early case detection. This operational research was performed to identify approaches best suited for early case detection, determine community-based leprosy epidemiology, and identify hidden leprosy cases early and respond with prompt treatment.MethodsActive case detection was undertaken in two Nepali provinces with the greatest burden of leprosy, Madhesh Province (40% national cases) and Lumbini Province (18%) and at-risk prison populations in Madhesh, Lumbini and Bagmati provinces. Case detection was performed by (1) house-to-house visits among vulnerable populations (n = 26,469); (2) contact examination and tracing (n = 7608); in Madhesh and Lumbini Provinces and, (3) screening prison populations (n = 4428) in Madhesh, Lumbini and Bagmati Provinces of Nepal. Per case direct medical and non-medical costs for each approach were calculated.ResultsNew case detection rates were highest for contact tracing (250), followed by house-to-house visits (102) and prison screening (45) per 100,000 population screened. However, the cost per case identified was cheapest for house-to-house visits [Nepalese rupee (NPR) 76,500/case], followed by contact tracing (NPR 90,286/case) and prison screening (NPR 298,300/case). House-to-house and contact tracing case paucibacillary/multibacillary (PB:MB) ratios were 59:41 and 68:32; female/male ratios 63:37 and 57:43; pediatric cases 11% in both approaches; and grade-2 disabilities (G2D) 11% and 5%, respectively. Developing leprosy was not significantly different among household and neighbor contacts [odds ratios (OR) = 1.4, 95% confidence interval (CI): 0.24-5.85] and for contacts of MB versus PB cases (OR = 0.7, 95% CI 0.26-2.0). Attack rates were not significantly different among household contacts of MB cases (0.32%, 95% CI 0.07-0.94%) and PB cases (0.13%, 95% CI 0.03-0.73) (χ2 = 0.07, df = 1, P = 0.9) and neighbor contacts of MB cases (0.23%, 0.1-0.46) and PB cases (0.48%, 0.19-0.98) (χ2 = 0.8, df = 1, P = 0.7). BCG vaccination with scar presence had a significant protective effect against leprosy (OR = 0.42, 0.22-0.81).ConclusionsThe most effective case identification approach here is contact tracing, followed by house-to-house visits in vulnerable populations and screening in prisons, although house-to-house visits are cheaper. The findings suggest that hidden cases, recent transmission, and late diagnosis in the community exist and highlight the importance of early case detection.
| S-EPMC10698884 | biostudies-literature
Project description:BackgroundPathology synopses consist of semi-structured or unstructured text summarizing visual information by observing human tissue. Experts write and interpret these synopses with high domain-specific knowledge to extract tissue semantics and formulate a diagnosis in the context of ancillary testing and clinical information. The limited number of specialists available to interpret pathology synopses restricts the utility of the inherent information. Deep learning offers a tool for information extraction and automatic feature generation from complex datasets.MethodsUsing an active learning approach, we developed a set of semantic labels for bone marrow aspirate pathology synopses. We then trained a transformer-based deep-learning model to map these synopses to one or more semantic labels, and extracted learned embeddings (i.e., meaningful attributes) from the model's hidden layer.ResultsHere we demonstrate that with a small amount of training data, a transformer-based natural language model can extract embeddings from pathology synopses that capture diagnostically relevant information. On average, these embeddings can be used to generate semantic labels mapping patients to probable diagnostic groups with a micro-average F1 score of 0.779 Â ± 0.025.ConclusionsWe provide a generalizable deep learning model and approach to unlock the semantic information inherent in pathology synopses toward improved diagnostics, biodiscovery and AI-assisted computational pathology.
| S-EPMC9053264 | biostudies-literature
Project description:BackgroundKey to the success of e-Science is the ability to computationally evaluate expert-composed hypotheses for validity against experimental data. Researchers face the challenge of collecting, evaluating and integrating large amounts of diverse information to compose and evaluate a hypothesis. Confronted with rapidly accumulating data, researchers currently do not have the software tools to undertake the required information integration tasks.ResultsWe present HyQue, a Semantic Web tool for querying scientific knowledge bases with the purpose of evaluating user submitted hypotheses. HyQue features a knowledge model to accommodate diverse hypotheses structured as events and represented using Semantic Web languages (RDF/OWL). Hypothesis validity is evaluated against experimental and literature-sourced evidence through a combination of SPARQL queries and evaluation rules. Inference over OWL ontologies (for type specifications, subclass assertions and parthood relations) and retrieval of facts stored as Bio2RDF linked data provide support for a given hypothesis. We evaluate hypotheses of varying levels of detail about the genetic network controlling galactose metabolism in Saccharomyces cerevisiae to demonstrate the feasibility of deploying such semantic computing tools over a growing body of structured knowledge in Bio2RDF.ConclusionsHyQue is a query-based hypothesis evaluation system that can currently evaluate hypotheses about the galactose metabolism in S. cerevisiae. Hypotheses as well as the supporting or refuting data are represented in RDF and directly linked to one another allowing scientists to browse from data to hypothesis and vice versa. HyQue hypotheses and data are available at http://semanticscience.org/projects/hyque.
| S-EPMC3102892 | biostudies-literature
Project description:The Semantically Annotated Media (SAM) project aims to provide a flexible platform for searching, browsing, and indexing medical learning objects (MLOs) based on a semantic network derived from established classification systems. Primarily, SAM supports the Aachen emedia skills lab, but SAM is ready for indexing distributed content and the Simple Knowledge Organizing System standard provides a means for easily upgrading or even exchanging SAM's semantic network. There is a lack of research addressing the usability of MLO indexes or search portals like SAM and the user behavior with such platforms.The purpose of this study was to assess the usability of SAM by investigating characteristic user behavior of medical students accessing MLOs via SAM.In this study, we chose a mixed-methods approach. Lean usability testing was combined with usability inspection by having the participants complete four typical usage scenarios before filling out a questionnaire. The questionnaire was based on the IsoMetrics usability inventory. Direct user interaction with SAM (mouse clicks and pages accessed) was logged.The study analyzed the typical usage patterns and habits of students using a semantic network for accessing MLOs. Four scenarios capturing characteristics of typical tasks to be solved by using SAM yielded high ratings of usability items and showed good results concerning the consistency of indexing by different users. Long-tail phenomena emerge as they are typical for a collaborative Web 2.0 platform. Suitable but nonetheless rarely used keywords were assigned to MLOs by some users.It is possible to develop a Web-based tool with high usability and acceptance for indexing and retrieval of MLOs. SAM can be applied to indexing multicentered repositories of MLOs collaboratively.
| S-EPMC5041367 | biostudies-other
Project description:Program code has recently become a valuable active data source for training various data science models, from code classification to controlled code synthesis. Annotating code snippets play an essential role in such tasks. This article presents a novel approach that leverages CodeBERT, a powerful transformer-based model, to classify code snippets extracted from Code4ML automatically. Code4ML is a comprehensive machine learning code corpus compiled from Kaggle, a renowned data science competition platform. The corpus includes code snippets and information about the respective kernels and competitions, but it is limited in the quality of the tagged data, which is ~0.2%. Our method addresses the lack of labeled snippets for supervised model training by exploiting the internal ambiguity in particular labeled snippets where multiple class labels are combined. Using a specially designed algorithm, we effectively separate these ambiguous fragments, thereby expanding the pool of training data. This data augmentation approach greatly increases the amount of labeled data and improves the overall quality of the trained models. The experimental results demonstrate the prowess of the proposed code classifier, achieving an impressive F1 test score of ~89%. This achievement not only enhances the practicality of CodeBERT for classifying code snippets but also highlights the importance of enriching large-scale annotated machine learning code datasets such as Code4ML. With a significant increase in accurately annotated code snippets, Code4ML is becoming an even more valuable resource for learning and improving various data processing models.
| S-EPMC10703005 | biostudies-literature
Project description:Total kidney volume (TKV) is the main imaging biomarker used to monitor disease progression and to classify patients affected by autosomal dominant polycystic kidney disease (ADPKD) for clinical trials. However, patients with similar TKVs may have drastically different cystic presentations and phenotypes. In an effort to quantify these cystic differences, we developed the first 3D semantic instance cyst segmentation algorithm for kidneys in MR images. We have reformulated both the object detection/localization task and the instance-based segmentation task into a semantic segmentation task. This allowed us to solve this unique imaging problem efficiently, even for patients with thousands of cysts. To do this, a convolutional neural network (CNN) was trained to learn cyst edges and cyst cores. Images were converted from instance cyst segmentations to semantic edge-core segmentations by applying a 3D erosion morphology operator to up-sampled versions of the images. The reduced cysts were labeled as core; the eroded areas were dilated in 2D and labeled as edge. The network was trained on 30 MR images and validated on 10 MR images using a fourfold cross-validation procedure. The final ensemble model was tested on 20 MR images not seen during the initial training/validation. The results from the test set were compared to segmentations from two readers. The presented model achieved an averaged R2 value of 0.94 for cyst count, 1.00 for total cyst volume, 0.94 for cystic index, and an averaged Dice coefficient of 0.85. These results demonstrate the feasibility of performing cyst segmentations automatically in ADPKD patients.
| S-EPMC8455788 | biostudies-literature
Project description:IntroductionWith the increasing number of Covid-19 cases as well as care costs, chest diseases have gained increasing interest in several communities, particularly in medical and computer vision. Clinical and analytical exams are widely recognized techniques for diagnosing and handling Covid-19 cases. However, strong detection tools can help avoid damage to chest tissues. The proposed method provides an important way to enhance the semantic segmentation process using combined potential deep learning (DL) modules to increase consistency. Based on Covid-19 CT images, this work hypothesized that a novel model for semantic segmentation might be able to extract definite graphical features of Covid-19 and afford an accurate clinical diagnosis while optimizing the classical test and saving time.MethodsCT images were collected considering different cases (normal chest CT, pneumonia, typical viral causes, and Covid-19 cases). The study presents an advanced DL method to deal with chest semantic segmentation issues. The approach employs a modified version of the U-net to enable and support Covid-19 detection from the studied images.ResultsThe validation tests demonstrated competitive results with important performance rates: Precision (90.96% ± 2.5) with an F-score of (91.08% ± 3.2), an accuracy of (93.37% ± 1.2), a sensitivity of (96.88% ± 2.8) and a specificity of (96.91% ± 2.3). In addition, the visual segmentation results are very close to the Ground truth.ConclusionThe findings of this study reveal the proof-of-principle for using cooperative components to strengthen the semantic segmentation modules for effective and truthful Covid-19 diagnosis.Implications for practiceThis paper has highlighted that DL based approach, with several modules, may be contributing to provide strong support for radiographers and physicians, and that further use of DL is required to design and implement performant automated vision systems to detect chest diseases.
| S-EPMC9595354 | biostudies-literature
Project description:A major obstacle for reusing and integrating existing data is finding the data that is most relevant in a given context. The primary metadata resource is the scientific literature describing the experiments that produced the data. To stimulate the development of natural language processing methods for extracting this information from articles, we have manually annotated 100 recent open access publications in Analytical Chemistry as semantic graphs. We focused on articles mentioning mass spectrometry in their experimental sections, as we are particularly interested in the topic, which is also within the domain of several ontologies and controlled vocabularies. The resulting gold standard dataset is publicly available and directly applicable to validating automated methods for retrieving this metadata from the literature. In the process, we also made a number of observations on the structure and description of experiments and open access publication in this journal.
| S-EPMC9647698 | biostudies-literature
Project description:MotivationRadiologists have used algorithms for Computer-Aided Diagnosis (CAD) for decades. These algorithms use machine learning with engineered features, and there have been mixed findings on whether they improve radiologists' interpretations. Deep learning offers superior performance but requires more training data and has not been evaluated in joint algorithm-radiologist decision systems.ResultsWe developed the Computer-Aided Note and Diagnosis Interface (CANDI) for collaboratively annotating radiographs and evaluating how algorithms alter human interpretation. The annotation app collects classification, segmentation, and image captioning training data, and the evaluation app randomizes the availability of CAD tools to facilitate clinical trials on radiologist enhancement.Availability and implementationDemonstrations and source code are hosted at (https://candi.nextgenhealthcare.org), and (https://github.com/mbadge/candi), respectively, under GPL-3 license.Supplementary informationSupplementary material is available at Bioinformatics online.
| S-EPMC6499410 | biostudies-literature