Dataset Information

Categorizing metadata to help mobilize computable biomedical knowledge.

ABSTRACT:

Introduction

Computable biomedical knowledge artifacts (CBKs) are digital objects conveying biomedical knowledge in machine-interpretable structures. As more CBKs are produced and their complexity increases, the value obtained from sharing CBKs grows. Mobilizing CBKs and sharing them widely can only be achieved if the CBKs are findable, accessible, interoperable, reusable, and trustable (FAIR+T). To help mobilize CBKs, we describe our efforts to outline metadata categories to make CBKs FAIR+T.

Methods

We examined the literature regarding metadata with the potential to make digital artifacts FAIR+T. We also examined metadata available online today for actual CBKs of 12 different types. With iterative refinement, we came to a consensus on key categories of metadata that, when taken together, can make CBKs FAIR+T. We use subject-predicate-object triples to more clearly differentiate metadata categories.

Results

We defined 13 categories of CBK metadata most relevant to making CBKs FAIR+T. Eleven of these categories (type, domain, purpose, identification, location, CBK-to-CBK relationships, technical, authorization and rights management, provenance, evidential basis, and evidence from use metadata) are evident today where CBKs are stored online. Two additional categories (preservation and integrity metadata) were not evident in our examples. We provide a research agenda to guide further study and development of these and other metadata categories.

Conclusion

A wide variety of metadata elements in various categories is needed to make CBKs FAIR+T. More work is needed to develop a common framework for CBK metadata that can make CBKs FAIR+T for all stakeholders.

SUBMITTER: Alper BS

PROVIDER: S-EPMC8753304 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionWhile data repositories are well-established in clinical and research enterprises, knowledge repositories with shareable computable biomedical knowledge (CBK) are relatively new entities to the digital health ecosystem. Trustworthy knowledge repositories are necessary for learning health systems, but the policies, standards, and practices to promote trustworthy CBK artifacts and methods to share, and safely and effectively use them are not well studied.MethodsWe conducted an online survey of 24 organizations in the United States known to be involved in the development or deployment of CBK. The aim of the survey was to assess the current policies and practices governing these repositories and to identify best practices. Descriptive statistics methods were applied to data from 13 responding organizations, to identify common practices and policies instantiating the TRUST principles of Transparency, Responsibility, User Focus, Sustainability, and Technology.ResultsAll 13 respondents indicated to different degrees adherence to policies that convey TRUST. Transparency is conveyed by having policies pertaining to provenance, credentialed contributors, and provision of metadata. Repositories provide knowledge in machine-readable formats, include implementation guidelines, and adhere to standards to convey Responsibility. Repositories report having Technology functions that enable end-users to verify, search, and filter for knowledge products. Less common TRUST practices are User Focused procedures that enable consumers to know about user licensing requirements or query the use of knowledge artifacts. Related to Sustainability, less than a majority post describe their sustainability plans. Few organizations publicly describe whether patients play any role in their decision-making.ConclusionIt is essential that knowledge repositories identify and apply a baseline set of criteria to lay a robust foundation for their trustworthiness leading to optimum uptake, and safe, reliable, and effective use to promote sharing of CBK. Identifying current practices suggests a set of desiderata for the CBK ecosystem in its continued evolution.

Project description:BackgroundOntology term labels can be ambiguous and have multiple senses. While this is no problem for human annotators, it is a challenge to automated methods, which identify ontology terms in text. Classical approaches to word sense disambiguation use co-occurring words or terms. However, most treat ontologies as simple terminologies, without making use of the ontology structure or the semantic similarity between terms. Another useful source of information for disambiguation are metadata. Here, we systematically compare three approaches to word sense disambiguation, which use ontologies and metadata, respectively.ResultsThe 'Closest Sense' method assumes that the ontology defines multiple senses of the term. It computes the shortest path of co-occurring terms in the document to one of these senses. The 'Term Cooc' method defines a log-odds ratio for co-occurring terms including co-occurrences inferred from the ontology structure. The 'MetaData' approach trains a classifier on metadata. It does not require any ontology, but requires training data, which the other methods do not. To evaluate these approaches we defined a manually curated training corpus of 2600 documents for seven ambiguous terms from the Gene Ontology and MeSH. All approaches over all conditions achieve 80% success rate on average. The 'MetaData' approach performed best with 96%, when trained on high-quality data. Its performance deteriorates as quality of the training data decreases. The 'Term Cooc' approach performs better on Gene Ontology (92% success) than on MeSH (73% success) as MeSH is not a strict is-a/part-of, but rather a loose is-related-to hierarchy. The 'Closest Sense' approach achieves on average 80% success rate.ConclusionMetadata is valuable for disambiguation, but requires high quality training data. Closest Sense requires no training, but a large, consistently modelled ontology, which are two opposing conditions. Term Cooc achieves greater 90% success given a consistently modelled ontology. Overall, the results show that well structured ontologies can play a very important role to improve disambiguation.AvailabilityThe three benchmark datasets created for the purpose of disambiguation are available in Additional file 1.