Qualitative analysis of manual annotations of clinical text with SNOMED CT.
Ontology highlight
ABSTRACT:
SUBMITTER: Minarro-Gimenez JA
PROVIDER: S-EPMC6307753 | biostudies-literature | 2018
REPOSITORIES: biostudies-literature
Publications
Qualitative analysis of manual annotations of clinical text with SNOMED CT.
Miñarro-Giménez Jose Antonio JA Martínez-Costa Catalina C Karlsson Daniel D Schulz Stefan S Gøeg Kirstine Rosenbeck KR
PloS one 20181227 12
SNOMED CT provides about 300,000 codes with fine-grained concept definitions to support interoperability of health data. Coding clinical texts with medical terminologies it is not a trivial task and is prone to disagreements between coders. We conducted a qualitative analysis to identify sources of disagreements on an annotation experiment which used a subset of SNOMED CT with some restrictions. A corpus of 20 English clinical text fragments from diverse origins and languages was annotated indep ...[more]
Similar Datasets
Project description:BackgroundInteroperability and secondary use of data is a challenge in health care. Specifically, the reuse of clinical free text remains an unresolved problem. The Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) has become the universal language of health care and presents characteristics of a natural language. Its use to represent clinical free text could constitute a solution to improve interoperability.ObjectiveAlthough the use of SNOMED and SNOMED CT has already been reviewed, its specific use in processing and representing unstructured data such as clinical free text has not. This review aims to better understand SNOMED CT's use for representing free text in medicine.MethodsA scoping review was performed on the topic by searching MEDLINE, Embase, and Web of Science for publications featuring free-text processing and SNOMED CT. A recursive reference review was conducted to broaden the scope of research. The review covered the type of processed data, the targeted language, the goal of the terminology binding, the method used and, when appropriate, the specific software used.ResultsIn total, 76 publications were selected for an extensive study. The language targeted by publications was 91% (n=69) English. The most frequent types of documents for which the terminology was used are complementary exam reports (n=18, 24%) and narrative notes (n=16, 21%). Mapping to SNOMED CT was the final goal of the research in 21% (n=16) of publications and a part of the final goal in 33% (n=25). The main objectives of mapping are information extraction (n=44, 39%), feature in a classification task (n=26, 23%), and data normalization (n=23, 20%). The method used was rule-based in 70% (n=53) of publications, hybrid in 11% (n=8), and machine learning in 5% (n=4). In total, 12 different software packages were used to map text to SNOMED CT concepts, the most frequent being Medtex, Mayo Clinic Vocabulary Server, and Medical Text Extraction Reasoning and Mapping System. Full terminology was used in 64% (n=49) of publications, whereas only a subset was used in 30% (n=23) of publications. Postcoordination was proposed in 17% (n=13) of publications, and only 5% (n=4) of publications specifically mentioned the use of the compositional grammar.ConclusionsSNOMED CT has been largely used to represent free-text data, most frequently with rule-based approaches, in English. However, currently, there is no easy solution for mapping free text to this terminology and to perform automatic postcoordination. Most solutions conceive SNOMED CT as a simple terminology rather than as a compositional bag of ontologies. Since 2012, the number of publications on this subject per year has decreased. However, the need for formal semantic representation of free text in health care is high, and automatic encoding into a compositional ontology could be a solution.
| S-EPMC7872838 | biostudies-literature
Project description:BackgroundThe HL7 Clinical Document Architecture (CDA) constrains the HL7 Reference Information model (RIM) to specify the format of HL7-compliant clinical documents, dubbed CDA documents. The use of clinical terminologies such as SNOMED CT® further improves interoperability as they provide a shared understanding of concepts used in clinical documents. However, despite the use of the RIM and of shared terminologies such as SNOMED CT®, gaps remain as to how to use both the RIM and SNOMED CT® in HL7 clinical documents. The HL7 implementation guide on Using SNOMED CT in HL7 Version 3 is an effort to close this gap. It is, however, a human-readable document that is not suited for automatic processing. As such, health care professionals designing clinical documents need to ensure validity of documents manually.ResultsWe represent the CDA using the Ontology Web Language OWL and further use the OWL version of SNOMED CT® to enable the translation of CDA documents to so-called OWL ontologies. We formalize a subset of the constraints in the implementation guide on Using SNOMED CT in HL7 Version 3 as OWL Integrity Constraints and show that we can automatically validate CDA documents using OWL reasoners such as Pellet. Finally, we evaluate our approach via a prototype implementation that plugs in the Open Health Workbench.ConclusionsWe present a methodology to automatically check the validity of CDA documents which make reference to SNOMED CT® terminology. The methodology relies on semantic technologies such as OWL. As such it removes the burden from IT health care professionals of having to manually implement such guidelines in systems that use HL7 Version 3 documents.
| S-EPMC3152505 | biostudies-literature
Project description:BackgroundData harmonisation is essential in real-world data (RWD) research projects based on hospital information systems databases, as coding systems differ between countries. The Hungarian hospital information systems and the national claims database use internationally known diagnosis codes, but data on medical procedures are recorded using national codes. There is no simple or standard solution for mapping the national codes to a standard coding system. Our aim was to map the Hungarian procedure codes (OENO) to SNOMED CT as part of the European Health Data Evidence Network (EHDEN) project.MethodsWe recruited 25 professionals from different specialties to manually map the procedure codes used between 2011 and 2021. A mapping protocol and training material were developed, results were regularly revised, and the challenges of mapping were recorded. Approximately 7% of the codes were mapped by more people in different specialties for validation purposes.ResultsWe mapped 4661 OENO codes to standard vocabularies, mostly SNOMED CT. We categorized the challenges into three main areas: semantic, matching, and methodological. Semantic refers to the occasionally unclear meaning of the OENO codes, matching to the different granularity and purpose of the OENO and SNOMED CT vocabularies. Lastly, methodological challenges were used to describe issues related to the design of the above-mentioned two vocabularies.ConclusionsThe challenges and solutions presented here may help other researchers to design their process to map their national codes to standard vocabularies in order to achieve greater consistency in mapping results. Moreover, we believe that our work will allow for better use of RWD collected in Hungary in international research collaborations.
| S-EPMC10585817 | biostudies-literature
Project description:Clinicians often include references to diseases in clinical notes, which have not been diagnosed in their patients. For some diseases terms, the majority of disease references written in the patient notes may not refer to true disease diagnosis. These references occur because clinicians often use their clinical notes to speculate about disease existence (differential diagnosis) or to state that the disease has been ruled out. To train classifiers for disambiguating disease references, previous researchers built training sets by manually annotating sentences. We show how to create very large training sets without the need for manual annotation. We obtain state-of- the-art classification performance with a bidirectional long short-term memory model trained to distinguish disease references between patients with or without the disease diagnosis in veterinary clinical notes.
| S-EPMC6550178 | biostudies-literature
Project description:MotivationIn this paper we demonstrate the usage of RIO; a framework for detecting syntactic regularities using cluster analysis of the entities in the signature of an ontology. Quality assurance in ontologies is vital for their use in real applications, as well as a complex and difficult task. It is also important to have such methods and tools when the ontology lacks documentation and the user cannot consult the ontology developers to understand its construction. One aspect of quality assurance is checking how well an ontology complies with established 'coding standards'; is the ontology regular in how descriptions of different types of entities are axiomatised? Is there a similar way to describe them and are there any corner cases that are not covered by a pattern? Detection of regularities and irregularities in axiom patterns should provide ontology authors and quality inspectors with a level of abstraction such that compliance to coding standards can be automated. However, there is a lack of such reverse ontology engineering methods and tools.ResultsRIO framework allows regularities to be detected in an OWL ontology, i.e. repetitive structures in the axioms of an ontology. We describe the use of standard machine learning approaches to make clusters of similar entities and generalise over their axioms to find regularities. This abstraction allows matches to, and deviations from, an ontology's patterns to be shown. We demonstrate its usage with the inspection of three modules from SNOMED-CT, a large medical terminology, that cover "Present" and "Absent" findings, as well as "Chronic" and "Acute" findings. The module sizes are 5 065, 20 688 and 19 812 asserted axioms. They are analysed in terms of their types and number of regularities and irregularities in the asserted axioms of the ontology. The analysis showed that some modules of the terminology, which were expected to instantiate a pattern described in the SNOMED-CT technical guide, were found to have a high number of regularity deviations. A subset of these were categorised as "design defects" by verifying them with past work on the quality assurance of SNOMED-CT. These were mainly incomplete descriptions. In the worst case, the expected patterns described in the technical guide were followed by only 5% of the axioms in the module.ConclusionIt is possible to automatically detect regularities and then inspect irregularities in an ontology. We argue that RIO is a tool to find and report such matches and mismatches, for evaluations by the domain experts. We have demonstrated that standard clustering techniques from machine learning can offer a tool in the drive for quality assurance in ontologies.Availabilityhttp://riotool.sourceforge.net/Contacthttp://eleni.mikroyannidi@manchester.ac.uk, http://robert.stevens@manchehster.ac.uk.
| S-EPMC3637289 | biostudies-literature
Project description:OBJECTIVE: The aim of this paper is to report on the use of the systematised nomenclature of medicine clinical terms (SNOMED CT) by providing an overview of published papers. METHODS: Published papers on SNOMED CT between 2001 and 2012 were identified using PubMed and Embase databases using the keywords 'systematised nomenclature of medicine' and 'SNOMED CT'. For each paper the following characteristics were retrieved: SNOMED CT focus category (ie, indeterminate, theoretical, pre-development/design, implementation and evaluation/commodity), usage category (eg, prospective content coverage, used to classify or code in a study), medical domain and country. RESULTS: Our search strategy identified 488 papers. A comparison between the papers published between 2001-6 and 2007-12 showed an increase in every SNOMED CT focus category. The number of papers classified as 'theoretical' increased from 46 to 78, 'pre-development/design' increased from 61 to 173 and 'implementation' increased from 10 to 34. Papers classified as 'evaluation/commodity' only started to appear from 2010. CONCLUSIONS: The majority of studies focused on 'theoretical' and 'pre-development/design'. This is still encouraging as SNOMED CT is being harmonized with other standardized terminologies and is being evaluated to determine the content coverage of local terms, which is usually one of the first steps towards adoption. Most implementations are not published in the scientific literature, requiring a look beyond the scientific literature to gain insights into SNOMED CT implementations.
| S-EPMC3957381 | biostudies-other
Project description:BACKGROUND:Defining clinical conditions from electronic health record (EHR) data underpins population health activities, clinical decision support, and analytics. In an EHR, defining a condition commonly employs a diagnosis value set or "grouper." For constructing value sets, Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) offers high clinical fidelity, a hierarchical ontology, and wide implementation in EHRs as the standard interoperability vocabulary for problems. OBJECTIVE:This article demonstrates a practical approach to defining conditions with combinations of SNOMED CT concept hierarchies, and evaluates sharing of definitions for clinical and analytic uses. METHODS:We constructed diagnosis value sets for EHR patient registries using SNOMED CT concept hierarchies combined with Boolean logic, and shared them for clinical decision support, reporting, and analytic purposes. RESULTS:A total of 125 condition-defining "standard" SNOMED CT diagnosis value sets were created within our EHR. The median number of SNOMED CT concept hierarchies needed was only 2 (25th-75th percentiles: 1-5). Each value set, when compiled as an EHR diagnosis grouper, was associated with a median of 22 International Classification of Diseases (ICD)-9 and ICD-10 codes (25th-75th percentiles: 8-85) and yielded a median of 155 clinical terms available for selection by clinicians in the EHR (25th-75th percentiles: 63-976). Sharing of standard groupers for population health, clinical decision support, and analytic uses was high, including 57 patient registries (with 362 uses of standard groupers), 132 clinical decision support records, 190 rules, 124 EHR reports, 125 diagnosis dimension slicers for self-service analytics, and 111 clinical quality measure calculations. Identical SNOMED CT definitions were created in an EHR-agnostic tool enabling application across disparate organizations and EHRs. CONCLUSION:SNOMED CT-based diagnosis value sets are simple to develop, concise, understandable to clinicians, useful in the EHR and for analytics, and shareable. Developing curated SNOMED CT hierarchy-based condition definitions for public use could accelerate cross-organizational population health efforts, "smarter" EHR feature configuration, and clinical-translational research employing EHR-derived data.
| S-EPMC6115233 | biostudies-literature
Project description:ObjectivePatient phenotype definitions based on terminologies are required for the computational use of electronic health records. Within UK primary care research databases, such definitions have typically been represented as flat lists of Read terms, but Systematized Nomenclature of Medicine-Clinical Terms (SNOMED CT) (a widely employed international reference terminology) enables the use of relationships between concepts, which could facilitate the phenotyping process. We implemented SNOMED CT-based phenotyping approaches and investigated their performance in the CPRD Aurum primary care database.Materials and methodsWe developed SNOMED CT phenotype definitions for 3 exemplar diseases: diabetes mellitus, asthma, and heart failure, using 3 methods: "primary" (primary concept and its descendants), "extended" (primary concept, descendants, and additional relations), and "value set" (based on text searches of term descriptions). We also derived SNOMED CT codelists in a semiautomated manner for 276 disease phenotypes used in a study of health across the lifecourse. Cohorts selected using each codelist were compared to "gold standard" manually curated Read codelists in a sample of 500 000 patients from CPRD Aurum.ResultsSNOMED CT codelists selected a similar set of patients to Read, with F1 scores exceeding 0.93, and age and sex distributions were similar. The "value set" and "extended" codelists had slightly greater recall but lower precision than "primary" codelists. We were able to represent 257 of the 276 phenotypes by a single concept hierarchy, and for 135 phenotypes, the F1 score was greater than 0.9.ConclusionsSNOMED CT provides an efficient way to define disease phenotypes, resulting in similar patient populations to manually curated codelists.
| S-EPMC9846670 | biostudies-literature
Project description:ObjectiveThis article reviews recent literature on the use of SNOMED CT as an extension of Lee et al's 2014 review on the same topic. The Lee et al's article covered literature published from 2001-2012, and the scope of this review was 2013-2020.Materials and methodsIn line with Lee et al's methods, we searched the PubMed and Embase databases and identified 1002 articles for review, including studies from January 2013 to September 2020. The retrieved articles were categorized and analyzed according to SNOMED CT focus categories (ie, indeterminate, theoretical, pre-development, implementation, and evaluation/commodity), usage categories (eg, illustrate terminology systems theory, prospective content coverage, used to classify or code in a study, retrieve or analyze patient data, etc.), medical domains, and countries.ResultsAfter applying inclusion and exclusion criteria, 622 articles were selected for final review. Compared to the papers published between 2001 and 2012, papers published between 2013 and 2020 revealed an increase in more mature usage of SNOMED CT, and the number of papers classified in the "implementation" and "evaluation/commodity" focus categories expanded. When analyzed by decade, papers in the "pre-development," "implementation," and "evaluation/commodity" categories were much more numerous in 2011-2020 than in 2001-2010, increasing from 169 to 293, 30 to 138, and 3 to 65, respectively.ConclusionPublished papers in more mature usage categories have substantially increased since 2012. From 2013 to present, SNOMED CT has been increasingly implemented in more practical settings. Future research should concentrate on addressing whether SNOMED CT influences improvement in patient care.
| S-EPMC8363812 | biostudies-literature
Project description:BackgroundGenes and gene products are frequently annotated with Gene Ontology concepts based on the evidence provided in genomics articles. Manually locating and curating information about a genomic entity from the biomedical literature requires vast amounts of human effort. Hence, there is clearly a need forautomated computational tools to annotate the genes and gene products with Gene Ontology concepts by computationally capturing the related knowledge embedded in textual data.ResultsIn this article, we present an automated genomic entity annotation system, GEANN, which extracts information about the characteristics of genes and gene products in article abstracts from PubMed, and translates the discoveredknowledge into Gene Ontology (GO) concepts, a widely-used standardized vocabulary of genomic traits. GEANN utilizes textual "extraction patterns", and a semantic matching framework to locate phrases matching to a pattern and produce Gene Ontology annotations for genes and gene products. In our experiments, GEANN has reached to the precision level of 78% at therecall level of 61%. On a select set of Gene Ontology concepts, GEANN either outperforms or is comparable to two other automated annotation studies. Use of WordNet for semantic pattern matching improves the precision and recall by 24% and 15%, respectively, and the improvement due to semantic pattern matching becomes more apparent as the Gene Ontology terms become more general.ConclusionGEANN is useful for two distinct purposes: (i) automating the annotation of genomic entities with Gene Ontology concepts, and (ii) providing existing annotations with additional "evidence articles" from the literature. The use of textual extraction patterns that are constructed based on the existing annotations achieve high precision. The semantic pattern matching framework provides a more flexible pattern matching scheme with respect to "exactmatching" with the advantage of locating approximate pattern occurrences with similar semantics. Relatively low recall performance of our pattern-based approach may be enhanced either by employing a probabilistic annotation framework based on the annotation neighbourhoods in textual data, or, alternatively, the statistical enrichment threshold may be adjusted to lower values for applications that put more value on achieving higher recall values.
| S-EPMC2335285 | biostudies-literature