Dataset Information

An open access medical knowledge base for community driven diagnostic decision support system development.

ABSTRACT:

Introduction

While early diagnostic decision support systems were built around knowledge bases, more recent systems employ machine learning to consume large amounts of health data. We argue curated knowledge bases will remain an important component of future diagnostic decision support systems by providing ground truth and facilitating explainable human-computer interaction, but that prototype development is hampered by the lack of freely available computable knowledge bases.

Methods

We constructed an open access knowledge base and evaluated its potential in the context of a prototype decision support system. We developed a modified set-covering algorithm to benchmark the performance of our knowledge base compared to existing platforms. Testing was based on case reports from selected literature and medical student preparatory material.

Results

The knowledge base contains over 2000 ICD-10 coded diseases and 450 RX-Norm coded medications, with over 8000 unique observations encoded as SNOMED or LOINC semantic terms. Using 117 medical cases, we found the accuracy of the knowledge base and test algorithm to be comparable to established diagnostic tools such as Isabel and DXplain. Our prototype, as well as DXplain, showed the correct answer as "best suggestion" in 33% of the cases. While we identified shortcomings during development and evaluation, we found the knowledge base to be a promising platform for decision support systems.

Conclusion

We built and successfully evaluated an open access knowledge base to facilitate the development of new medical diagnostic assistants. This knowledge base can be expanded and curated by users and serve as a starting point to facilitate new technology development and system improvement in many contexts.

SUBMITTER: Muller L

PROVIDER: S-EPMC6486985 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Despite rapid evolution in the area of microbial natural products chemistry, there is currently no open access database containing all microbially produced natural product structures. Lack of availability of these data is preventing the implementation of new technologies in natural products science. Specifically, development of new computational strategies for compound characterization and identification are being hampered by the lack of a comprehensive database of known compounds against which to compare experimental data. The creation of an open access, community-maintained database of microbial natural product structures would enable the development of new technologies in natural products discovery and improve the interoperability of existing natural products data resources. However, these data are spread unevenly throughout the historical scientific literature, including both journal articles and international patents. These documents have no standard format, are often not digitized as machine readable text, and are not publicly available. Further, none of these documents have associated structure files (e.g., MOL, InChI, or SMILES), instead containing images of structures. This makes extraction and formatting of relevant natural products data a formidable challenge. Using a combination of manual curation and automated data mining approaches we have created a database of microbial natural products (The Natural Products Atlas, www.npatlas.org) that includes 24 594 compounds and contains referenced data for structure, compound names, source organisms, isolation references, total syntheses, and instances of structural reassignment. This database is accompanied by an interactive web portal that permits searching by structure, substructure, and physical properties. The Web site also provides mechanisms for visualizing natural products chemical space and dashboards for displaying author and discovery timeline data. These interactive tools offer a powerful knowledge base for natural products discovery with a central interface for structure and property-based searching and presents new viewpoints on structural diversity in natural products. The Natural Products Atlas has been developed under FAIR principles (Findable, Accessible, Interoperable, and Reusable) and is integrated with other emerging natural product databases, including the Minimum Information About a Biosynthetic Gene Cluster (MIBiG) repository, and the Global Natural Products Social Molecular Networking (GNPS) platform. It is designed as a community-supported resource to provide a central repository for known natural product structures from microorganisms and is the first comprehensive, open access resource of this type. It is expected that the Natural Products Atlas will enable the development of new natural products discovery modalities and accelerate the process of structural characterization for complex natural products libraries.

Project description:BackgroundContinuously growing medical knowledge and the increasing amount of data make it difficult for medical professionals to keep track of all new information and to place it in the context of existing information. A variety of digital technologies and artificial intelligence-based methods are currently available as persuasive tools to empower physicians in clinical decision-making and improve health care quality. A novel diagnostic decision support system (DDSS) prototype developed by Ada Health GmbH with a focus on traceability, transparency, and usability will be examined more closely in this study.ObjectiveThe aim of this study is to test the feasibility and functionality of a novel DDSS prototype, exploring its potential and performance in identifying the underlying cause of acute dyspnea in patients at the University Hospital Basel.MethodsA prospective, observational feasibility study was conducted at the emergency department (ED) and internal medicine ward of the University Hospital Basel, Switzerland. A convenience sample of 20 adult patients admitted to the ED with dyspnea as the chief complaint and a high probability of inpatient admission was selected. A study physician followed the patients admitted to the ED throughout the hospitalization without interfering with the routine clinical work. Routinely collected health-related personal data from these patients were entered into the DDSS prototype. The DDSS prototype's resulting disease probability list was compared with the gold-standard main diagnosis provided by the treating physician.ResultsThe DDSS presented information with high clarity and had a user-friendly, novel, and transparent interface. The DDSS prototype was not perfectly suited for the ED as case entry was time-consuming (1.5-2 hours per case). It provided accurate decision support in the clinical inpatient setting (average of cases in which the correct diagnosis was the first diagnosis listed: 6/20, 30%, SD 2.10%; average of cases in which the correct diagnosis was listed as one of the top 3: 11/20, 55%, SD 2.39%; average of cases in which the correct diagnosis was listed as one of the top 5: 14/20, 70%, SD 2.26%) in patients with dyspnea as the main presenting complaint.ConclusionsThe study of the feasibility and functionality of the tool was successful, with some limitations. Used in the right place, the DDSS has the potential to support physicians in their decision-making process by showing new pathways and unintentionally ignored diagnoses. The DDSS prototype had some limitations regarding the process of data input, diagnostic accuracy, and completeness of the integrated medical knowledge. The results of this study provide a basis for the tool's further development. In addition, future studies should be conducted with the aim to overcome the current limitations of the tool and study design.Trial registrationClinicalTrials.gov NCT04827342; https://clinicaltrials.gov/ct2/show/NCT04827342.

Project description:BackgroundMoving to electronic health records (EHRs) confers substantial benefits but risks unintended consequences. Modern EHRs consist of complex software code with extensive local configurability options, which can introduce defects. Defects in clinical decision support (CDS) tools are surprisingly common. Feasible approaches to prevent and detect defects in EHR configuration, including CDS tools, are needed. In complex software systems, use of test-driven development and automated regression testing promotes reliability. Test-driven development encourages modular, testable design and expanding regression test coverage. Automated regression test suites improve software quality, providing a "safety net" for future software modifications. Each automated acceptance test serves multiple purposes, as requirements (prior to build), acceptance testing (on completion of build), regression testing (once live), and "living" design documentation. Rapid-cycle development or "agile" methods are being successfully applied to CDS development. The agile practice of automated test-driven development is not widely adopted, perhaps because most EHR software code is vendor-developed. However, key CDS advisory configuration design decisions and rules stored in the EHR may prove amenable to automated testing as "executable requirements."ObjectiveWe aimed to establish feasibility of acceptance test-driven development of clinical decision support advisories in a commonly used EHR, using an open source automated acceptance testing framework (FitNesse).MethodsAcceptance tests were initially constructed as spreadsheet tables to facilitate clinical review. Each table specified one aspect of the CDS advisory's expected behavior. Table contents were then imported into a test suite in FitNesse, which queried the EHR database to automate testing. Tests and corresponding CDS configuration were migrated together from the development environment to production, with tests becoming part of the production regression test suite.ResultsWe used test-driven development to construct a new CDS tool advising Emergency Department nurses to perform a swallowing assessment prior to administering oral medication to a patient with suspected stroke. Test tables specified desired behavior for (1) applicable clinical settings, (2) triggering action, (3) rule logic, (4) user interface, and (5) system actions in response to user input. Automated test suite results for the "executable requirements" are shown prior to building the CDS alert, during build, and after successful build.ConclusionsAutomated acceptance test-driven development and continuous regression testing of CDS configuration in a commercial EHR proves feasible with open source software. Automated test-driven development offers one potential contribution to achieving high-reliability EHR configuration. Vetting acceptance tests with clinicians elicits their input on crucial configuration details early during initial CDS design and iteratively during rapid-cycle optimization.

Project description:Making complex medical decisions is becoming an increasingly challenging task due to the growing amount of available evidence to consider and the higher demand for personalized treatment and patient care. IT systems for the provision of clinical decision support (CDS) can provide sustainable relief if decisions are automatically evaluated and processed. In this paper, we propose an approach for quantifying similarity between new and previously recorded medical cases to enable significant knowledge transfer for reasoning tasks on a patient-level. Methodologically, 102 medical cases with oropharyngeal carcinoma were analyzed retrospectively. Based on independent disease characteristics, patient-specific data vectors including relevant information entities for primary and adjuvant treatment decisions were created. Utilizing the ϕK correlation coefficient as the methodological foundation of our approach, we were able to determine the predictive impact of each characteristic, thus enabling significant reduction of the feature space to allow for further analysis of the intra-variable distances between the respective feature states. The results revealed a significant feature-space reduction from initially 19 down to only 6 diagnostic variables (ϕK correlation coefficient ≥ 0.3, ϕK significance test ≥ 2.5) for the primary and 7 variables (from initially 14) for the adjuvant treatment setting. Further investigation on the resulting characteristics showed a non-linear behavior in relation to the corresponding distances on intra-variable level. Through the implementation of a 10-fold cross-validation procedure, we were further able to identify 8 (primary treatment) matching cases with an evaluation score of 1.0 and 9 (adjuvant treatment) matching cases with an evaluation score of 0.957 based on their shared treatment procedure as the endpoint for similarity definition. Based on those promising results, we conclude that our proposed method for using data-driven similarity measures for application in medical decision-making is able to offer valuable assistance for physicians. Furthermore, we consider our approach as universal in regard to other clinical use-cases, which would allow for an easy-to-implement adaptation for a range of further medical decision-making scenarios.

Project description:Most mobile health (mHealth) decision support systems currently available for chronic obstructive respiratory diseases (CORDs) are not supported by clinical evidence or lack clinical validation. The development of the knowledge base that will feed the clinical decision support system is a crucial step that involves the collection and systematization of clinical knowledge from relevant scientific sources and its representation in a human-understandable and computer-interpretable way. This work describes the development and initial validation of a clinical knowledge base that can be integrated into mHealth decision support systems developed for patients with CORDs. A multidisciplinary team of health care professionals with clinical experience in respiratory diseases, together with data science and IT professionals, defined a new framework that can be used in other evidence-based systems. The knowledge base development began with a thorough review of the relevant scientific sources (eg, disease guidelines) to identify the recommendations to be implemented in the decision support system based on a consensus process. Recommendations were selected according to predefined inclusion criteria: (1) applicable to individuals with CORDs or to prevent CORDs, (2) directed toward patient self-management, (3) targeting adults, and (4) within the scope of the knowledge domains and subdomains defined. Then, the selected recommendations were prioritized according to (1) a harmonized level of evidence (reconciled from different sources); (2) the scope of the source document (international was preferred); (3) the entity that issued the source document; (4) the operability of the recommendation; and (5) health care professionals' perceptions of the relevance, potential impact, and reach of the recommendation. A total of 358 recommendations were selected. Next, the variables required to trigger those recommendations were defined (n=116) and operationalized into logical rules using Boolean logical operators (n=405). Finally, the knowledge base was implemented in an intelligent individualized coaching component and pretested with an asthma use case. Initial validation of the knowledge base was conducted internally using data from a population-based observational study of individuals with or without asthma or rhinitis. External validation of the appropriateness of the recommendations with the highest priority level was conducted independently by 4 physicians. In addition, a strategy for knowledge base updates, including an easy-to-use rules editor, was defined. Using this process, based on consensus and iterative improvement, we developed and conducted preliminary validation of a clinical knowledge base for CORDs that translates disease guidelines into personalized patient recommendations. The knowledge base can be used as part of mHealth decision support systems. This process could be replicated in other clinical areas.

Project description:BackgroundDiagnostic errors pose significant health risks and contribute to patient mortality. With the growing accessibility of electronic health records, machine learning models offer a promising avenue for enhancing diagnosis quality. Current research has primarily focused on a limited set of diseases with ample training data, neglecting diagnostic scenarios with limited data availability.ObjectiveThis study aims to develop an information retrieval (IR)-based framework that accommodates data sparsity to facilitate broader diagnostic decision support.MethodsWe introduced an IR-based diagnostic decision support framework called CliniqIR. It uses clinical text records, the Unified Medical Language System Metathesaurus, and 33 million PubMed abstracts to classify a broad spectrum of diagnoses independent of training data availability. CliniqIR is designed to be compatible with any IR framework. Therefore, we implemented it using both dense and sparse retrieval approaches. We compared CliniqIR's performance to that of pretrained clinical transformer models such as Clinical Bidirectional Encoder Representations from Transformers (ClinicalBERT) in supervised and zero-shot settings. Subsequently, we combined the strength of supervised fine-tuned ClinicalBERT and CliniqIR to build an ensemble framework that delivers state-of-the-art diagnostic predictions.ResultsOn a complex diagnosis data set (DC3) without any training data, CliniqIR models returned the correct diagnosis within their top 3 predictions. On the Medical Information Mart for Intensive Care III data set, CliniqIR models surpassed ClinicalBERT in predicting diagnoses with <5 training samples by an average difference in mean reciprocal rank of 0.10. In a zero-shot setting where models received no disease-specific training, CliniqIR still outperformed the pretrained transformer models with a greater mean reciprocal rank of at least 0.10. Furthermore, in most conditions, our ensemble framework surpassed the performance of its individual components, demonstrating its enhanced ability to make precise diagnostic predictions.ConclusionsOur experiments highlight the importance of IR in leveraging unstructured knowledge resources to identify infrequently encountered diagnoses. In addition, our ensemble framework benefits from combining the complementary strengths of the supervised and retrieval-based models to diagnose a broad spectrum of diseases.