Combining External Medical Knowledge for Improving Obstetric Intelligent Diagnosis: Model Development and Validation.
Ontology highlight
ABSTRACT: Background
Data-driven medical health information processing has become a new development trend in obstetrics. Electronic medical records (EMRs) are the basis of evidence-based medicine and an important information source for intelligent diagnosis. To obtain diagnostic results, doctors combine clinical experience and medical knowledge in their diagnosis process. External medical knowledge provides strong support for diagnosis. Therefore, it is worth studying how to make full use of EMRs and medical knowledge in intelligent diagnosis.Objective
This study aims to improve the performance of intelligent diagnosis in EMRs by combining medical knowledge.Methods
As an EMR usually contains multiple types of diagnostic results, the intelligent diagnosis can be treated as a multilabel classification task. We propose a novel neural network knowledge-aware hierarchical diagnosis model (KHDM) in which Chinese obstetric EMRs and external medical knowledge can be synchronously and effectively used for intelligent diagnostics. In KHDM, EMRs and external knowledge documents are integrated by the attention mechanism contained in the hierarchical deep learning framework. In this way, we enrich the language model with curated knowledge documents, combining the advantages of both to make a knowledge-aware diagnosis.Results
We evaluate our model on a real-world Chinese obstetric EMR dataset and showed that KHDM achieves an accuracy of 0.8929, which exceeds that of the most advanced classification benchmark methods. We also verified the model's interpretability advantage.Conclusions
In this paper, an improved model combining medical knowledge and an attention mechanism is proposed, based on the problem of diversity of diagnostic results in Chinese EMRs. KHDM can effectively integrate domain knowledge to greatly improve the accuracy of diagnosis.
SUBMITTER: Zhang K
PROVIDER: S-EPMC8145091 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectivesAn obstetric comorbidity index has been developed recently with superior performance characteristics relative to general comorbidity measures in an obstetric population. This study aimed to externally validate this index and to examine the impact of including hospitalisation/delivery records only when estimating comorbidity prevalence and discriminative performance of the obstetric comorbidity index.DesignValidation study.SettingAlberta, Canada.PopulationPregnant women who delivered a live or stillborn infant in hospital (n = 5995).MethodsAdministrative databases were linked to create a population-based cohort. Comorbid conditions were identified from diagnoses for the delivery hospitalisation, all hospitalisations and all healthcare contacts (i.e. hospitalisations, emergency room visits and physician visits) that occurred during pregnancy and 3 months pre-conception. Logistic regression was used to test the discriminative performance of the comorbidity index.Main outcome measuresMaternal end-organ damage and extended length of stay for delivery.ResultsAlthough prevalence estimates for comorbid conditions were consistently lower in delivery records and hospitalisation data than in data for all healthcare contacts, the discriminative performance of the comorbidity index was constant for maternal end-organ damage [all healthcare contacts area under the receiver operating characteristic curve (AUC) = 0.70; hospitalisation data AUC = 0.67; delivery data AUC = 0.65] and extended length of stay for delivery (all healthcare contacts AUC = 0.60; hospitalisation data AUC = 0.58; delivery data AUC = 0.58).ConclusionsThe obstetric comorbidity index shows similar performance characteristics in an external population and is a valid measure of comorbidity in an obstetric population. Furthermore, the discriminative performance of the comorbidity index was similar for comorbidities ascertained at the time of delivery, in hospitalisation data or through all healthcare contacts.
| S-EPMC5006847 | biostudies-literature
Project description:BackgroundBenefiting from big data, powerful computation and new algorithmic techniques, we have been witnessing the renaissance of deep learning, particularly the combination of natural language processing (NLP) and deep neural networks. The advent of electronic medical records (EMRs) has not only changed the format of medical records but also helped users to obtain information faster. However, there are many challenges regarding researching directly using Chinese EMRs, such as low quality, huge quantity, imbalance, semi-structure and non-structure, particularly the high density of the Chinese language compared with English. Therefore, effective word segmentation, word representation and model architecture are the core technologies in the literature on Chinese EMRs.ResultsIn this paper, we propose a deep learning framework to study intelligent diagnosis using Chinese EMR data, which incorporates a convolutional neural network (CNN) into an EMR classification application. The novelty of this paper is reflected in the following: (1) We construct a pediatric medical dictionary based on Chinese EMRs. (2) Word2vec adopted in word embedding is used to achieve the semantic description of the content of Chinese EMRs. (3) A fine-tuning CNN model is constructed to feed the pediatric diagnosis with Chinese EMR data. Our results on real-world pediatric Chinese EMRs demonstrate that the average accuracy and F1-score of the CNN models are up to 81%, which indicates the effectiveness of the CNN model for the classification of EMRs. Particularly, a fine-tuning one-layer CNN performs best among all CNNs, recurrent neural network (RNN) (long short-term memory, gated recurrent unit) and CNN-RNN models, and the average accuracy and F1-score are both up to 83%.ConclusionThe CNN framework that includes word segmentation, word embedding and model training can serve as an intelligent auxiliary diagnosis tool for pediatricians. Particularly, a fine-tuning one-layer CNN performs well, which indicates that word order does not appear to have a useful effect on our Chinese EMRs.
| S-EPMC6359854 | biostudies-literature
Project description:Ovarian cancer, a group of heterogeneous diseases, presents with extensive characteristics with the highest mortality among gynecological malignancies. Accurate and early diagnosis of ovarian cancer is of great significance. Here, we present OvcaFinder, an interpretable model constructed from ultrasound images-based deep learning (DL) predictions, Ovarian-Adnexal Reporting and Data System scores from radiologists, and routine clinical variables. OvcaFinder outperforms the clinical model and the DL model with area under the curves (AUCs) of 0.978, and 0.947 in the internal and external test datasets, respectively. OvcaFinder assistance led to improved AUCs of radiologists and inter-reader agreement. The average AUCs were improved from 0.927 to 0.977 and from 0.904 to 0.941, and the false positive rates were decreased by 13.4% and 8.3% in the internal and external test datasets, respectively. This highlights the potential of OvcaFinder to improve the diagnostic accuracy, and consistency of radiologists in identifying ovarian cancer.
| S-EPMC10973484 | biostudies-literature
Project description:The major challenge in the diagnosis of disseminated intravascular coagulation (DIC) comes from the lack of specific biomarkers, leading to developing composite scoring systems. DIC scores are simple and rapidly applicable. However, optimal fibrin-related markers and their cut-off values remain to be defined, requiring optimization for use. The aim of this study is to optimize the use of DIC-related parameters through machine learning (ML)-approach. Further, we evaluated whether this approach could provide a diagnostic value in DIC diagnosis. For this, 46 DIC-related parameters were investigated for both clinical findings and laboratory results. We retrospectively reviewed 656 DIC-suspected cases at an initial order for full DIC profile and labeled their evaluation results (Set 1; DIC, n = 228; non-DIC, n = 428). Several ML algorithms were tested, and an artificial neural network (ANN) model was established via independent training and testing using 32 selected parameters. This model was externally validated from a different hospital with 217 DIC-suspected cases (Set 2; DIC, n = 80; non-DIC, n = 137). The ANN model represented higher AUC values than the three scoring systems in both set 1 (ANN 0.981; ISTH 0.945; JMHW 0.943; and JAAM 0.928) and set 2 (AUC ANN 0.968; ISTH 0.946). Additionally, the relative importance of the 32 parameters was evaluated. Most parameters had contextual importance, however, their importance in ML-approach was different from the traditional scoring system. Our study demonstrates that ML could optimize the use of clinical parameters with robustness for DIC diagnosis. We believe that this approach could play a supportive role in physicians' medical decision by integrated into electrical health record system. Further prospective validation is required to assess the clinical consequence of ML-approach and their clinical benefit.
| S-EPMC5931474 | biostudies-literature
Project description:PurposeAccurate assessment of a molecular classifier that guides patient care is of paramount importance in precision oncology. Recent years have seen an increasing use of external validation for such assessment. However, little is known about how it is affected by ubiquitous unwanted variations in test data because of disparate experimental handling and by the use of data normalization for alleviating such variations.MethodsIn this paper, we studied these issues using two microarray data sets for the same set of tumor samples and additional data simulated by resampling under various levels of signal-to-noise ratio and different designs for array-to-sample allocation.ResultsWe showed that (1) unwanted variations can lead to biased classifier assessment and (2) data normalization mitigates the bias to varying extents depending on the specific method used. In particular, frozen normalization methods for test data outperform their conventional forms in terms of both reducing the bias in accuracy estimation and increasing robustness to handling effects. We make available our benchmarking tool as an R package on GitHub for performing such evaluation on additional methods for normalization and classification.ConclusionOur findings thus highlight the importance of proper test-data normalization for valid assessment by external validation and call for caution on the choice of normalization method for molecular classifier development.
| S-EPMC8345919 | biostudies-literature
Project description:BackgroundPrognostic stratification is the cornerstone of management in nonmetastatic prostate cancer (PCa). However, existing prognostic models are inadequate-often using treatment outcomes rather than survival, stratifying by broad heterogeneous groups and using heavily treated cohorts. To address this unmet need, we developed an individualised prognostic model that contextualises PCa-specific mortality (PCSM) against other cause mortality, and estimates the impact of treatment on survival.Methods and findingsUsing records from the United Kingdom National Cancer Registration and Analysis Service (NCRAS), data were collated for 10,089 men diagnosed with nonmetastatic PCa between 2000 and 2010 in Eastern England. Median follow-up was 9.8 years with 3,829 deaths (1,202 PCa specific). Totals of 19.8%, 14.1%, 34.6%, and 31.5% of men underwent conservative management, prostatectomy, radiotherapy (RT), and androgen deprivation monotherapy, respectively. A total of 2,546 men diagnosed in Singapore over a similar time period represented an external validation cohort. Data were randomly split 70:30 into model development and validation cohorts. Fifteen-year PCSM and non-PCa mortality (NPCM) were explored using separate multivariable Cox models within a competing risks framework. Fractional polynomials (FPs) were utilised to fit continuous variables and baseline hazards. Model accuracy was assessed by discrimination and calibration using the Harrell C-index and chi-squared goodness of fit, respectively, within both validation cohorts. A multivariable model estimating individualised 10- and 15-year survival outcomes was constructed combining age, prostate-specific antigen (PSA), histological grade, biopsy core involvement, stage, and primary treatment, which were each independent prognostic factors for PCSM, and age and comorbidity, which were prognostic for NPCM. The model demonstrated good discrimination, with a C-index of 0.84 (95% CI: 0.82-0.86) and 0.84 (95% CI: 0.80-0.87) for 15-year PCSM in the UK and Singapore validation cohorts, respectively, comparing favourably to international risk-stratification criteria. Discrimination was maintained for overall mortality, with C-index 0.77 (95% CI: 0.75-0.78) and 0.76 (95% CI: 0.73-0.78). The model was well calibrated with no significant difference between predicted and observed PCa-specific (p = 0.19) or overall deaths (p = 0.43) in the UK cohort. Key study limitations were a relatively small external validation cohort, an inability to account for delayed changes to treatment beyond 12 months, and an absence of tumour-stage subclassifications.Conclusions'PREDICT Prostate' is an individualised multivariable PCa prognostic model built from baseline diagnostic information and the first to our knowledge that models potential treatment benefits on overall survival. Prognostic power is high despite using only routinely collected clinicopathological information.
| S-EPMC6413892 | biostudies-literature
Project description:BackgroundThe diagnosis of neurosyphilis is challenging due to the requirement of a lumbar puncture and cerebrospinal fluid (CSF) laboratory tests. Therefore, a convenient diagnostic nomogram for neurosyphilis is warranted. This study aimed to construct diagnostic models for diagnosing neurosyphilis.MethodsThis cross-sectional study included data of two patient cohorts from Western China Hospital of Sichuan University between September 2015 and April 2021 and Shangjin Hospital between September 2019 and April 2021 as the development cohort and the external validation cohort, respectively. A diagnostic model using logistic regression analysis was constructed to readily provide the probability of diagnosis at point of care and presented as a nomogram. The clinical usefulness of the diagnostic models was assessed using a receiver operating characteristic (ROC) and Harrell concordance (Harrell C) index for discrimination and calibration plots for accuracy, which adopted bootstrap resampling 500 times.ResultsOne hundred forty-eight and 67 patients were included in the development and validation cohorts, respectively. Of those, 131 were diagnosed as having reactive neurosyphilis under the criteria of positive results in both CSF treponemal and non-treponemal tests. In the development cohort, male, psychiatric behaviour disorders, and serum toluidine red unheated serum test were selected as diagnostic indicators applying a stepwise procedure in multivariable logistic model. The model reached 80% specificity, 79% sensitivity, and 0·85 area under the curves (AUC) (95% confidence interval, 0·76-0·91). In the validation cohorts, the Harrell C index for the diagnostic possibility of reactive neurosyphilis was 0·71.ConclusionsA convenient model using gender, presence of psychiatric behaviour disorders, and serum TRUST titre was developed and validated to indicate diagnostic results in patients suspected of neurosyphilis. Checking the model value of factors on nomogram is a feasible way to assist clinicians and primary health servers in updating patients' medical charts and making a quantitatively informed decision on neurosyphilis diagnosis.Trial registrationThis research was retrospectively registered in the Ethics committee on biomedical research, West China Hospital of Sichuan University. The research registration and committee's reference number was 1163 in 2020 approval.
| S-EPMC8600785 | biostudies-literature
Project description:BACKGROUND:A number of first-trimester prediction models addressing important obstetric outcomes have been published. However, most models have not been externally validated. External validation is essential before implementing a prediction model in clinical practice. OBJECTIVE:The objective of this paper is to describe the design of a study to externally validate existing first trimester obstetric prediction models, based upon maternal characteristics and standard measurements (eg, blood pressure), for the risk of pre-eclampsia (PE), gestational diabetes mellitus (GDM), spontaneous preterm birth (PTB), small-for-gestational-age (SGA) infants, and large-for-gestational-age (LGA) infants among Dutch pregnant women (Expect Study I). The results of a pilot study on the feasibility and acceptability of the recruitment process and the comprehensibility of the Pregnancy Questionnaire 1 are also reported. METHODS:A multicenter prospective cohort study was performed in The Netherlands between July 1, 2013 and December 31, 2015. First trimester obstetric prediction models were systematically selected from the literature. Predictor variables were measured by the Web-based Pregnancy Questionnaire 1 and pregnancy outcomes were established using the Postpartum Questionnaire 1 and medical records. Information about maternal health-related quality of life, costs, and satisfaction with Dutch obstetric care was collected from a subsample of women. A pilot study was carried out before the official start of inclusion. External validity of the models will be evaluated by assessing discrimination and calibration. RESULTS:Based on the pilot study, minor improvements were made to the recruitment process and online Pregnancy Questionnaire 1. The validation cohort consists of 2614 women. Data analysis of the external validation study is in progress. CONCLUSIONS:This study will offer insight into the generalizability of existing, non-invasive first trimester prediction models for various obstetric outcomes in a Dutch obstetric population. An impact study for the evaluation of the best obstetric prediction models in the Dutch setting with respect to their effect on clinical outcomes, costs, and quality of life-Expect Study II-is being planned. TRIAL REGISTRATION:Netherlands Trial Registry (NTR): NTR4143; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4143 (Archived by WebCite at http://www.webcitation.org/6t8ijtpd9).
| S-EPMC5680517 | biostudies-literature
Project description:Machine learning promises to revolutionize clinical decision making and diagnosis. In medical diagnosis a doctor aims to explain a patient's symptoms by determining the diseases causing them. However, existing machine learning approaches to diagnosis are purely associative, identifying diseases that are strongly correlated with a patients symptoms. We show that this inability to disentangle correlation from causation can result in sub-optimal or dangerous diagnoses. To overcome this, we reformulate diagnosis as a counterfactual inference task and derive counterfactual diagnostic algorithms. We compare our counterfactual algorithms to the standard associative algorithm and 44 doctors using a test set of clinical vignettes. While the associative algorithm achieves an accuracy placing in the top 48% of doctors in our cohort, our counterfactual algorithm places in the top 25% of doctors, achieving expert clinical accuracy. Our results show that causal reasoning is a vital missing ingredient for applying machine learning to medical diagnosis.
| S-EPMC7419549 | biostudies-literature
Project description:Background: A comorbidity summary score may support early and systematic identification of women at high risk for adverse obstetric outcomes. The objective of this study was to conduct the initial development and validation of an obstetrics comorbidity risk score for automated implementation in the electronic health record (EHR) for clinical use. Methods: The score was developed and validated using EHR data for a retrospective cohort of pregnancies with delivery between 2010 and 2018 at Kaiser Permanente Northern California, an integrated health care system. The outcome used for model development consisted of adverse obstetric events from delivery hospitalization (e.g., eclampsia, hemorrhage, death). Candidate predictors included maternal age, parity, multiple gestation, and any maternal diagnoses assigned in health care encounters in the 12 months before admission for delivery. We used penalized regression for variable selection, logistic regression to fit the model, and internal validation for model evaluation. We also evaluated prenatal model performance at 18 weeks of pregnancy. Results: The development cohort (n = 227,405 pregnancies) had an outcome rate of 3.8% and the validation cohort (n = 41,683) had an outcome rate of 2.9%. Of 276 candidate predictors, 37 were included in the final model. The final model had a validation c-statistic of 0.72 (95% confidence interval [CI] 0.70-0.73). When evaluated at 18 weeks of pregnancy, discrimination was modestly diminished (c-statistic 0.68 [95% CI 0.67-0.70]). Conclusions: The obstetric comorbidity score demonstrated good discrimination for adverse obstetric outcomes. After additional appropriate validation, the score can be automated in the EHR to support early identification of high-risk women and assist efforts to ensure risk-appropriate maternal care.
| S-EPMC8617587 | biostudies-literature