Diagnostic performance of an artificial intelligence-driven cardiac-structured reporting system for myocardial perfusion SPECT imaging.
Ontology highlight
ABSTRACT: Objectives
To describe and validate an artificial intelligence (AI)-driven structured reporting system by direct comparison of automatically generated reports to results from actual clinical reports generated by nuclear cardiology experts.Background
Quantitative parameters extracted from myocardial perfusion imaging (MPI) studies are used by our AI reporting system to generate automatically a guideline-compliant structured report (sR).Method
A new nonparametric approach generates distribution functions of rest and stress, perfusion, and thickening, for each of 17 left ventricle segments that are then transformed to certainty factors (CFs) that a segment is hypoperfused, ischemic. These CFs are then input to our set of heuristic rules used to reach diagnostic findings and impressions propagated into a sR referred as an AI-driven structured report (AIsR). The diagnostic accuracy of the AIsR for detecting coronary artery disease (CAD) and ischemia was tested in 1,000 patients who had undergone rest/stress SPECT MPI.Results
At the high-specificity (SP) level, in a subset of 100 patients, there were no statistical differences in the agreements between the AIsr, and nine experts' impressions of CAD (P = .33) or ischemia (P = .37). This high-SP level also yielded the highest accuracy across global and regional results in the 1,000 patients. These accuracies were statistically significantly better than the other two levels [sensitivity (SN)/SP tradeoff, high SN] across all comparisons.Conclusions
This AI reporting system automatically generates a structured natural language report with a diagnostic performance comparable to those of experts.
SUBMITTER: Garcia EV
PROVIDER: S-EPMC6414293 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundMyocardial perfusion reflects the macro- and microvascular coronary circulation. Recent quantitation developments using cardiovascular magnetic resonance perfusion permit automated measurement clinically. We explored the prognostic significance of stress myocardial blood flow (MBF) and myocardial perfusion reserve (MPR, the ratio of stress to rest MBF).MethodsA 2-center study of patients with both suspected and known coronary artery disease referred clinically for perfusion assessment. Image analysis was performed automatically using a novel artificial intelligence approach deriving global and regional stress and rest MBF and MPR. Cox proportional hazard models adjusting for comorbidities and cardiovascular magnetic resonance parameters sought associations of stress MBF and MPR with death and major adverse cardiovascular events (MACE), including myocardial infarction, stroke, heart failure hospitalization, late (>90 day) revascularization, and death.ResultsA total of 1049 patients were included with a median follow-up of 605 (interquartile range, 464-814) days. There were 42 (4.0%) deaths and 188 MACE in 174 (16.6%) patients. Stress MBF and MPR were independently associated with both death and MACE. For each 1 mL·g-1·min-1 decrease in stress MBF, the adjusted hazard ratios for death and MACE were 1.93 (95% CI, 1.08-3.48, P=0.028) and 2.14 (95% CI, 1.58-2.90, P<0.0001), respectively, even after adjusting for age and comorbidity. For each 1 U decrease in MPR, the adjusted hazard ratios for death and MACE were 2.45 (95% CI, 1.42-4.24, P=0.001) and 1.74 (95% CI, 1.36-2.22, P<0.0001), respectively. In patients without regional perfusion defects on clinical read and no known macrovascular coronary artery disease (n=783), MPR remained independently associated with death and MACE, with stress MBF remaining associated with MACE only.ConclusionsIn patients with known or suspected coronary artery disease, reduced MBF and MPR measured automatically inline using artificial intelligence quantification of cardiovascular magnetic resonance perfusion mapping provides a strong, independent predictor of adverse cardiovascular outcomes.
| S-EPMC7176346 | biostudies-literature
Project description:Purpose:Next-generation sequencing technologies are actively applied in clinical oncology. Bioinformatics pipeline analysis is an integral part of this process; however, humans cannot yet realize the full potential of the highly complex pipeline output. As a result, the decision to include a variant in the final report during routine clinical sign-out remains challenging. Methods:We used an artificial intelligence approach to capture the collective clinical sign-out experience of six board-certified molecular pathologists to build and validate a decision support tool for variant reporting. We extracted all reviewed and reported variants from our clinical database and tested several machine learning models. We used 10-fold cross-validation for our variant call prediction model, which derives a contiguous prediction score from 0 to 1 (no to yes) for clinical reporting. Results:For each of the 19,594 initial training variants, our pipeline generates approximately 500 features, which results in a matrix of > 9 million data points. From a comparison of naive Bayes, decision trees, random forests, and logistic regression models, we selected models that allow human interpretability of the prediction score. The logistic regression model demonstrated 1% false negativity and 2% false positivity. The final models' Youden indices were 0.87 and 0.77 for screening and confirmatory cutoffs, respectively. Retraining on a new assay and performance assessment in 16,123 independent variants validated our approach (Youden index, 0.93). We also derived individual pathologist-centric models (virtual consensus conference function), and a visual drill-down functionality allows assessment of how underlying features contributed to a particular score or decision branch for clinical implementation. Conclusion:Our decision support tool for variant reporting is a practically relevant artificial intelligence approach to harness the next-generation sequencing bioinformatics pipeline output when the complexity of data interpretation exceeds human capabilities.
| S-EPMC6198661 | biostudies-literature
Project description:Background:Free-text sections of pathology reports contain the most important information from a diagnostic standpoint. However, this information is largely underutilized for computer-based analytics. The vast majority of NLP-based methods lack a capacity to accurately extract complex diagnostic entities and relationships among them as well as to provide an adequate knowledge representation for downstream data-mining applications. Methods:In this paper, we introduce a novel informatics pipeline that extends open information extraction (openIE) techniques with artificial intelligence (AI) based modeling to extract and transform complex diagnostic entities and relationships among them into Knowledge Graphs (KGs) of relational triples (RTs). Results:Evaluation studies have demonstrated that the pipeline's output significantly differs from a random process. The semantic similarity with original reports is high (Mean Weighted Overlap of 0.83). The precision and recall of extracted RTs based on experts' assessment were 0.925 and 0.841 respectively (P <0.0001). Inter-rater agreement was significant at 93.6% and inter-rated reliability was 81.8%. Conclusion:The results demonstrated important properties of the pipeline such as high accuracy, minimality and adequate knowledge representation. Therefore, we conclude that the pipeline can be used in various downstream data-mining applications to assist diagnostic medicine.
| S-EPMC7045509 | biostudies-literature
Project description:IntroductionStandards for Reporting of Diagnostic Accuracy Study (STARD) was developed to improve the completeness and transparency of reporting in studies investigating diagnostic test accuracy. However, its current form, STARD 2015 does not address the issues and challenges raised by artificial intelligence (AI)-centred interventions. As such, we propose an AI-specific version of the STARD checklist (STARD-AI), which focuses on the reporting of AI diagnostic test accuracy studies. This paper describes the methods that will be used to develop STARD-AI.Methods and analysisThe development of the STARD-AI checklist can be distilled into six stages. (1) A project organisation phase has been undertaken, during which a Project Team and a Steering Committee were established; (2) An item generation process has been completed following a literature review, a patient and public involvement and engagement exercise and an online scoping survey of international experts; (3) A three-round modified Delphi consensus methodology is underway, which will culminate in a teleconference consensus meeting of experts; (4) Thereafter, the Project Team will draft the initial STARD-AI checklist and the accompanying documents; (5) A piloting phase among expert users will be undertaken to identify items which are either unclear or missing. This process, consisting of surveys and semistructured interviews, will contribute towards the explanation and elaboration document and (6) On finalisation of the manuscripts, the group's efforts turn towards an organised dissemination and implementation strategy to maximise end-user adoption.Ethics and disseminationEthical approval has been granted by the Joint Research Compliance Office at Imperial College London (reference number: 19IC5679). A dissemination strategy will be aimed towards five groups of stakeholders: (1) academia, (2) policy, (3) guidelines and regulation, (4) industry and (5) public and non-specific stakeholders. We anticipate that dissemination will take place in Q3 of 2021.
| S-EPMC8240576 | biostudies-literature
Project description:Myocardial perfusion imaging (MPI) plays an important role in patients with suspected and documented coronary artery disease (CAD). Machine Learning (ML) algorithms have been developed for many medical applications with excellent performance. This study used ML algorithms to discern normal and abnormal gated Single Photon Emission Computed Tomography (SPECT) images. We analyzed one thousand and seven polar maps from a database of patients referred to a university hospital for clinically indicated MPI between January 2016 and December 2018. These studies were reported and evaluated by two different expert readers. The image features were extracted from a specific type of polar map segmentation based on horizontal and vertical slices. A senior expert reading was the comparator (gold standard). We used cross-validation to divide the dataset into training and testing subsets, using data augmentation in the training set, and evaluated 04 ML models. All models had accuracy >90% and area under the receiver operating characteristics curve (AUC) >0.80 except for Adaptive Boosting (AUC = 0.77), while all precision and sensitivity obtained were >96 and 92%, respectively. Random Forest had the best performance (AUC: 0.853; accuracy: 0,938; precision: 0.968; sensitivity: 0.963). ML algorithms performed very well in image classification. These models were capable of distinguishing polar maps remarkably into normal and abnormal.
| S-EPMC8660123 | biostudies-literature
Project description:Left ventricular (LV) segmentation, including accurate assignment of LV contours, is essential for the quantitative assessment of myocardial perfusion SPECT (MPS). Two major types of segmentation failures are observed in clinical practices: incorrect LV shape determination and incorrect valve-plane (VP) positioning. We have developed a technique to automatically detect these failures for both nongated and gated studies.A standard Cedars-Sinai perfusion SPECT (quantitative perfusion SPECT [QPS]) algorithm was applied to derive LV contours in 318 consecutive (99m)Tc-sestamibi rest/stress MPS studies consisting of stress/rest scans with or without attenuation correction and gated stress/rest images (1,903 scans total). Two numeric parameters, shape quality control (SQC) and valve-plane quality control, were derived to categorize the respective contour segmentation failures. The results were compared with the visual classification of automatic contour adequacy by 3 experienced observers.The overall success of automatic LV segmentation in the 1,903 scans ranged from 66% on nongated images (incorrect shape, 8%; incorrect VP, 26%) to 87% on gated images (incorrect shape, 3%; incorrect VP, 10%). The overall interobserver agreement for visual classification of automatic LV segmentation was 61% for nongated scans and 80% for gated images; the agreement between gray-scale and color-scale display for these scans was 86% and 91%, respectively. To improve the reliability of visual evaluation as a reference, the cases with intra- and interobserver discrepancies were excluded, and the remaining 1,277 datasets were considered (101 with incorrect LV shape and 102 with incorrect VP position). For the SQC, the receiver-operating-characteristic area under the curve (ROC-AUC) was 1.0 +/- 0.00 for the overall dataset, with an optimal sensitivity of 100% and a specificity of 98%. The ROC-AUC was 1.0 in all specific datasets. The algorithm was also able to detect the VP position errors: VP overshooting with ROC-AUC, 0.91 +/- 0.01; sensitivity, 100%; and specificity, 70%; and VP undershooting with ROC-AUC, 0.96 +/- 0.01; sensitivity, 100%; and specificity, 70%.A new automated method for quality control of LV MPS contours has been developed and shows high accuracy for the detection of failures in LV segmentation with a variety of acquisition protocols. This technique may lead to an improvement in the objective, automated quantitative analysis of MPS.
| S-EPMC2935909 | biostudies-other
Project description:BackgroundWe aim to establish a multicenter registry collecting clinical, imaging, and follow-up data for patients who undergo myocardial perfusion imaging (MPI) with the latest generation SPECT scanners.MethodsREFINE SPECT (REgistry of Fast Myocardial Perfusion Imaging with NExt generation SPECT) uses a collaborative design with multicenter contribution of clinical data and images into a comprehensive clinical-imaging database. All images are processed by quantitative software. Over 290 individual imaging variables are automatically extracted from each image dataset and merged with clinical variables. In the prognostic cohort, patient follow-up is performed for major adverse cardiac events. In the diagnostic cohort (patients with correlating invasive angiography), angiography and revascularization results within 6 months are obtained.ResultsTo date, collected prognostic data include scans from 20,418 patients in 5 centers (57% male, 64.0 ± 12.1 years) who underwent exercise (48%) or pharmacologic stress (52%). Diagnostic data include 2079 patients in 9 centers (67% male, 64.7 ± 11.2 years) who underwent exercise (39%) or pharmacologic stress (61%).ConclusionThe REFINE SPECT registry will provide a resource for collaborative projects related to the latest generation SPECT-MPI. It will aid in the development of new artificial intelligence tools for automated diagnosis and prediction of prognostic outcomes.
| S-EPMC6301135 | biostudies-literature
Project description:Cardiovascular conditions remain the leading cause of mortality and morbidity worldwide, with genotype being a significant influence on disease risk. Cardiac imaging-genetics aims to identify and characterize the genetic variants that influence functional, physiological, and anatomical phenotypes derived from cardiovascular imaging. High-throughput DNA sequencing and genotyping have greatly accelerated genetic discovery, making variant interpretation one of the key challenges in contemporary clinical genetics. Heterogeneous, low-fidelity phenotyping and difficulties integrating and then analyzing large-scale genetic, imaging and clinical datasets using traditional statistical approaches have impeded process. Artificial intelligence (AI) methods, such as deep learning, are particularly suited to tackle the challenges of scalability and high dimensionality of data and show promise in the field of cardiac imaging-genetics. Here we review the current state of AI as applied to imaging-genetics research and discuss outstanding methodological challenges, as the field moves from pilot studies to mainstream applications, from one dimensional global descriptors to high-resolution models of whole-organ shape and function, from univariate to multivariate analysis and from candidate gene to genome-wide approaches. Finally, we consider the future directions and prospects of AI imaging-genetics for ultimately helping understand the genetic and environmental underpinnings of cardiovascular health and disease.
| S-EPMC6985036 | biostudies-literature
Project description:To compare the content, clarity, and clinical usefulness of conventional (ie, free-form) and structured radiology reports of body computed tomographic (CT) scans, as evaluated by referring physicians, attending radiologists, and radiology fellows at a tertiary care cancer center.The institutional review board approved the study as a quality improvement initiative; no written consent was required. Three radiologists, three radiology fellows, three surgeons, and two medical oncologists evaluated 330 randomly selected conventional and structured radiology reports of body CT scans. For nonradiologists, reports were randomly selected from patients with diagnoses relevant to the physician's area of specialization. Each physician read 15 reports in each format and rated both the content and clarity of each report from 1 (very dissatisfied or very confusing) to 10 (very satisfied or very clear). By using a previously published radiology report grading scale, physicians graded each report's effectiveness in advancing the patient's position on the clinical spectrum. Mixed-effects models were used to test differences between report types.Mean content satisfaction ratings were 7.61 (95% confidence interval [CI]: 7.12, 8.16) for conventional reports and 8.33 (95% CI: 7.82, 8.86) for structured reports, and the difference was significant (P < .0001). Mean clarity satisfaction ratings were 7.45 (95% CI: 6.89, 8.02) for conventional reports and 8.25 (95% CI: 7.68, 8.82) for structured reports, and the difference was significant (P < .0001). Grade ratings did not differ significantly between conventional and structured reports.Referring clinicians and radiologists found that structured reports had better content and greater clarity than conventional reports.
| S-EPMC3121011 | biostudies-other
Project description:Antimalarial drugs are becoming less effective due to the emergence of drug resistance. Resistance has been reported for all available malaria drugs, including artemisinin, thus creating a perpetual need for alternative drug candidates. The traditional drug discovery approach of high throughput screening (HTS) of large compound libraries for identification of new drug leads is time-consuming and resource intensive. While virtual in silico screening is a solution to this problem, however, the generalization of the models is not ideal. Artificial intelligence (AI), utilizing either structure-based or ligand-based approaches, has demonstrated highly accurate performances in the field of chemical property prediction. Leveraging the existing data, AI would be a suitable alternative to blind-search HTS or fingerprint-based virtual screening. The AI model would learn patterns within the data and help to search for hit compounds efficiently. In this work, we introduce DeepMalaria, a deep-learning based process capable of predicting the anti-Plasmodium falciparum inhibitory properties of compounds using their SMILES. A graph-based model is trained on 13,446 publicly available antiplasmodial hit compounds from GlaxoSmithKline (GSK) dataset that are currently being used to find novel drug candidates for malaria. We validated this model by predicting hit compounds from a macrocyclic compound library and already approved drugs that are used for repurposing. We have chosen macrocyclic compounds as these ligand-binding structures are underexplored in malaria drug discovery. The in silico pipeline for this process also consists of additional validation of an in-house independent dataset consisting mostly of natural product compounds. Transfer learning from a large dataset was leveraged to improve the performance of the deep learning model. To validate the DeepMalaria generated hits, we used a commonly used SYBR Green I fluorescence assay based phenotypic screening. DeepMalaria was able to detect all the compounds with nanomolar activity and 87.5% of the compounds with greater than 50% inhibition. Further experiments to reveal the compounds' mechanism of action have shown that not only does one of the hit compounds, DC-9237, inhibits all asexual stages of Plasmodium falciparum, but is a fast-acting compound which makes it a strong candidate for further optimization.
| S-EPMC6974622 | biostudies-literature