Dataset Information

Diagnostic performance of an artificial intelligence-driven cardiac-structured reporting system for myocardial perfusion SPECT imaging.

ABSTRACT:

Objectives

To describe and validate an artificial intelligence (AI)-driven structured reporting system by direct comparison of automatically generated reports to results from actual clinical reports generated by nuclear cardiology experts.

Background

Quantitative parameters extracted from myocardial perfusion imaging (MPI) studies are used by our AI reporting system to generate automatically a guideline-compliant structured report (sR).

Method

A new nonparametric approach generates distribution functions of rest and stress, perfusion, and thickening, for each of 17 left ventricle segments that are then transformed to certainty factors (CFs) that a segment is hypoperfused, ischemic. These CFs are then input to our set of heuristic rules used to reach diagnostic findings and impressions propagated into a sR referred as an AI-driven structured report (AIsR). The diagnostic accuracy of the AIsR for detecting coronary artery disease (CAD) and ischemia was tested in 1,000 patients who had undergone rest/stress SPECT MPI.

Results

At the high-specificity (SP) level, in a subset of 100 patients, there were no statistical differences in the agreements between the AIsr, and nine experts' impressions of CAD (P = .33) or ischemia (P = .37). This high-SP level also yielded the highest accuracy across global and regional results in the 1,000 patients. These accuracies were statistically significantly better than the other two levels [sensitivity (SN)/SP tradeoff, high SN] across all comparisons.

Conclusions

This AI reporting system automatically generates a structured natural language report with a diagnostic performance comparable to those of experts.

SUBMITTER: Garcia EV

PROVIDER: S-EPMC6414293 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundMyocardial perfusion reflects the macro- and microvascular coronary circulation. Recent quantitation developments using cardiovascular magnetic resonance perfusion permit automated measurement clinically. We explored the prognostic significance of stress myocardial blood flow (MBF) and myocardial perfusion reserve (MPR, the ratio of stress to rest MBF).MethodsA 2-center study of patients with both suspected and known coronary artery disease referred clinically for perfusion assessment. Image analysis was performed automatically using a novel artificial intelligence approach deriving global and regional stress and rest MBF and MPR. Cox proportional hazard models adjusting for comorbidities and cardiovascular magnetic resonance parameters sought associations of stress MBF and MPR with death and major adverse cardiovascular events (MACE), including myocardial infarction, stroke, heart failure hospitalization, late (>90 day) revascularization, and death.ResultsA total of 1049 patients were included with a median follow-up of 605 (interquartile range, 464-814) days. There were 42 (4.0%) deaths and 188 MACE in 174 (16.6%) patients. Stress MBF and MPR were independently associated with both death and MACE. For each 1 mL·g-1·min-1 decrease in stress MBF, the adjusted hazard ratios for death and MACE were 1.93 (95% CI, 1.08-3.48, P=0.028) and 2.14 (95% CI, 1.58-2.90, P<0.0001), respectively, even after adjusting for age and comorbidity. For each 1 U decrease in MPR, the adjusted hazard ratios for death and MACE were 2.45 (95% CI, 1.42-4.24, P=0.001) and 1.74 (95% CI, 1.36-2.22, P<0.0001), respectively. In patients without regional perfusion defects on clinical read and no known macrovascular coronary artery disease (n=783), MPR remained independently associated with death and MACE, with stress MBF remaining associated with MACE only.ConclusionsIn patients with known or suspected coronary artery disease, reduced MBF and MPR measured automatically inline using artificial intelligence quantification of cardiovascular magnetic resonance perfusion mapping provides a strong, independent predictor of adverse cardiovascular outcomes.

Project description:Purpose:Next-generation sequencing technologies are actively applied in clinical oncology. Bioinformatics pipeline analysis is an integral part of this process; however, humans cannot yet realize the full potential of the highly complex pipeline output. As a result, the decision to include a variant in the final report during routine clinical sign-out remains challenging. Methods:We used an artificial intelligence approach to capture the collective clinical sign-out experience of six board-certified molecular pathologists to build and validate a decision support tool for variant reporting. We extracted all reviewed and reported variants from our clinical database and tested several machine learning models. We used 10-fold cross-validation for our variant call prediction model, which derives a contiguous prediction score from 0 to 1 (no to yes) for clinical reporting. Results:For each of the 19,594 initial training variants, our pipeline generates approximately 500 features, which results in a matrix of > 9 million data points. From a comparison of naive Bayes, decision trees, random forests, and logistic regression models, we selected models that allow human interpretability of the prediction score. The logistic regression model demonstrated 1% false negativity and 2% false positivity. The final models' Youden indices were 0.87 and 0.77 for screening and confirmatory cutoffs, respectively. Retraining on a new assay and performance assessment in 16,123 independent variants validated our approach (Youden index, 0.93). We also derived individual pathologist-centric models (virtual consensus conference function), and a visual drill-down functionality allows assessment of how underlying features contributed to a particular score or decision branch for clinical implementation. Conclusion:Our decision support tool for variant reporting is a practically relevant artificial intelligence approach to harness the next-generation sequencing bioinformatics pipeline output when the complexity of data interpretation exceeds human capabilities.

Project description:IntroductionStandards for Reporting of Diagnostic Accuracy Study (STARD) was developed to improve the completeness and transparency of reporting in studies investigating diagnostic test accuracy. However, its current form, STARD 2015 does not address the issues and challenges raised by artificial intelligence (AI)-centred interventions. As such, we propose an AI-specific version of the STARD checklist (STARD-AI), which focuses on the reporting of AI diagnostic test accuracy studies. This paper describes the methods that will be used to develop STARD-AI.Methods and analysisThe development of the STARD-AI checklist can be distilled into six stages. (1) A project organisation phase has been undertaken, during which a Project Team and a Steering Committee were established; (2) An item generation process has been completed following a literature review, a patient and public involvement and engagement exercise and an online scoping survey of international experts; (3) A three-round modified Delphi consensus methodology is underway, which will culminate in a teleconference consensus meeting of experts; (4) Thereafter, the Project Team will draft the initial STARD-AI checklist and the accompanying documents; (5) A piloting phase among expert users will be undertaken to identify items which are either unclear or missing. This process, consisting of surveys and semistructured interviews, will contribute towards the explanation and elaboration document and (6) On finalisation of the manuscripts, the group's efforts turn towards an organised dissemination and implementation strategy to maximise end-user adoption.Ethics and disseminationEthical approval has been granted by the Joint Research Compliance Office at Imperial College London (reference number: 19IC5679). A dissemination strategy will be aimed towards five groups of stakeholders: (1) academia, (2) policy, (3) guidelines and regulation, (4) industry and (5) public and non-specific stakeholders. We anticipate that dissemination will take place in Q3 of 2021.