External validation of a deep learning electrocardiogram algorithm to detect ventricular dysfunction.
Ontology highlight
ABSTRACT: Objective
To validate a novel artificial-intelligence electrocardiogram algorithm (AI-ECG) to detect left ventricular systolic dysfunction (LVSD) in an external population.Background
LVSD, even when asymptomatic, confers increased morbidity and mortality. We recently derived AI-ECG to detect LVSD using ECGs based on a large sample of patients treated at the Mayo Clinic.Methods
We performed an external validation study with subjects from the Know Your Heart Study, a cross-sectional study of adults aged 35-69 years residing in two cities in Russia, who had undergone both ECG and transthoracic echocardiography. LVSD was defined as left ventricular ejection fraction ≤ 35%. We assessed the performance of the AI-ECG to identify LVSD in this distinct patient population.Results
Among 4277 subjects in this external population-based validation study, 0.6% had LVSD (compared to 7.8% of the original clinical derivation study). The overall performance of the AI-ECG to detect LVSD was robust with an area under the receiver operating curve of 0.82. When using the LVSD probability cut-off of 0.256 from the original derivation study, the sensitivity, specificity, and accuracy in this population were 26.9%, 97.4%, 97.0%, respectively. Other probability cut-offs were analysed for different sensitivity values.Conclusions
The AI-ECG detected LVSD with robust test performance in a population that was very different from that used to develop the algorithm. Population-specific cut-offs may be necessary for clinical implementation. Differences in population characteristics, ECG and echocardiographic data quality may affect test performance.
SUBMITTER: Attia IZ
PROVIDER: S-EPMC7955278 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectivesThis study sought to develop DL models capable of comprehensively quantifying left and right ventricular dysfunction from ECG data in a large, diverse population.BackgroundRapid evaluation of left and right ventricular function using deep learning (DL) on electrocardiograms (ECGs) can assist diagnostic workflow. However, DL tools to estimate right ventricular (RV) function do not exist, whereas those to estimate left ventricular (LV) function are restricted to quantification of very low LV function only.MethodsA multicenter study was conducted with data from 5 New York City hospitals: 4 for internal testing and 1 serving as external validation. We created novel DL models to classify left ventricular ejection fraction (LVEF) into categories derived from the latest universal definition of heart failure, estimate LVEF through regression, and predict a composite outcome of either RV systolic dysfunction or RV dilation.ResultsWe obtained echocardiogram LVEF estimates for 147,636 patients paired to 715,890 ECGs. We used natural language processing (NLP) to extract RV size and systolic function information from 404,502 echocardiogram reports paired to 761,510 ECGs for 148,227 patients. For LVEF classification in internal testing, area under curve (AUC) at detection of LVEF ≤40%, 40% < LVEF ≤50%, and LVEF >50% was 0.94 (95% CI: 0.94-0.94), 0.82 (95% CI: 0.81-0.83), and 0.89 (95% CI: 0.89-0.89), respectively. For external validation, these results were 0.94 (95% CI: 0.94-0.95), 0.73 (95% CI: 0.72-0.74), and 0.87 (95% CI: 0.87-0.88). For regression, the mean absolute error was 5.84% (95% CI: 5.82%-5.85%) for internal testing and 6.14% (95% CI: 6.13%-6.16%) in external validation. For prediction of the composite RV outcome, AUC was 0.84 (95% CI: 0.84-0.84) in both internal testing and external validation.ConclusionsDL on ECG data can be used to create inexpensive screening, diagnostic, and predictive tools for both LV and RV dysfunction. Such tools may bridge the applicability of ECGs and echocardiography and enable prioritization of patients for further interventions for either sided failure progressing to biventricular disease.
| S-EPMC8917975 | biostudies-literature
Project description:Cardiovascular diseases (CVDs) have become the number 1 threat to human health. Their numerous complications mean that many countries remain unable to prevent the rapid growth of such diseases, although significant health resources have been invested toward their prevention and management. Electrocardiogram (ECG) is the most important non-invasive physiological signal for CVD screening and diagnosis. For exploring the heartbeat event classification model using single- or multiple-lead ECG signals, we proposed a novel deep learning algorithm and conducted a systemic comparison based on the different methods and databases. This new approach aims to improve accuracy and reduce training time by combining the convolutional neural network (CNN) with the bidirectional long short-term memory (BiLSTM). To our knowledge, this approach has not been investigated to date. In this study, Database I with single-lead ECG and Database II with 12-lead ECG were used to explore a practical and viable heartbeat event classification model. An evolutionary neural system approach (Method I) and a deep learning approach (Method II) that combines CNN with BiLSTM network were compared and evaluated in processing heartbeat event classification. Overall, Method I achieved slightly better performance than Method II. However, Method I took, on average, 28.3 h to train the model, whereas Method II needed only 1 h. Method II achieved an accuracy of 80, 82.6, and 85% compared with the China Physiological Signal Challenge 2018, PhysioNet Challenge 2017, and Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) Arrhythmia datasets, respectively. These results are impressive compared with the performance of state-of-the-art algorithms used for the same purpose.
| S-EPMC7566908 | biostudies-literature
Project description:(1) Background: Acute pericarditis is often confused with ST-segment elevation myocardial infarction (STEMI) among patients presenting with acute chest pain in the emergency department (ED). Since a deep learning model (DLM) has been validated to accurately identify STEMI cases via 12-lead electrocardiogram (ECG), this study aimed to develop another DLM for the detection of acute pericarditis in the ED. (2) Methods: This study included 128 ECGs from patients with acute pericarditis and 66,633 ECGs from patients visiting the ED between 1 January 2010 and 31 December 2020. The ECGs were randomly allocated based on patients to the training, tuning, and validation sets, at a 3:1:1 ratio. We used raw ECG signals to train a pericarditis-DLM and used traditional ECG features to train a machine learning model. A human-machine competition was conducted using a subset of the validation set, and the performance of the Philips automatic algorithm was also compared. STEMI cases in the validation set were extracted to analyze the DLM ability of differential diagnosis between acute pericarditis and STEMI using ECG. We also followed the hospitalization events in non-pericarditis cases to explore the meaning of false-positive predictions. (3) Results: The pericarditis-DLM exceeded the performance of all participating human experts and algorithms based on traditional ECG features in the human-machine competition. In the validation set, the pericarditis-DLM could detect acute pericarditis with an area under the receiver operating characteristic curve (AUC) of 0.954, a sensitivity of 78.9%, and a specificity of 97.7%. However, our pericarditis-DLM also misinterpreted 10.2% of STEMI ECGs as pericarditis cases. Therefore, we generated an integrating strategy combining pericarditis-DLM and a previously developed STEMI-DLM, which provided a sensitivity of 73.7% and specificity of 99.4%, to identify acute pericarditis in patients with chest pains. Compared to the true-negative cases, patients with false-positive results using this strategy were associated with higher risk of hospitalization within 3 days due to cardiac disorders (hazard ratio (HR): 8.09; 95% confidence interval (CI): 3.99 to 16.39). (4) Conclusions: The AI-enhanced algorithm may be a powerful tool to assist clinicians in the early detection of acute pericarditis and differentiate it from STEMI using 12-lead ECGs.
| S-EPMC9324403 | biostudies-literature
Project description:Background and objectivesWe aim to explore the additional discriminative accuracy of a deep learning (DL) algorithm using repeated-measures data for identifying people at high risk for cardiovascular disease (CVD), compared to Cox hazard regression.MethodsTwo CVD prediction models were developed from National Health Insurance Service-Health Screening Cohort (NHIS-HEALS): a Cox regression model and a DL model. Performance of each model was assessed in the internal and 2 external validation cohorts in Koreans (National Health Insurance Service-National Sample Cohort; NHIS-NSC) and in Europeans (Rotterdam Study). A total of 412,030 adults in the NHIS-HEALS; 178,875 adults in the NHIS-NSC; and the 4,296 adults in Rotterdam Study were included.ResultsMean ages was 52 years (46% women) and there were 25,777 events (6.3%) in NHIS-HEALS during the follow-up. In internal validation, the DL approach demonstrated a C-statistic of 0.896 (95% confidence interval, 0.886-0.907) in men and 0.921 (0.908-0.934) in women and improved reclassification compared with Cox regression (net reclassification index [NRI], 24.8% in men, 29.0% in women). In external validation with NHIS-NSC, DL demonstrated a C-statistic of 0.868 (0.860-0.876) in men and 0.889 (0.876-0.898) in women, and improved reclassification compared with Cox regression (NRI, 24.9% in men, 26.2% in women). In external validation applied to the Rotterdam Study, DL demonstrated a C-statistic of 0.860 (0.824-0.897) in men and 0.867 (0.830-0.903) in women, and improved reclassification compared with Cox regression (NRI, 36.9% in men, 31.8% in women).ConclusionsA DL algorithm exhibited greater discriminative accuracy than Cox model approaches.Trial registrationClinicalTrials.gov Identifier: NCT02931500.
| S-EPMC6923233 | biostudies-literature
Project description:BackgroundCurrent electrocardiogram (ECG) criteria of left ventricular hypertrophy (LVH) have low sensitivity. Deep learning (DL) techniques have been widely used to detect cardiac diseases due to its ability of automatic feature extraction of ECG. However, DL was rarely applied in LVH diagnosis. Our study aimed to construct a DL model for rapid and effective detection of LVH using 12-lead ECG.MethodsWe built a DL model based on convolutional neural network-long short-term memory (CNN-LSTM) to detect LVH using 12-lead ECG. The echocardiogram and ECG of 1,863 patients obtained within 1 week after hospital admission were analyzed. Patients were evenly allocated into 3 sets at 3:1:1 ratio: the training set (n = 1,120), the validation set (n = 371) and the test set 1 (n = 372). In addition, we recruited 453 hospitalized patients into the internal test set 2. Different DL model of each subgroup was developed according to gender and relative wall thickness (RWT).ResultsThe LVH was predicted by the CNN-LSTM model with an area under the curve (AUC) of 0.62 (sensitivity 68%, specificity 57%) in the test set 1, which outperformed Cornell voltage criteria (AUC: 0.57, sensitivity 48%, specificity 72%) and Sokolow-Lyon voltage (AUC: 0.51, sensitivity 14%, specificity 96%). In the internal test set 2, the CNN-LSTM model had a stable performance in predicting LVH with an AUC of 0.59 (sensitivity 65%, specificity 57%). In the subgroup analysis, the CNN-LSTM model predicted LVH by 12-lead ECG with an AUC of 0.66 (sensitivity 72%, specificity 60%) for male patients, which performed better than that for female patients (AUC: 0.59, sensitivity 50%, specificity 71%).ConclusionOur study established a CNN-LSTM model to diagnose LVH by 12-lead ECG with higher sensitivity than current ECG diagnostic criteria. This CNN-LSTM model may be a simple and effective screening tool of LVH.
| S-EPMC9406285 | biostudies-literature
Project description:AimsDeep learning models (DLMs) have shown superiority in electrocardiogram (ECG) analysis and have been applied to diagnose dyskalaemias. However, no study has explored the performance of DLM-enabled ECG in continuous follow-up scenarios. Therefore, we proposed a dynamic revision of DLM-enabled ECG to use personal pre-annotated ECGs to enhance the accuracy in patients with multiple visits.Methods and resultsWe retrospectively collected 168 450 ECGs with corresponding serum potassium (K+) levels from 103 091 patients as development samples. In the internal/external validation sets, the numbers of ECGs with corresponding K+ were 37 246/47 604 from 13 555/20 058 patients. Our dynamic revision method showed better performance than the traditional direct prediction for diagnosing hypokalaemia [area under the receiver operating characteristic curve (AUC) = 0.730/0.720-0.788/0.778] and hyperkalaemia (AUC = 0.884/0.888-0.915/0.908) in patients with multiple visits.ConclusionOur method has shown a distinguishable improvement in DLMs for diagnosing dyskalaemias in patients with multiple visits, and we also proved its application in ejection fraction prediction, which could further improve daily clinical practice.
| S-EPMC9890087 | biostudies-literature
Project description:BackgroundAcute respiratory distress syndrome (ARDS) is a common, but under-recognised, critical illness syndrome associated with high mortality. An important factor in its under-recognition is the variability in chest radiograph interpretation for ARDS. We sought to train a deep convolutional neural network (CNN) to detect ARDS findings on chest radiographs.MethodsCNNs were pretrained on 595 506 radiographs from two centres to identify common chest findings (eg, opacity and effusion), and then trained on 8072 radiographs annotated for ARDS by multiple physicians using various transfer learning approaches. The best performing CNN was tested on chest radiographs in an internal and external cohort, including a subset reviewed by six physicians, including a chest radiologist and physicians trained in intensive care medicine. Chest radiograph data were acquired from four US hospitals.FindingsIn an internal test set of 1560 chest radiographs from 455 patients with acute hypoxaemic respiratory failure, a CNN could detect ARDS with an area under the receiver operator characteristics curve (AUROC) of 0·92 (95% CI 0·89-0·94). In the subgroup of 413 images reviewed by at least six physicians, its AUROC was 0·93 (95% CI 0·88-0·96), sensitivity 83·0% (95% CI 74·0-91·1), and specificity 88·3% (95% CI 83·1-92·8). Among images with zero of six ARDS annotations (n=155), the median CNN probability was 11%, with six (4%) assigned a probability above 50%. Among images with six of six ARDS annotations (n=27), the median CNN probability was 91%, with two (7%) assigned a probability below 50%. In an external cohort of 958 chest radiographs from 431 patients with sepsis, the AUROC was 0·88 (95% CI 0·85-0·91). When radiographs annotated as equivocal were excluded, the AUROC was 0·93 (0·92-0·95).InterpretationA CNN can be trained to achieve expert physician-level performance in ARDS detection on chest radiographs. Further research is needed to evaluate the use of these algorithms to support real-time identification of ARDS patients to ensure fidelity with evidence-based care or to support ongoing ARDS research.FundingNational Institutes of Health, Department of Defense, and Department of Veterans Affairs.
| S-EPMC8182690 | biostudies-literature
Project description:BackgroundRecent guidelines propose N-terminal pro-B-type natriuretic peptide (NT-proBNP) for recognition of asymptomatic left ventricular (LV) dysfunction (Stage B Heart Failure, SBHF) in type 2 diabetes mellitus (T2DM). Wavelet Transform based signal-processing transforms electrocardiogram (ECG) waveforms into an energy distribution waveform (ew)ECG, providing frequency and energy features that machine learning can use as additional inputs to improve the identification of SBHF. Accordingly, we sought whether machine learning model based on ewECG features was superior to NT-proBNP, as well as a conventional screening tool-the Atherosclerosis Risk in Communities (ARIC) HF risk score, in SBHF screening among patients with T2DM.MethodsParticipants in two clinical trials of SBHF (defined as diastolic dysfunction [DD], reduced global longitudinal strain [GLS ≤ 18%] or LV hypertrophy [LVH]) in T2DM underwent 12-lead ECG with additional ewECG feature and echocardiography. Supervised machine learning was adopted to identify the optimal combination of ewECG extracted features for SBHF screening in 178 participants in one trial and tested in 97 participants in the other trial. The accuracy of the ewECG model in SBHF screening was compared with NT-proBNP and ARIC HF.ResultsSBHF was identified in 128 (72%) participants in the training dataset (median 72 years, 41% female) and 64 (66%) in the validation dataset (median 70 years, 43% female). Fifteen ewECG features showed an area under the curve (AUC) of 0.81 (95% CI 0.787-0.794) in identifying SBHF, significantly better than both NT-proBNP (AUC 0.56, 95% CI 0.44-0.68, p < 0.001) and ARIC HF (AUC 0.67, 95%CI 0.56-0.79, p = 0.002). ewECG features were also led to robust models screening for DD (AUC 0.74, 95% CI 0.73-0.74), reduced GLS (AUC 0.76, 95% CI 0.73-0.74) and LVH (AUC 0.90, 95% CI 0.88-0.89).ConclusionsMachine learning based modelling using additional ewECG extracted features are superior to NT-proBNP and ARIC HF in SBHF screening among patients with T2DM, providing an alternative HF screening strategy for asymptomatic patients and potentially act as a guidance tool to determine those who required echocardiogram to confirm diagnosis. Trial registration LEAVE-DM, ACTRN 12619001393145 and Vic-ELF, ACTRN 12617000116325.
| S-EPMC10918872 | biostudies-literature
Project description:Atrial fibrillation (AF) is the most prevalent arrhythmia and is associated with increased morbidity and mortality. Its early detection is challenging because of the low detection yield of conventional methods. We aimed to develop a deep learning-based algorithm to identify AF during normal sinus rhythm (NSR) using 12-lead electrocardiogram (ECG) findings. We developed a new deep neural network to detect subtle differences in paroxysmal AF (PAF) during NSR using digital data from standard 12-lead ECGs. Raw digital data of 2,412 12-lead ECGs were analyzed. The artificial intelligence (AI) model showed that the optimal interval to detect subtle changes in PAF was within 0.24 s before the QRS complex in the 12-lead ECG. We allocated the enrolled ECGs to the training, internal validation, and testing datasets in a 7:1:2 ratio. Regarding AF identification, the AI-based algorithm showed the following values in the internal and external validation datasets: area under the receiver operating characteristic curve, 0.79 and 0.75; recall, 82% and 77%; specificity, 78% and 72%; F1 score, 75% and 74%; and overall accuracy, 72.8% and 71.2%, respectively. The deep learning-based algorithm using 12-lead ECG demonstrated high accuracy for detecting AF during NSR.
| S-EPMC8211689 | biostudies-literature
Project description:Coronary artery disease (CAD), the most common manifestation of cardiovascular disease, remains the most common cause of mortality in the United States. Risk assessment is key for primary prevention of coronary events and coronary artery calcium (CAC) scoring using computed tomography (CT) is one such non-invasive tool. Despite the proven clinical value of CAC, the current clinical practice implementation for CAC has limitations such as the lack of insurance coverage for the test, need for capital-intensive CT machines, specialized imaging protocols, and accredited 3D imaging labs for analysis (including personnel and software). Perhaps the greatest gap is the millions of patients who undergo routine chest CT exams and demonstrate coronary artery calcification, but their presence is not often reported or quantitation is not feasible. We present two deep learning models that automate CAC scoring demonstrating advantages in automated scoring for both dedicated gated coronary CT exams and routine non-gated chest CTs performed for other reasons to allow opportunistic screening. First, we trained a gated coronary CT model for CAC scoring that showed near perfect agreement (mean difference in scores = -2.86; Cohen's Kappa = 0.89, P < 0.0001) with current conventional manual scoring on a retrospective dataset of 79 patients and was found to perform the task faster (average time for automated CAC scoring using a graphics processing unit (GPU) was 3.5 ± 2.1 s vs. 261 s for manual scoring) in a prospective trial of 55 patients with little difference in scores compared to three technologists (mean difference in scores = 3.24, 5.12, and 5.48, respectively). Then using CAC scores from paired gated coronary CT as a reference standard, we trained a deep learning model on our internal data and a cohort from the Multi-Ethnic Study of Atherosclerosis (MESA) study (total training n = 341, Stanford test n = 42, MESA test n = 46) to perform CAC scoring on routine non-gated chest CT exams with validation on external datasets (total n = 303) obtained from four geographically disparate health systems. On identifying patients with any CAC (i.e., CAC ≥ 1), sensitivity and PPV was high across all datasets (ranges: 80-100% and 87-100%, respectively). For CAC ≥ 100 on routine non-gated chest CTs, which is the latest recommended threshold to initiate statin therapy, our model showed sensitivities of 71-94% and positive predictive values in the range of 88-100% across all the sites. Adoption of this model could allow more patients to be screened with CAC scoring, potentially allowing opportunistic early preventive interventions.
| S-EPMC8169744 | biostudies-literature