Dataset Information

Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis.

ABSTRACT: BACKGROUND:Deep learning algorithms derived in homogeneous populations may be poorly generalizable and have the potential to reflect, perpetuate, and even exacerbate racial/ethnic disparities in health and health care. In this study, we aimed to (1) assess whether the performance of a deep learning algorithm designed to detect low left ventricular ejection fraction using the 12-lead ECG varies by race/ethnicity and to (2) determine whether its performance is determined by the derivation population or by racial variation in the ECG. METHODS:We performed a retrospective cohort analysis that included 97 829 patients with paired ECGs and echocardiograms. We tested the model performance by race/ethnicity for convolutional neural network designed to identify patients with a left ventricular ejection fraction ?35% from the 12-lead ECG. RESULTS:The convolutional neural network that was previously derived in a homogeneous population (derivation cohort, n=44 959; 96.2% non-Hispanic white) demonstrated consistent performance to detect low left ventricular ejection fraction across a range of racial/ethnic subgroups in a separate testing cohort (n=52 870): non-Hispanic white (n=44 524; area under the curve [AUC], 0.931), Asian (n=557; AUC, 0.961), black/African American (n=651; AUC, 0.937), Hispanic/Latino (n=331; AUC, 0.937), and American Indian/Native Alaskan (n=223; AUC, 0.938). In secondary analyses, a separate neural network was able to discern racial subgroup category (black/African American [AUC, 0.84], and white, non-Hispanic [AUC, 0.76] in a 5-class classifier), and a network trained only in non-Hispanic whites from the original derivation cohort performed similarly well across a range of racial/ethnic subgroups in the testing cohort with an AUC of at least 0.930 in all racial/ethnic subgroups. CONCLUSIONS:Our study demonstrates that while ECG characteristics vary by race, this did not impact the ability of a convolutional neural network to predict low left ventricular ejection fraction from the ECG. We recommend reporting of performance among diverse ethnic, racial, age, and sex groups for all new artificial intelligence tools to ensure responsible use of artificial intelligence in medicine.

SUBMITTER: Noseworthy PA

PROVIDER: S-EPMC7158877 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis.

Noseworthy Peter A PA Attia Zachi I ZI Brewer LaPrincess C LC Hayes Sharonne N SN Yao Xiaoxi X Kapa Suraj S Friedman Paul A PA Lopez-Jimenez Francisco F

Circulation. Arrhythmia and electrophysiology 20200216 3

<h4>Background</h4>Deep learning algorithms derived in homogeneous populations may be poorly generalizable and have the potential to reflect, perpetuate, and even exacerbate racial/ethnic disparities in health and health care. In this study, we aimed to (1) assess whether the performance of a deep learning algorithm designed to detect low left ventricular ejection fraction using the 12-lead ECG varies by race/ethnicity and to (2) determine whether its performance is determined by the derivation ...[more]

PMID: 32064914

Similar Datasets

Project description:Over 80% of bias-motivated violent victimization is motivated by race or ethnicity and over 50% of bias victimization occurs in non-Hispanic Whites (NHW). Our aim was to determine the risk and health impacts of race/ethnicity-motivated violent victimization by victim race/ethnicity. We examined data from the National Crime Victimization Survey (2003-2015) to estimate violent victimization risk by victim race/ethnicity across race/ethnicity bias victimization, other types of bias victimizations, and non-bias violent victimizations. We examined incident and offender characteristics for race/ethnicity-motivated victimization by victim race/ethnicity. The risk of race/ethnicity-motivated violent victimization was greater for non-Hispanic Blacks (NHB) and Hispanics than for NHWs (incidence rate ratios [IRR] = 1.4; 95% confidence interval [CI] = [1.0, 2.0], and IRR = 1.6; 95% CI = [1.2, 2.1]). This translates into an additional 46.7 incidents per 100,000 person-years (95% CI = [1.4, 92.1]) for the NHB population and an additional 60.3 incidents per 100,000 person-years (95% CI = [20.3, 100.4]) for the Hispanic population. Violent incidents for NHB victims more frequently resulted in injury or medical care. Nearly 40% of NHB victims reported difficulties at school or work related to the incident where only 21.5% of NHWs and 11.7% of Hispanic victims reported similar problems. Roughly 37% of NHB victims identified a NHW offender and 45% of NHW victims identified a NHB offender. Hispanic victims identified NHB or NHW offenders in over 70% of incidents. Although literature suggests that NHWs account for the majority of bias victimizations, the risk of non-fatal violent victimization motivated by race/ethnicity is greater for NHBs and Hispanics. Crimes perpetrated against NHBs are likely more severe and victim/offender racial incongruity is common. Findings provide empiric evidence on race/ethnicity-related structural disadvantage with adverse health consequences.

Project description:Coronary artery calcium (CAC), which can be measured in various types of computed tomography (CT) examinations, is a hallmark of coronary artery atherosclerosis. However, despite the clinical value of CAC scores in predicting cardiovascular events, routine measurement of CAC scores is limited due to high cost, radiation exposure, and lack of widespread availability. It would be of great clinical significance if CAC could be predicted by electrocardiograms (ECGs), which are cost-effective and routinely performed during various medical checkups. We aimed to develop binary classification artificial intelligence (AI) models that predict CAC using only ECGs as input. Moreover, we aimed to address the generalizability of our model in different environments by externally validating our model on a dataset from a different institution. Among adult patients, standard 12-lead ECGs were extracted if measured within 60 days before or after the CAC scores, and labeled with the corresponding CAC scores. We constructed deep convolutional neural network models based on residual networks using only the raw waveforms of the ECGs as input, predicting CAC at different levels, namely CAC score ≥100, ≥400 and ≥1,000. Our AI models performed well in predicting CAC in the training and internal validation dataset [area under the receiver operating characteristics curve (AUROC) 0.753 ± 0.009, 0.802 ± 0.027, and 0.835 ± 0.024 for the CAC score ≥100, ≥400, and ≥1,000 model, respectively]. Our models also performed well in the external validation dataset (AUROC 0.718, 0.777 and 0.803 for the CAC score ≥100, ≥400, and ≥1,000 model, respectively), indicating that our model can generalize well to different but plausibly related populations. Model performance in terms of AUROC increased in the order of CAC score ≥100, ≥400, and ≥1,000 model, indicating that higher CAC scores might be associated with more prominent structural changes of the heart detected by the model. With our AI models, a substantial proportion of previously unrecognized CAC can be afforded with a risk stratification of CAC, enabling initiation of prophylactic therapy, and reducing the adverse consequences related to ischemic heart disease.

Dataset Information

Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis.

Publications

Assessing and Mitigating Bias in Medical Artificial Intelligence: The Effects of Race and Ethnicity on a Deep Learning Model for ECG Analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets