Dataset Information

Racial disparities in automated speech recognition.

ABSTRACT: Automated speech recognition (ASR) systems, which use sophisticated machine-learning algorithms to convert spoken language to text, have become increasingly widespread, powering popular virtual assistants, facilitating automated closed captioning, and enabling digital dictation platforms for health care. Over the last several years, the quality of these systems has dramatically improved, due both to advances in deep learning and to the collection of large-scale datasets used to train the systems. There is concern, however, that these tools do not work equally well for all subgroups of the population. Here, we examine the ability of five state-of-the-art ASR systems-developed by Amazon, Apple, Google, IBM, and Microsoft-to transcribe structured interviews conducted with 42 white speakers and 73 black speakers. In total, this corpus spans five US cities and consists of 19.8 h of audio matched on the age and gender of the speaker. We found that all five ASR systems exhibited substantial racial disparities, with an average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers. We trace these disparities to the underlying acoustic models used by the ASR systems as the race gap was equally large on a subset of identical phrases spoken by black and white individuals in our corpus. We conclude by proposing strategies-such as using more diverse training datasets that include African American Vernacular English-to reduce these performance differences and ensure speech recognition technology is inclusive.

SUBMITTER: Koenecke A

PROVIDER: S-EPMC7149386 | biostudies-literature | 2020 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Racial disparities in automated speech recognition.

Koenecke Allison A Nam Andrew A Lake Emily E Nudell Joe J Quartey Minnie M Mengesha Zion Z Toups Connor C Rickford John R JR Jurafsky Dan D Goel Sharad S

Proceedings of the National Academy of Sciences of the United States of America 20200323 14

Automated speech recognition (ASR) systems, which use sophisticated machine-learning algorithms to convert spoken language to text, have become increasingly widespread, powering popular virtual assistants, facilitating automated closed captioning, and enabling digital dictation platforms for health care. Over the last several years, the quality of these systems has dramatically improved, due both to advances in deep learning and to the collection of large-scale datasets used to train the systems ...[more]

PMID: 32205437

Similar Datasets

Project description:ObjectiveTo evaluate racial and ethnic differences in women's postpartum pain scores, inpatient opioid administration, and discharge opioid prescriptions.MethodsWe conducted a retrospective cohort study of all deliveries at a single high-volume tertiary care center from December 1, 2015, through November 30, 2016. Women were included if they self-identified as non-Hispanic white, non-Hispanic black, or Hispanic; were at least 18 years of age; and did not have documented allergies to nonsteroidal antiinflammatory drugs or morphine. Medical records were queried for three outcomes: 1) patient-reported postpartum pain score (on a scale of 0-10) at discharge (dichotomized less than 5 or 5 or higher), 2) inpatient opioid dosing during postpartum hospitalization (reported as morphine milligram equivalents [MMEs] per postpartum day), and 3) receipt of an opioid prescription at discharge. The associations between each of these outcomes and maternal race-ethnicity were assessed using multivariable logistic regression models with random effects to account for clustering by discharge physician. A sensitivity analysis was conducted in which women of different race and ethnicity were matched using propensity scores.ResultsA total of 9,900 postpartum women were eligible for analysis. Compared with non-Hispanic white women, Hispanic and non-Hispanic black women had significantly greater odds of reporting a pain score of 5 or higher (adjusted odds ratio [aOR] 1.61, 95% 1.26-2.06 and aOR 2.18, 95% 1.63-2.91, respectively) but received significantly fewer inpatient MMEs/d (adjusted β -5.03, 95% CI -6.91 to -3.15, and adjusted β -3.54, 95% CI -5.88 to -1.20, respectively). Additionally, Hispanic and non-Hispanic black women were significantly less likely to receive an opioid prescription at discharge (aOR 0.80, 95% CI 0.67 to -0.96 and aOR 0.78, 95% CI 0.62-0.98) compared with non-Hispanic white women. Results of the propensity score analysis largely corroborated those of the primary analysis, with the exception that the difference in inpatient MMEs/d between non-Hispanic white and non-Hispanic black women did not reach statistical significance.ConclusionHispanic and non-Hispanic black women experience disparities in pain management in the postpartum setting that cannot be explained by less perceived pain.

Dataset Information

Racial disparities in automated speech recognition.

Publications

Racial disparities in automated speech recognition.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets