Dataset Information

Generalisation Gap of Keyword Spotters in a Cross-Speaker Low-Resource Scenario.

ABSTRACT: Models for keyword spotting in continuous recordings can significantly improve the experience of navigating vast libraries of audio recordings. In this paper, we describe the development of such a keyword spotting system detecting regions of interest in Polish call centre conversations. Unfortunately, in spite of recent advancements in automatic speech recognition systems, human-level transcription accuracy reported on English benchmarks does not reflect the performance achievable in low-resource languages, such as Polish. Therefore, in this work, we shift our focus from complete speech-to-text conversion to acoustic similarity matching in the hope of reducing the demand for data annotation. As our primary approach, we evaluate Siamese and prototypical neural networks trained on several datasets of English and Polish recordings. While we obtain usable results in English, our models' performance remains unsatisfactory when applied to Polish speech, both after mono- and cross-lingual training. This performance gap shows that generalisation with limited training resources is a significant obstacle for actual deployments in low-resource languages. As a potential countermeasure, we implement a detector using audio embeddings generated with a generic pre-trained model provided by Google. It has a much more favourable profile when applied in a cross-lingual setup to detect Polish audio patterns. Nevertheless, despite these promising results, its performance on out-of-distribution data are still far from stellar. It would indicate that, in spite of the richness of internal representations created by more generic models, such speech embeddings are not entirely malleable to cross-language transfer.

SUBMITTER: Lepak L

PROVIDER: S-EPMC8704929 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundUniversal health coverage promises equity in access to and quality of health services. However, there is variability in the quality of the care (QoC) delivered at health facilities in low and middle-income countries (LMICs). Detecting gaps in implementation of clinical guidelines is key to prioritizing the efforts to improve quality of care. The aim of this study was to present statistical methods that maximize the use of existing electronic medical records (EMR) to monitor compliance with evidence-based care guidelines in LMICs.MethodsWe used iSanté, Haiti's largest EMR to assess adherence to treatment guidelines and retention on treatment of HIV patients across Haitian HIV care facilities. We selected three processes of care - (1) implementation of a 'test and start' approach to antiretroviral therapy (ART), (2) implementation of HIV viral load testing, and (3) uptake of multi-month scripting for ART, and three continuity of care indicators - (4) timely ART pick-up, (5) 6-month ART retention of pregnant women and (6) 6-month ART retention of non-pregnant adults. We estimated these six indicators using a model-based approach to account for their volatility and measurement error. We added a case-mix adjustment for continuity of care indicators to account for the effect of factors other than medical care (biological, socio-economic). We combined the six indicators in a composite measure of appropriate care based on adherence to treatment guidelines.ResultsWe analyzed data from 65,472 patients seen in 89 health facilities between June 2016 and March 2018. Adoption of treatment guidelines differed greatly between facilities; several facilities displayed 100% compliance failure, suggesting implementation issues. Risk-adjusted continuity of care indicators showed less variability, although several facilities had patient retention rates that deviated significantly from the national average. Based on the composite measure, we identified two facilities with consistently poor performance and two star performers.ConclusionsOur work demonstrates the potential of EMRs to detect gaps in appropriate care processes, and thereby to guide quality improvement efforts. Closing quality gaps will be pivotal in achieving equitable access to quality care in LMICs.

Project description:BackgroundNumerous trauma scoring systems have been developed in an attempt to accurately and efficiently predict the prognosis of emergent trauma cases. However, it has been questioned as to whether the accuracy and pragmatism of such systems still hold in lower-resource settings that exist in many hospitals in lower- and middle-income countries (LMICs). In this study, it was hypothesized that the physiologically-based Revised Trauma Score (RTS), Mechanism/Glasgow Coma Scale/Age/Pressure (MGAP) score, and Glasgow Coma Scale/Age/Pressure (GAP) score would be effective at predicting mortality outcomes using clinical data at presentation in a representative LMIC hospital in Upper Egypt.MethodsThis was a retrospective analysis of the medical records of trauma patients at Beni-Suef University Hospital. Medical records of all trauma patients admitted to the hospital over the 8-month period from January to August 2016 were reviewed. For each case, the RTS, MGAP, and GAP scores were calculated using clinical data at presentation, and mortality prediction was correlated to the actual in-hospital outcome.ResultsThe Area Under the Receiver Operating Characteristic (AUROC) was calculated to be 0.879, 0.890, and 0.881 for the MGAP, GAP, and RTS respectively, with all three scores showing good discriminatory ability. With regards to prevalence-dependent statistics, all three scores demonstrated efficacy in ruling out mortality upon presentation with negative predictive values > 95%, while the MGAP score best captured the mortality subgroup with a sensitivity of 94%. Adjustment of cutoff scores showed a steep trade-off between optimizing the positive predictive values versus the sensitivities.ConclusionThe RTS, MGAP, and GAP all showed good discriminatory capabilities per AUROC. Given the relative simplicity and potentially added clinical benefit in capturing critically ill patients, the MGAP score should be further studied for stratifying risk of incoming trauma patients to the emergency department, allowing for more efficacious triage of patients in lower-resource healthcare settings.

Dataset Information

Generalisation Gap of Keyword Spotters in a Cross-Speaker Low-Resource Scenario.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets