Dataset Information

Machine Learning for Health: Algorithm Auditing & Quality Control.

ABSTRACT: Developers proposing new machine learning for health (ML4H) tools often pledge to match or even surpass the performance of existing tools, yet the reality is usually more complicated. Reliable deployment of ML4H to the real world is challenging as examples from diabetic retinopathy or Covid-19 screening show. We envision an integrated framework of algorithm auditing and quality control that provides a path towards the effective and reliable application of ML systems in healthcare. In this editorial, we give a summary of ongoing work towards that vision and announce a call for participation to the special issue Machine Learning for Health: Algorithm Auditing & Quality Control in this journal to advance the practice of ML4H auditing.

SUBMITTER: Oala L

PROVIDER: S-EPMC8562935 | biostudies-literature | 2021 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Machine Learning for Health: Algorithm Auditing & Quality Control.

Oala Luis L Murchison Andrew G AG Balachandran Pradeep P Choudhary Shruti S Fehr Jana J Leite Alixandro Werneck AW Goldschmidt Peter G PG Johner Christian C Schörverth Elora D M EDM Nakasi Rose R Meyer Martin M Cabitza Federico F Baird Pat P Prabhu Carolin C Weicken Eva E Liu Xiaoxuan X Wenzel Markus M Vogler Steffen S Akogo Darlington D Alsalamah Shada S Kazim Emre E Koshiyama Adriano A Piechottka Sven S Macpherson Sheena S Shadforth Ian I Geierhofer Regina R Matek Christian C Krois Joachim J Sanguinetti Bruno B Arentz Matthew M Bielik Pavol P Calderon-Ramirez Saul S Abbood Auss A Langer Nicolas N Haufe Stefan S Kherif Ferath F Pujari Sameer S Samek Wojciech W Wiegand Thomas T

Journal of medical systems 20211102 12

Developers proposing new machine learning for health (ML4H) tools often pledge to match or even surpass the performance of existing tools, yet the reality is usually more complicated. Reliable deployment of ML4H to the real world is challenging as examples from diabetic retinopathy or Covid-19 screening show. We envision an integrated framework of algorithm auditing and quality control that provides a path towards the effective and reliable application of ML systems in healthcare. In this editor ...[more]

PMID: 34729675

Similar Datasets

Project description:BackgroundLinguistic accessibility has an important impact on the reception and utilization of translated health resources among multicultural and multilingual populations. Linguistic understandability of health translation has been understudied.ObjectiveOur study aimed to develop novel machine learning models for the study of the linguistic accessibility of health translations comparing Chinese translations of the World Health Organization health materials with original Chinese health resources developed by the Chinese health authorities.MethodsUsing natural language processing tools for the assessment of the readability of Chinese materials, we explored and compared the readability of Chinese health translations from the World Health Organization with original Chinese materials from the China Center for Disease Control and Prevention.ResultsA pairwise adjusted t test showed that the following 3 new machine learning models achieved statistically significant improvement over the baseline logistic regression in terms of area under the curve: C5.0 decision tree (95% CI -0.249 to -0.152; P<0.001), random forest (95% CI 0.139-0.239; P<0.001) and extreme gradient boosting tree (95% CI 0.099-0.193; P<0.001). There was, however, no significant difference between C5.0 decision tree and random forest (P=0.513). The extreme gradient boosting tree was the best model, achieving statistically significant improvement over the C5.0 model (P=0.003) and the random forest model (P=0.006) at an adjusted Bonferroni P value at 0.008.ConclusionsThe development of machine learning algorithms significantly improved the accuracy and reliability of current approaches to the evaluation of the linguistic accessibility of Chinese health information, especially Chinese health translations in relation to original health resources. Although the new algorithms developed were based on Chinese health resources, they can be adapted for other languages to advance current research in accessible health translation, communication, and promotion.

Project description:BackgroundA major problem in treating acute kidney injury (AKI) is that clinical criteria for recognition are markers of established kidney damage or impaired function; treatment before such damage manifests is desirable. Clinicians could intervene during what may be a crucial stage for preventing permanent kidney injury if patients with incipient AKI and those at high risk of developing AKI could be identified.ObjectiveIn this study, we evaluate a machine learning algorithm for early detection and prediction of AKI.DesignWe used a machine learning technique, boosted ensembles of decision trees, to train an AKI prediction tool on retrospective data taken from more than 300 000 inpatient encounters.SettingData were collected from inpatient wards at Stanford Medical Center and intensive care unit patients at Beth Israel Deaconess Medical Center.PatientsPatients older than the age of 18 whose hospital stays lasted between 5 and 1000 hours and who had at least one documented measurement of heart rate, respiratory rate, temperature, serum creatinine (SCr), and Glasgow Coma Scale (GCS).MeasurementsWe tested the algorithm's ability to detect AKI at onset and to predict AKI 12, 24, 48, and 72 hours before onset.MethodsWe tested AKI detection and prediction using the National Health Service (NHS) England AKI Algorithm as a gold standard. We additionally tested the algorithm's ability to detect AKI as defined by the Kidney Disease: Improving Global Outcomes (KDIGO) guidelines. We compared the algorithm's 3-fold cross-validation performance to the Sequential Organ Failure Assessment (SOFA) score for AKI identification in terms of area under the receiver operating characteristic (AUROC).ResultsThe algorithm demonstrated high AUROC for detecting and predicting NHS-defined AKI at all tested time points. The algorithm achieves AUROC of 0.872 (95% confidence interval [CI], 0.867-0.878) for AKI detection at time of onset. For prediction 12 hours before onset, the algorithm achieves an AUROC of 0.800 (95% CI, 0.792-0.809). For 24-hour predictions, the algorithm achieves AUROC of 0.795 (95% CI, 0.785-0.804). For 48-hour and 72-hour predictions, the algorithm achieves AUROC values of 0.761 (95% CI, 0.753-0.768) and 0.728 (95% CI, 0.719-0.737), respectively.LimitationsBecause of the retrospective nature of this study, we cannot draw any conclusions about the impact the algorithm's predictions will have on patient outcomes in a clinical setting.ConclusionsThe results of these experiments suggest that a machine learning-based AKI prediction tool may offer important prognostic capabilities for determining which patients are likely to suffer AKI, potentially allowing clinicians to intervene before kidney damage manifests.

Dataset Information

Machine Learning for Health: Algorithm Auditing & Quality Control.

Publications

Machine Learning for Health: Algorithm Auditing & Quality Control.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets