Transparency and reproducibility in artificial intelligence.
Ontology highlight
ABSTRACT:
SUBMITTER: Haibe-Kains B
PROVIDER: S-EPMC8144864 | biostudies-literature | 2020 Oct
REPOSITORIES: biostudies-literature
Publications
Transparency and reproducibility in artificial intelligence.
Haibe-Kains Benjamin B Adam George Alexandru GA Hosny Ahmed A Khodakarami Farnoosh F Waldron Levi L Wang Bo B McIntosh Chris C Goldenberg Anna A Kundaje Anshul A Greene Casey S CS Broderick Tamara T Hoffman Michael M MM Leek Jeffrey T JT Korthauer Keegan K Huber Wolfgang W Brazma Alvis A Pineau Joelle J Tibshirani Robert R Hastie Trevor T Ioannidis John P A JPA Quackenbush John J Aerts Hugo J W L HJWL
Nature 20201014 7829
Similar Datasets
Project description: Not available
| S-EPMC10810855 | biostudies-literature
Project description:TopicThis scoping review summarizes artificial intelligence (AI) reporting in ophthalmology literature in respect to model development and validation. We characterize the state of transparency in reporting of studies prospectively validating models for disease classification.Clinical relevanceUnderstanding what elements authors currently describe regarding their AI models may aid in the future standardization of reporting. This review highlights the need for transparency to facilitate the critical appraisal of models prior to clinical implementation, to minimize bias and inappropriate use. Transparent reporting can improve effective and equitable use in clinical settings.MethodsEligible articles (as of January 2022) from PubMed, Embase, Web of Science, and CINAHL were independently screened by 2 reviewers. All observational and clinical trial studies evaluating the performance of an AI model for disease classification of ophthalmic conditions were included. Studies were evaluated for reporting of parameters derived from reporting guidelines (CONSORT-AI, MI-CLAIM) and our previously published editorial on model cards. The reporting of these factors, which included basic model and dataset details (source, demographics), and prospective validation outcomes, were summarized.ResultsThirty-seven prospective validation studies were included in the scoping review. Eleven additional associated training and/or retrospective validation studies were included if this information could not be determined from the primary articles. These 37 studies validated 27 unique AI models; multiple studies evaluated the same algorithms (EyeArt, IDx-DR, and Medios AI). Details of model development were variably reported; 18 of 27 models described training dataset annotation and 10 of 27 studies reported training data distribution. Demographic information of training data was rarely reported; 7 of the 27 unique models reported age and gender and only 2 reported race and/or ethnicity. At the level of prospective clinical validation, age and gender of populations was more consistently reported (29 and 28 of 37 studies, respectively), but only 9 studies reported race and/or ethnicity data. Scope of use was difficult to discern for the majority of models. Fifteen studies did not state or imply primary users.ConclusionOur scoping review demonstrates variable reporting of information related to both model development and validation. The intention of our study was not to assess the quality of the factors we examined, but to characterize what information is, and is not, regularly reported. Our results suggest the need for greater transparency in the reporting of information necessary to determine the appropriateness and fairness of these tools prior to clinical use.Financial disclosuresProprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
| S-EPMC11000111 | biostudies-literature
Project description:The literature on artificial intelligence (AI) or machine learning (ML) in mental health and psychiatry lacks consensus on what "explainability" means. In the more general XAI (eXplainable AI) literature, there has been some convergence on explainability meaning model-agnostic techniques that augment a complex model (with internal mechanics intractable for human understanding) with a simpler model argued to deliver results that humans can comprehend. Given the differing usage and intended meaning of the term "explainability" in AI and ML, we propose instead to approximate model/algorithm explainability by understandability defined as a function of transparency and interpretability. These concepts are easier to articulate, to "ground" in our understanding of how algorithms and models operate and are used more consistently in the literature. We describe the TIFU (Transparency and Interpretability For Understandability) framework and examine how this applies to the landscape of AI/ML in mental health research. We argue that the need for understandablity is heightened in psychiatry because data describing the syndromes, outcomes, disorders and signs/symptoms possess probabilistic relationships to each other-as do the tentative aetiologies and multifactorial social- and psychological-determinants of disorders. If we develop and deploy AI/ML models, ensuring human understandability of the inputs, processes and outputs of these models is essential to develop trustworthy systems fit for deployment.
| S-EPMC9849399 | biostudies-literature
Project description:Social media dominate today's information ecosystem and provide valuable information for social research. Market researchers, social scientists, policymakers, government entities, public health researchers, and practitioners recognize the potential for social data to inspire innovation, support products and services, characterize public opinion, and guide decisions. The appeal of mining these rich datasets is clear. However, there is potential risk of data misuse, underscoring an equally huge and fundamental flaw in the research: there are no procedural standards and little transparency. Transparency across the processes of collecting and analyzing social media data is often limited due to proprietary algorithms. Spurious findings and biases introduced by artificial intelligence (AI) demonstrate the challenges this lack of transparency poses for research. Social media research remains a virtual "wild west," with no clear standards for reporting regarding data retrieval, preprocessing steps, analytic methods, or interpretation. Use of emerging generative AI technologies to augment social media analytics can undermine validity and replicability of findings, potentially turning this research into a "black box" enterprise. Clear guidance for social media analyses and reporting is needed to assure the quality of the resulting research. In this article, we propose criteria for evaluating the quality of studies using social media data, grounded in established scientific practice. We offer clear documentation guidelines to ensure that social data are used properly and transparently in research and applications. A checklist of disclosure elements to meet minimal reporting standards is proposed. These criteria will make it possible for scholars and practitioners to assess the quality, credibility, and comparability of research findings using digital data.
| S-EPMC10795517 | biostudies-literature
Project description:Trustworthy medical AI requires transparency about the development and testing of underlying algorithms to identify biases and communicate potential risks of harm. Abundant guidance exists on how to achieve transparency for medical AI products, but it is unclear whether publicly available information adequately informs about their risks. To assess this, we retrieved public documentation on the 14 available CE-certified AI-based radiology products of the II b risk category in the EU from vendor websites, scientific publications, and the European EUDAMED database. Using a self-designed survey, we reported on their development, validation, ethical considerations, and deployment caveats, according to trustworthy AI guidelines. We scored each question with either 0, 0.5, or 1, to rate if the required information was "unavailable", "partially available," or "fully available." The transparency of each product was calculated relative to all 55 questions. Transparency scores ranged from 6.4% to 60.9%, with a median of 29.1%. Major transparency gaps included missing documentation on training data, ethical considerations, and limitations for deployment. Ethical aspects like consent, safety monitoring, and GDPR-compliance were rarely documented. Furthermore, deployment caveats for different demographics and medical settings were scarce. In conclusion, public documentation of authorized medical AI products in Europe lacks sufficient public transparency to inform about safety and risks. We call on lawmakers and regulators to establish legally mandated requirements for public and substantive transparency to fulfill the promise of trustworthy AI for health.
| S-EPMC10919164 | biostudies-literature
Project description:BackgroundArtificial intelligence (AI) is as a branch of computer science that uses advanced computational methods such as machine learning (ML), to calculate and/or predict health outcomes and address patient and provider health needs. While these technologies show great promise for improving healthcare, especially in diabetes management, there are usability and safety concerns for both patients and providers about the use of AI/ML in healthcare management.ObjectivesTo support and ensure safe use of AI/ML technologies in healthcare, the team worked to better understand: 1) patient information and training needs, 2) the factors that influence patients' perceived value and trust in AI/ML healthcare applications; and 3) on how best to support safe and appropriate use of AI/ML enabled devices and applications among people living with diabetes.MethodsTo understand general patient perspectives and information needs related to the use of AI/ML in healthcare, we conducted a series of focus groups (n=9) and interviews (n=3) with patients (n=40) and interviews with providers (n=6) in Alaska, Idaho, and Virginia. Grounded Theory guided data gathering, synthesis, and analysis. Thematic content and constant comparison analysis were used to identify relevant themes and sub-themes. Inductive approaches were used to link data to key concepts including preferred patient-provider-interactions, patient perceptions of trust, accuracy, value, assurances, and information transparency.ResultsKey summary themes and recommendations focused on: 1) patient preferences for AI/ML enabled device and/or application information; 2) patient and provider AI/ML-related device and/or application training needs; 3) factors contributing to patient and provider trust in AI/ML enabled devices and/or application; and 4) AI/ML-related device and/or application functionality and safety considerations. A number of participant (patients and providers) recommendations to improve device functionality to guide information and labeling mandates (e.g., links to online video resources, and access to 24/7 live in-person or virtual emergency support). Other patient recommendations include: 1) access to practice devices; 2) connection to local supports and reputable community resources; 3) simplified display and alert limits.ConclusionRecommendations from both patients and providers could be used by Federal Oversight Agencies to improve utilization of AI/ML monitoring of technology use in diabetes, improving device safety and efficacy.
| S-EPMC10851077 | biostudies-literature
Project description:ImportanceClinical artificial intelligence (AI) algorithms have the potential to improve clinical care, but fair, generalizable algorithms depend on the clinical data on which they are trained and tested.ObjectiveTo assess whether data sets used for training diagnostic AI algorithms addressing skin disease are adequately described and to identify potential sources of bias in these data sets.Data sourcesIn this scoping review, PubMed was used to search for peer-reviewed research articles published between January 1, 2015, and November 1, 2020, with the following paired search terms: deep learning and dermatology, artificial intelligence and dermatology, deep learning and dermatologist, and artificial intelligence and dermatologist.Study selectionStudies that developed or tested an existing deep learning algorithm for triage, diagnosis, or monitoring using clinical or dermoscopic images of skin disease were selected, and the articles were independently reviewed by 2 investigators to verify that they met selection criteria.Consensus processData set audit criteria were determined by consensus of all authors after reviewing existing literature to highlight data set transparency and sources of bias.ResultsA total of 70 unique studies were included. Among these studies, 1 065 291 images were used to develop or test AI algorithms, of which only 257 372 (24.2%) were publicly available. Only 14 studies (20.0%) included descriptions of patient ethnicity or race in at least 1 data set used. Only 7 studies (10.0%) included any information about skin tone in at least 1 data set used. Thirty-six of the 56 studies developing new AI algorithms for cutaneous malignant neoplasms (64.3%) met the gold standard criteria for disease labeling. Public data sets were cited more often than private data sets, suggesting that public data sets contribute more to new development and benchmarks.Conclusions and relevanceThis scoping review identified 3 issues in data sets that are used to develop and test clinical AI algorithms for skin disease that should be addressed before clinical translation: (1) sparsity of data set characterization and lack of transparency, (2) nonstandard and unverified disease labels, and (3) inability to fully assess patient diversity used for algorithm development and testing.
| S-EPMC9379852 | biostudies-literature
Project description:PurposeThis study aimed to evaluate the quality of artificial intelligence (AI)/machine learning (ML) studies on hemorrhagic stroke using the Minimum Information for Medical AI Reporting (MINIMAR) and Minimum Information About Clinical Artificial Intelligence Modeling (MI-CLAIM) frameworks to promote clinical application.Materials and methodsPubMed, MEDLINE, and Embase were searched for AI/ML studies on hemorrhagic stroke. Out of the 531 articles found, 29 relevant original research articles were included. MINIMAR and MI-CLAIM scores were assigned by two experienced radiologists to assess the quality of the studies.ResultsWe analyzed 29 investigations that utilized AI/ML in the field of hemorrhagic stroke, involving a median of 224.5 patients. The majority of studies focused on diagnostic outcomes using computed tomography scans (89.7%) and were published in computer science journals (48.3%). The overall adherence rates to reporting guidelines, as assessed through the MINIMAR and MI-CLAIM frameworks, were 47.6% and 46.0%, respectively. In MINIMAR, none of the studies reported the socioeconomic status of the patients or how missing values had been addressed. In MI-CLAIM, only two studies applied model-examination techniques to improve model interpretability. Transparency and reproducibility were limited, as only 10.3% of the studies had publicly shared their code. Cohen's kappa between the two radiologists was 0.811 and 0.779 for MINIMAR and MI-CLAIM, respectively.ConclusionThe overall reporting quality of published AI/ML studies on hemorrhagic stroke is suboptimal. It is necessary to incorporate model examination techniques for interpretability and promote code openness to enhance transparency and increase the clinical applicability of AI/ML studies.
| S-EPMC11427125 | biostudies-literature
Project description:When developing artificial intelligence (AI) software for applications in radiology, the underlying research must be transferable to other real-world problems. To verify to what degree this is true, we reviewed research on AI algorithms for computed tomography of the head. A systematic review was conducted according to the preferred reporting items for systematic reviews and meta-analyses. We identified 83 articles and analyzed them in terms of transparency of data and code, pre-processing, type of algorithm, architecture, hyperparameter, performance measure, and balancing of dataset in relation to epidemiology. We also classified all articles by their main functionality (classification, detection, segmentation, prediction, triage, image reconstruction, image registration, fusion of imaging modalities). We found that only a minority of authors provided open source code (10.15%, n 0 7), making the replication of results difficult. Convolutional neural networks were predominantly used (32.61%, n = 15), whereas hyperparameters were less frequently reported (32.61%, n = 15). Data sets were mostly from single center sources (84.05%, n = 58), increasing the susceptibility of the models to bias, which increases the error rate of the models. The prevalence of brain lesions in the training (0.49 ± 0.30) and testing (0.45 ± 0.29) datasets differed from real-world epidemiology (0.21 ± 0.28), which may overestimate performances. This review highlights the need for open source code, external validation, and consideration of disease prevalence.
| S-EPMC9613832 | biostudies-literature
Project description:IntroductionWe aimed to assess the reproducibility of empirical research by determining the availability of components required for replication of a study, including materials, raw data, analysis scripts, protocols, and preregistration.MethodsWe used the National Library of Medicine catalog to identify MEDLINE-indexed emergency medicine (EM) journals. Thirty journals met the inclusion criteria. From January 1, 2014-December 31, 2018, 300 publications were randomly sampled using a PubMed search. Additionally, we included four high-impact general medicine journals, which added 106 publications. Two investigators were blinded for independent extraction. Extracted data included statements regarding the availability of materials, data, analysis scripts, protocols, and registration.ResultsAfter the search, we found 25,473 articles, from which we randomly selected 300. Of the 300, only 287 articles met the inclusion criteria. Additionally, we added 106 publications from high-impact journals of which 77 met the inclusion criteria. Together, 364 publications were included, of which 212 articles contained empirical data to analyze. Of the eligible empirical articles, 2.49%, (95% confidence interval [CI], 0.33% to 4.64%] provided a material statement, 9.91% (95% CI, 5.88% to 13.93%) provided a data statement, 0 provided access to analysis scripts, 25.94% (95% CI, 20.04% to 31.84%) linked the protocol, and 39.15% (95% CI, 32.58% to 45.72%) were preregistered.ConclusionStudies in EM lack indicators required for reproducibility. The majority of studies fail to report factors needed to reproduce research to ensure credibility. Thus, an intervention is required and can be achieved through the collaboration of researchers, peer reviewers, funding agencies, and journals.
| S-EPMC8328179 | biostudies-literature