Evaluating Artificial Intelligence Responses to Public Health Questions.
Ontology highlight
ABSTRACT:
SUBMITTER: Ayers JW
PROVIDER: S-EPMC10248742 | biostudies-literature | 2023 Jun
REPOSITORIES: biostudies-literature
Publications
Evaluating Artificial Intelligence Responses to Public Health Questions.
Ayers John W JW Zhu Zechariah Z Poliak Adam A Leas Eric C EC Dredze Mark M Hogarth Michael M Smith Davey M DM
JAMA network open 20230601 6
Similar Datasets
Project description: Not available
| S-EPMC10666700 | biostudies-literature
Project description:IntroductionOptic disc drusen (ODD) are acellular deposits in the optic nerve head, which are most often benign and asymptomatic. Patients may develop visual field defects and be at increased risk of ischemic co-morbidities. As ODD can be difficult to distinguish from papilledema, patients are at risk of unnecessary clinical workups. Patient information is a key aspect of ODD management. In this study, we explored the accuracy of ChatGPT responses for typical patient questions on ODD.MethodsTwo content experts reached consensus on 20 typical patient questions. We retrieved five separate responses for each question from ChatGPT, totaling 100 responses. Each response was evaluated on a 5-point Likert-scale on accuracy by each content expert in an individual fashion.ResultsThe two experts were in fair/substantial agreement in the evaluation of responses (Cronbach's alpha: 0.64). Of the 100 responses, 17 were relevant and without any inaccuracies, 78 were relevant and with inaccuracies not being harmful, and five were relevant and with inaccuracies potentially harmful. The lowest accuracy scores were obtained for questions dealing with treatment and prognosis.ConclusionsChatGPT often provides relevant answers for patient questions on ODD, but inaccuracies become potentially harmful when questions deal with treatment and prognosis.
| S-EPMC10640407 | biostudies-literature
Project description: Not available
| S-EPMC11347124 | biostudies-literature
Project description:The integration of artificial intelligence (AI) into health communication systems has introduced a transformative approach to public health management, particularly during public health emergencies, capable of reaching billions through familiar digital channels. This paper explores the utility and implications of generalist conversational artificial intelligence (CAI) advanced AI systems trained on extensive datasets to handle a wide range of conversational tasks across various domains with human-like responsiveness. The specific focus is on the application of generalist CAI within messaging services, emphasizing its potential to enhance public health communication. We highlight the evolution and current applications of AI-driven messaging services, including their ability to provide personalized, scalable, and accessible health interventions. Specifically, we discuss the integration of large language models and generative AI in mainstream messaging platforms, which potentially outperform traditional information retrieval systems in public health contexts. We report a critical examination of the advantages of generalist CAI in delivering health information, with a case of its operationalization during the COVID-19 pandemic and propose the strategic deployment of these technologies in collaboration with public health agencies. In addition, we address significant challenges and ethical considerations, such as AI biases, misinformation, privacy concerns, and the required regulatory oversight. We envision a future with leverages generalist CAI in messaging apps, proposing a multiagent approach to enhance the reliability and specificity of health communications. We hope this commentary initiates the necessary conversations and research toward building evaluation approaches, adaptive strategies, and robust legal and technical frameworks to fully realize the benefits of AI-enhanced communications in public health, aiming to ensure equitable and effective health outcomes across diverse populations.
| S-EPMC11791462 | biostudies-literature
Project description:IntroductionAI technologies have the potential to transform patient care. AI has been used to aid in differential diagnosis and treatment planning for psychiatric disorders, administer therapeutic protocols, assist with interpretation of cognitive testing, and patient treatment planning. Despite advancements, AI has notable limitations and remains understudied and further research on its strengths and limitations in patient care is required. This study explored the responses of AI (Chat-GPT 3.5) and trained clinicians to commonly asked patient questions.MethodsThree clinicians and AI provided responses to five dementia/geriatric healthcare-related questions. Responses were analyzed by a fourth, blinded clinician for clarity, accuracy, relevance, depth, and ease of understanding and to determine which response was AI generated.ResultsAI responses were rated highest in ease of understanding and depth across all responses and tied for first for clarity, accuracy, and relevance. The rating for AI generated responses was 4.6/5 (SD = 0.26); the clinician s' responses were 4.3 (SD = 0.67), 4.2 (SD = 0.52), and 3.9 (SD = 0.59), respectively. The AI generated answers were identified in 4/5 instances.ConclusionsAI responses were rated more highly and consistently on each question individually and overall than clinician answers demonstrating that AI could produce good responses to potential patient questions. However, AI responses were easily distinguishable from those of clinicians. Although AI has the potential to positively impact healthcare, concerns are raised regarding difficulties discerning AI from human generated material, the increased potential for proliferation of misinformation, data security concerns, and more.
| S-EPMC11306168 | biostudies-literature
Project description:The growing use of artificial intelligence (AI) in health care has raised questions about who should be held liable for medical errors that result from care delivered jointly by physicians and algorithms. In this survey study comparing views of physicians and the U.S. public, we find that the public is significantly more likely to believe that physicians should be held responsible when an error occurs during care delivered with medical AI, though the majority of both physicians and the public hold this view (66.0% vs 57.3%; P = .020). Physicians are more likely than the public to believe that vendors (43.8% vs 32.9%; P = .004) and healthcare organizations should be liable for AI-related medical errors (29.2% vs 22.6%; P = .05). Views of medical liability did not differ by clinical specialty. Among the general public, younger people are more likely to hold nearly all parties liable.
| S-EPMC8279784 | biostudies-literature
Project description:The coronavirus disease 2019 (COVID-19) pandemic has overwhelmed healthcare services, faced with the twin challenges in acutely meeting the medical needs of patients with COVID-19 while continuing essential services for non-COVID-19 illnesses. The need to re-invent, re-organize and transform healthcare and co-ordinate clinical services at a population level is urgent as countries that controlled initial outbreaks start to experience resurgences. A wide range of digital health solutions have been proposed, although the extent of successful real-world applications of these technologies is unclear. This study aims to review applications of artificial intelligence (AI), telehealth, and other relevant digital health solutions for public health responses in the healthcare operating environment amidst the COVID-19 pandemic. A systematic scoping review was performed to identify potentially relevant reports. Key findings include a large body of evidence for various clinical and operational applications of telehealth (40.1%, n = 99/247). Although a large quantity of reports investigated applications of artificial intelligence (AI) (44.9%, n = 111/247) and big data analytics (36.0%, n = 89/247), weaknesses in study design limit generalizability and translation, highlighting the need for more pragmatic real-world investigations. There were also few descriptions of applications for the internet of things (IoT) (2.0%, n = 5/247), digital platforms for communication (DC) (10.9%, 27/247), digital solutions for data management (DM) (1.6%, n = 4/247), and digital structural screening (DS) (8.9%, n = 22/247); representing gaps and opportunities for digital public health. Finally, the performance of digital health technology for operational applications related to population surveillance and points of entry have not been adequately evaluated.
| S-EPMC7910557 | biostudies-literature
Project description:BackgroundMental disorders (MDs) impose heavy burdens on health care (HC) systems and affect a growing number of people worldwide. The use of mobile health (mHealth) apps empowered by artificial intelligence (AI) is increasingly being resorted to as a possible solution.ObjectiveThis study adopted a topic modeling (TM) approach to investigate the public trust in AI apps in mental health care (MHC) by identifying the dominant topics and themes in user reviews of the 8 most relevant mental health (MH) apps with the largest numbers of reviewers.MethodsWe searched Google Play for the top MH apps with the largest numbers of reviewers, from which we selected the most relevant apps. Subsequently, we extracted data from user reviews posted from January 1, 2020, to April 2, 2022. After cleaning the extracted data using the Python text processing tool spaCy, we ascertained the optimal number of topics, drawing on the coherence scores and used latent Dirichlet allocation (LDA) TM to generate the most salient topics and related terms. We then classified the ascertained topics into different theme categories by plotting them onto a 2D plane via multidimensional scaling using the pyLDAvis visualization tool. Finally, we analyzed these topics and themes qualitatively to better understand the status of public trust in AI apps in MHC.ResultsFrom the top 20 MH apps with the largest numbers of reviewers retrieved, we chose the 8 (40%) most relevant apps: (1) Wysa: Anxiety Therapy Chatbot; (2) Youper Therapy; (3) MindDoc: Your Companion; (4) TalkLife for Anxiety, Depression & Stress; (5) 7 Cups: Online Therapy for Mental Health & Anxiety; (6) BetterHelp-Therapy; (7) Sanvello; and (8) InnerHour. These apps provided 14.2% (n=559), 11.0% (n=431), 13.7% (n=538), 8.8% (n=356), 14.1% (n=554), 11.9% (n=468), 9.2% (n=362), and 16.9% (n=663) of the collected 3931 reviews, respectively. The 4 dominant topics were topic 4 (cheering people up; n=1069, 27%), topic 3 (calming people down; n=1029, 26%), topic 2 (helping figure out the inner world; n=963, 25%), and topic 1 (being an alternative or complement to a therapist; n=870, 22%). Based on topic coherence and intertopic distance, topics 3 and 4 were combined into theme 3 (dispelling negative emotions), while topics 2 and 1 remained 2 separate themes: theme 2 (helping figure out the inner world) and theme 1 (being an alternative or complement to a therapist), respectively. These themes and topics, though involving some dissenting voices, reflected an overall high status of trust in AI apps.ConclusionsThis is the first study to investigate the public trust in AI apps in MHC from the perspective of user reviews using the TM technique. The automatic text analysis and complementary manual interpretation of the collected data allowed us to discover the dominant topics hidden in a data set and categorize these topics into different themes to reveal an overall high degree of public trust. The dissenting voices from users, though only a few, can serve as indicators for health providers and app developers to jointly improve these apps, which will ultimately facilitate the treatment of prevalent MDs and alleviate the overburdened HC systems worldwide.
| S-EPMC9758643 | biostudies-literature
Project description:Artificial intelligence (AI) has the potential to improve public health's ability to promote the health of all people in all communities. To successfully realize this potential and use AI for public health functions it is important for public health organizations to thoughtfully develop strategies for AI implementation. Six key priorities for successful use of AI technologies by public health organizations are discussed: 1) Contemporary data governance; 2) Investment in modernized data and analytic infrastructure and procedures; 3) Addressing the skills gap in the workforce; 4) Development of strategic collaborative partnerships; 5) Use of good AI practices for transparency and reproducibility, and; 6) Explicit consideration of equity and bias.
| S-EPMC9682716 | biostudies-literature
Project description:Billions of dollars are being invested into developing medical artificial intelligence (AI) systems and yet public opinion of AI in the medical field seems to be mixed. Although high expectations for the future of medical AI do exist in the American public, anxiety and uncertainty about what it can do and how it works is widespread. Continuing evaluation of public opinion on AI in healthcare is necessary to ensure alignment between patient attitudes and the technologies adopted. We conducted a representative-sample survey (total N = 203) to measure the trust of the American public towards medical AI. Primarily, we contrasted preferences for AI and human professionals to be medical decision-makers. Additionally, we measured expectations for the impact and use of medical AI in the future. We present four noteworthy results: (1) The general public strongly prefers human medical professionals make medical decisions, while at the same time believing they are more likely to make culturally biased decisions than AI. (2) The general public is more comfortable with a human reading their medical records than an AI, both now and "100 years from now." (3) The general public is nearly evenly split between those who would trust their own doctor to use AI and those who would not. (4) Respondents expect AI will improve medical treatment but more so in the distant future than immediately.
| S-EPMC10635466 | biostudies-literature