Dataset Information

A Process Evaluation Examining the Performance, Adherence, and Acceptability of a Physical Activity and Diet Artificial Intelligence Virtual Health Assistant.

ABSTRACT: Artificial intelligence virtual health assistants are a promising emerging technology. This study is a process evaluation of a 12-week pilot physical activity and diet program delivered by virtual assistant "Paola". This single-arm repeated measures study (n = 28, aged 45-75 years) was evaluated on technical performance (accuracy of conversational exchanges), engagement (number of weekly check-ins completed), adherence (percentage of step goal and recommended food servings), and user feedback. Paola correctly asked scripted questions and responded to participants during the check-ins 97% and 96% of the time, respectively, but correctly responded to spontaneous exchanges only 21% of the time. Participants completed 63% of weekly check-ins and conducted a total of 3648 exchanges. Mean dietary adherence was 91% and was lowest for discretionary foods, grains, red meat, and vegetables. Participants met their step goal 59% of the time. Participants enjoyed the program and found Paola useful during check-ins but not for spontaneous exchanges. More in-depth knowledge, personalized advice and spontaneity were identified as important improvements. Virtual health assistants should ensure an adequate knowledge base and ability to recognize intents and entities, include personality and spontaneity, and provide ongoing technical troubleshooting of the virtual assistant to ensure the assistant remains effective.

SUBMITTER: Davis CR

PROVIDER: S-EPMC7729471 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Process Evaluation Examining the Performance, Adherence, and Acceptability of a Physical Activity and Diet Artificial Intelligence Virtual Health Assistant.

Davis Courtney R CR Murphy Karen J KJ Curtis Rachel G RG Maher Carol A CA

International journal of environmental research and public health 20201207 23

Artificial intelligence virtual health assistants are a promising emerging technology. This study is a process evaluation of a 12-week pilot physical activity and diet program delivered by virtual assistant "Paola". This single-arm repeated measures study (<i>n</i> = 28, aged 45-75 years) was evaluated on technical performance (accuracy of conversational exchanges), engagement (number of weekly check-ins completed), adherence (percentage of step goal and recommended food servings), and user feed ...[more]

PMID: 33297456

Similar Datasets

Project description:Simulation-based training is increasingly being used for assessment and training of psychomotor skills involved in medicine. The application of artificial intelligence and machine learning technologies has provided new methodologies to utilize large amounts of data for educational purposes. A significant criticism of the use of artificial intelligence in education has been a lack of transparency in the algorithms' decision-making processes. This study aims to 1) introduce a new framework using explainable artificial intelligence for simulation-based training in surgery, and 2) validate the framework by creating the Virtual Operative Assistant, an automated educational feedback platform. Twenty-eight skilled participants (14 staff neurosurgeons, 4 fellows, 10 PGY 4-6 residents) and 22 novice participants (10 PGY 1-3 residents, 12 medical students) took part in this study. Participants performed a virtual reality subpial brain tumor resection task on the NeuroVR simulator using a simulated ultrasonic aspirator and bipolar. Metrics of performance were developed, and leave-one-out cross validation was employed to train and validate a support vector machine in Matlab. The classifier was combined with a unique educational system to build the Virtual Operative Assistant which provides users with automated feedback on their metric performance with regards to expert proficiency performance benchmarks. The Virtual Operative Assistant successfully classified skilled and novice participants using 4 metrics with an accuracy, specificity and sensitivity of 92, 82 and 100%, respectively. A 2-step feedback system was developed to provide participants with an immediate visual representation of their standing related to expert proficiency performance benchmarks. The educational system outlined establishes a basis for the potential role of integrating artificial intelligence and virtual reality simulation into surgical educational teaching. The potential of linking expertise classification, objective feedback based on proficiency benchmarks, and instructor input creates a novel educational tool by integrating these three components into a formative educational paradigm.

Project description:BackgroundDue to an aging population, the demand for many services is exceeding the capacity of the clinical workforce. As a result, staff are facing a crisis of burnout from being pressured to deliver high-volume workloads, driving increasing costs for providers. Artificial intelligence (AI), in the form of conversational agents, presents a possible opportunity to enable efficiency in the delivery of care.ObjectiveThis study aims to evaluate the effectiveness, usability, and acceptability of Dora agent: Ufonia's autonomous voice conversational agent, an AI-enabled autonomous telemedicine call for the detection of postoperative cataract surgery patients who require further assessment. The objectives of this study are to establish Dora's efficacy in comparison with an expert clinician, determine baseline sensitivity and specificity for the detection of true complications, evaluate patient acceptability, collect evidence for cost-effectiveness, and capture data to support further development and evaluation.MethodsUsing an implementation science construct, the interdisciplinary study will be a mixed methods phase 1 pilot establishing interobserver reliability of the system, usability, and acceptability. This will be done using the following scales and frameworks: the system usability scale; assessment of Health Information Technology Interventions in Evidence-Based Medicine Evaluation Framework; the telehealth usability questionnaire; and the Non-Adoption, Abandonment, and Challenges to the Scale-up, Spread and Suitability framework.ResultsThe evaluation is expected to show that conversational technology can be used to conduct an accurate assessment and that it is acceptable to different populations with different backgrounds. In addition, the results will demonstrate how successfully the system can be delivered in organizations with different clinical pathways and how it can be integrated with their existing platforms.ConclusionsThe project's key contributions will be evidence of the effectiveness of AI voice conversational agents and their associated usability and acceptability.International registered report identifier (irrid)PRR1-10.2196/27227.

Project description:BackgroundChatbots empowered by artificial intelligence (AI) can increasingly engage in natural conversations and build relationships with users. Applying AI chatbots to lifestyle modification programs is one of the promising areas to develop cost-effective and feasible behavior interventions to promote physical activity and a healthy diet.ObjectiveThe purposes of this perspective paper are to present a brief literature review of chatbot use in promoting physical activity and a healthy diet, describe the AI chatbot behavior change model our research team developed based on extensive interdisciplinary research, and discuss ethical principles and considerations.MethodsWe conducted a preliminary search of studies reporting chatbots for improving physical activity and/or diet in four databases in July 2020. We summarized the characteristics of the chatbot studies and reviewed recent developments in human-AI communication research and innovations in natural language processing. Based on the identified gaps and opportunities, as well as our own clinical and research experience and findings, we propose an AI chatbot behavior change model.ResultsOur review found a lack of understanding around theoretical guidance and practical recommendations on designing AI chatbots for lifestyle modification programs. The proposed AI chatbot behavior change model consists of the following four components to provide such guidance: (1) designing chatbot characteristics and understanding user background; (2) building relational capacity; (3) building persuasive conversational capacity; and (4) evaluating mechanisms and outcomes. The rationale and evidence supporting the design and evaluation choices for this model are presented in this paper.ConclusionsAs AI chatbots become increasingly integrated into various digital communications, our proposed theoretical framework is the first step to conceptualize the scope of utilization in health behavior change domains and to synthesize all possible dimensions of chatbot features to inform intervention design and evaluation. There is a need for more interdisciplinary work to continue developing AI techniques to improve a chatbot's relational and persuasive capacities to change physical activity and diet behaviors with strong ethical principles.

Project description:BackgroundThis systematic review aimed to evaluate AI chatbot characteristics, functions, and core conversational capacities and investigate whether AI chatbot interventions were effective in changing physical activity, healthy eating, weight management behaviors, and other related health outcomes.MethodsIn collaboration with a medical librarian, six electronic bibliographic databases (PubMed, EMBASE, ACM Digital Library, Web of Science, PsycINFO, and IEEE) were searched to identify relevant studies. Only randomized controlled trials or quasi-experimental studies were included. Studies were screened by two independent reviewers, and any discrepancy was resolved by a third reviewer. The National Institutes of Health quality assessment tools were used to assess risk of bias in individual studies. We applied the AI Chatbot Behavior Change Model to characterize components of chatbot interventions, including chatbot characteristics, persuasive and relational capacity, and evaluation of outcomes.ResultsThe database search retrieved 1692 citations, and 9 studies met the inclusion criteria. Of the 9 studies, 4 were randomized controlled trials and 5 were quasi-experimental studies. Five out of the seven studies suggest chatbot interventions are promising strategies in increasing physical activity. In contrast, the number of studies focusing on changing diet and weight status was limited. Outcome assessments, however, were reported inconsistently across the studies. Eighty-nine and thirty-three percent of the studies specified a name and gender (i.e., woman) of the chatbot, respectively. Over half (56%) of the studies used a constrained chatbot (i.e., rule-based), while the remaining studies used unconstrained chatbots that resemble human-to-human communication.ConclusionChatbots may improve physical activity, but we were not able to make definitive conclusions regarding the efficacy of chatbot interventions on physical activity, diet, and weight management/loss. Application of AI chatbots is an emerging field of research in lifestyle modification programs and is expected to grow exponentially. Thus, standardization of designing and reporting chatbot interventions is warranted in the near future.Systematic review registrationInternational Prospective Register of Systematic Reviews (PROSPERO): CRD42020216761 .

Project description:The drug development process is a major challenge in the pharmaceutical industry since it takes a substantial amount of time and money to move through all the phases of developing of a new drug. One extensively used method to minimize the cost and time for the drug development process is computer-aided drug design (CADD). CADD allows better focusing on experiments, which can reduce the time and cost involved in researching new drugs. In this context, structure-based virtual screening (SBVS) is robust and useful and is one of the most promising in silico techniques for drug design. SBVS attempts to predict the best interaction mode between two molecules to form a stable complex, and it uses scoring functions to estimate the force of non-covalent interactions between a ligand and molecular target. Thus, scoring functions are the main reason for the success or failure of SBVS software. Many software programs are used to perform SBVS, and since they use different algorithms, it is possible to obtain different results from different software using the same input. In the last decade, a new technique of SBVS called consensus virtual screening (CVS) has been used in some studies to increase the accuracy of SBVS and to reduce the false positives obtained in these experiments. An indispensable condition to be able to utilize SBVS is the availability of a 3D structure of the target protein. Some virtual databases, such as the Protein Data Bank, have been created to store the 3D structures of molecules. However, sometimes it is not possible to experimentally obtain the 3D structure. In this situation, the homology modeling methodology allows the prediction of the 3D structure of a protein from its amino acid sequence. This review presents an overview of the challenges involved in the use of CADD to perform SBVS, the areas where CADD tools support SBVS, a comparison between the most commonly used tools, and the techniques currently used in an attempt to reduce the time and cost in the drug development process. Finally, the final considerations demonstrate the importance of using SBVS in the drug development process.

Project description:ObjectiveTo determine the rigor, generalizability, and reproducibility of published classification and detection artificial intelligence (AI) models for prostate cancer (PCa) on MRI using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) guidelines, a 42-item checklist that is considered a measure of best practice for presenting and reviewing medical imaging AI research.Materials and methodsThis review searched English literature for studies proposing PCa AI detection and classification models on MRI. Each study was evaluated with the CLAIM checklist. The additional outcomes for which data were sought included measures of AI model performance (eg, area under the curve [AUC], sensitivity, specificity, free-response operating characteristic curves), training and validation and testing group sample size, AI approach, detection versus classification AI, public data set utilization, MRI sequences used, and definition of gold standard for ground truth. The percentage of CLAIM checklist fulfillment was used to stratify studies into quartiles. Wilcoxon's rank-sum test was used for pair-wise comparisons.ResultsIn all, 75 studies were identified, and 53 studies qualified for analysis. The original CLAIM items that most studies did not fulfill includes item 12 (77% no): de-identification methods; item 13 (68% no): handling missing data; item 15 (47% no): rationale for choosing ground truth reference standard; item 18 (55% no): measurements of inter- and intrareader variability; item 31 (60% no): inclusion of validated interpretability maps; item 37 (92% no): inclusion of failure analysis to elucidate AI model weaknesses. An AUC score versus percentage CLAIM fulfillment quartile revealed a significant difference of the mean AUC scores between quartile 1 versus quartile 2 (0.78 versus 0.86, P = .034) and quartile 1 versus quartile 4 (0.78 versus 0.89, P = .003) scores. Based on additional information and outcome metrics gathered in this study, additional measures of best practice are defined. These new items include disclosure of public dataset usage, ground truth definition in comparison to other referenced works in the defined task, and sample size power calculation.ConclusionA large proportion of AI studies do not fulfill key items in CLAIM guidelines within their methods and results sections. The percentage of CLAIM checklist fulfillment is weakly associated with improved AI model performance. Additions or supplementations to CLAIM are recommended to improve publishing standards and aid reviewers in determining study rigor.

Project description:ImportanceThe transformative potential of artificial intelligence (AI), particularly via large language models, is increasingly being manifested in healthcare. Dietary interventions are foundational to weight management efforts, but whether AI techniques are presently capable of generating clinically applicable diet plans has not been evaluated.ObjectiveOur study sought to evaluate the potential of personalized AI-generated weight-loss diet plans for clinical applications by employing a survey-based assessment conducted by experts in the fields of obesity medicine and clinical nutrition.Design setting and participantsWe utilized ChatGPT (4.0) to create weight-loss diet plans and selected two control diet plans from tertiary medical centers for comparison. Dietitians, physicians, and nurse practitioners specializing in obesity medicine or nutrition were invited to provide feedback on the AI-generated plans. Each plan was assessed blindly based on its effectiveness, balanced-ness, comprehensiveness, flexibility, and applicability. Personalized plans for hypothetical patients with specific health conditions were also evaluated.Main outcomes and measuresThe primary outcomes measured included the indistinguishability of the AI diet plan from human-created plans, and the potential of personalized AI-generated diet plans for real-world clinical applications.ResultsOf 95 participants, 67 completed the survey and were included in the final analysis. No significant differences were found among the three weight-loss diet plans in any evaluation category. Among the 14 experts who believed that they could identify the AI plan, only five did so correctly. In an evaluation involving 57 experts, the AI-generated personalized weight-loss diet plan was assessed, with scores above neutral for all evaluation variables. Several limitations, of the AI-generated plans were highlighted, including conflicting dietary considerations, lack of affordability, and insufficient specificity in recommendations, such as exact portion sizes. These limitations suggest that refining inputs could enhance the quality and applicability of AI-generated diet plans.ConclusionDespite certain limitations, our study highlights the potential of AI-generated diet plans for clinical applications. AI-generated dietary plans were frequently indistinguishable from diet plans widely used at major tertiary medical centers. Although further refinement and prospective studies are needed, these findings illustrate the potential of AI in advancing personalized weight-centric care.

Project description:BackgroundThere has been an increasing trend toward pulmonary segmentectomies to treat early-stage lung cancer, small intrapulmonary metastases, and localized benign pathology. A complete preoperative understanding of pulmonary anatomy is essential for accurate surgical planning and case selection. Identifying intersegmental divisions is extremely difficult when performed on computed tomography. For the preoperative planning of segmentectomies, virtual reality (VR) and artificial intelligence could allow 3-dimensional visualization of the complex anatomy of pulmonary segmental divisions, vascular arborization, and bronchial anatomy. This technology can be applied by surgeons preoperatively to gain better insight into a patient's anatomy for planning segmentectomy.MethodsIn this prospective observational pilot study, we aim to assess and demonstrate the technical feasibility and clinical applicability of the first dedicated artificial intelligence-based and immersive 3-dimensional-VR platform (PulmoVR; jointly developed and manufactured by Department of Cardiothoracic Surgery [Erasmus Medical Center, Rotterdam, The Netherlands], MedicalVR [Amsterdam, The Netherlands], EVOCS Medical Image Communication [Fysicon BV, Oss, The Netherlands], and Thirona [Nijmegen, The Netherlands]) for preoperative planning of video-assisted thoracoscopic segmentectomies.ResultsA total of 10 eligible patients for segmentectomy were included in this study after referral through the institutional thoracic oncology multidisciplinary team. PulmoVR was successfully applied as a supplementary imaging tool to perform video-assisted thoracoscopic segmentectomies. In 40% of the cases, the surgical strategy was adjusted due to the 3-dimensional-VR-based evaluation of anatomy. This underlines the potential benefit of additional VR-guided planning of segmentectomy for both surgeon and patient.ConclusionsOur study demonstrates the successful development and clinical application of the first dedicated artificial intelligence and VR platform for the planning of pulmonary segmentectomy. This is the first study that shows an immersive virtual reality-based application for preoperative planning of segmentectomy to the best of our knowledge.

Dataset Information

A Process Evaluation Examining the Performance, Adherence, and Acceptability of a Physical Activity and Diet Artificial Intelligence Virtual Health Assistant.

Publications

A Process Evaluation Examining the Performance, Adherence, and Acceptability of a Physical Activity and Diet Artificial Intelligence Virtual Health Assistant.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets