Dataset Information

Technology Evaluation and Assessment Criteria for Health Apps (TEACH-Apps): Pilot Study.

ABSTRACT: BACKGROUND:Despite the emergence of app evaluation tools, there remains no well-defined process receptive to diverse local needs, rigorous standards, and current content. The need for such a process to assist in the implementation of app evaluation across all medical fields is evident. Such a process has the potential to increase stakeholder engagement and catalyze interest and engagement with present-day app evaluation models. OBJECTIVE:This study aimed to develop and pilot test the Technology Evaluation and Assessment Criteria for Health apps (TEACH-apps). METHODS:Tailoring a well-known implementation framework, Replicating Effective Programs, we present a new process to approach the challenges faced in implementing app evaluation tools today. As a culmination of our experience implementing this process and feedback from stakeholders, we present the four-part process to aid the implementation of mobile health technology. This paper outlines the theory, evidence, and initial versions of the process. RESULTS:The TEACH-apps process is designed to be broadly usable and widely applicable across all fields of health. The process comprises four parts: (1) preconditions (eg, gathering apps and considering local needs), (2) preimplementation (eg, customizing criteria and offering digital skills training), (3) implementation (eg, evaluating apps and creating educational handouts), and (4) maintenance and evolution (eg, repeating the process every 90 days and updating content). TEACH-apps has been tested internally at our hospital, and there is growing interest in partnering health care facilities to test the system at their sites. CONCLUSIONS:This implementation framework introduces a process that equips stakeholders, clinicians, and users with the foundational tools to make informed decisions around app use and increase app evaluation engagement. The application of this process may lead to the selection of more culturally appropriate and clinically relevant tools in health care.

SUBMITTER: Camacho E

PROVIDER: S-EPMC7484774 | biostudies-literature | 2020 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Technology Evaluation and Assessment Criteria for Health Apps (TEACH-Apps): Pilot Study.

Camacho Erica E Hoffman Liza L Lagan Sarah S Rodriguez-Villa Elena E Rauseo-Ricupero Natali N Wisniewski Hannah H Henson Philip P Torous John J

Journal of medical Internet research 20200827 8

<h4>Background</h4>Despite the emergence of app evaluation tools, there remains no well-defined process receptive to diverse local needs, rigorous standards, and current content. The need for such a process to assist in the implementation of app evaluation across all medical fields is evident. Such a process has the potential to increase stakeholder engagement and catalyze interest and engagement with present-day app evaluation models.<h4>Objective</h4>This study aimed to develop and pilot test ...[more]

PMID: 32535548

Similar Datasets

Project description:BackgroundAtrial fibrillation (AF) is a highly prevalent heart rhythm condition that has significant associated morbidity and requires chronic treatment. Mobile health (mHealth) technologies have the potential to enhance multiple aspects of AF care, including education, monitoring of symptoms, and encouraging and tracking medication adherence. We have previously implemented and tested relational agents to improve outcomes in chronic disease and sought to develop a smartphone-based relational agent for improving patient-centered outcomes in AF.ObjectiveThe objective of this study was to pilot a smartphone-based relational agent as preparation for a randomized clinical trial, the Atrial Fibrillation Health Literacy Information Technology Trial (AF-LITT).MethodsWe developed the relational agent for use by a smartphone consistent with our prior approaches. We programmed the relational agent as a computer-animated agent to simulate a face-to-face conversation and to serve as a health counselor or coach specific to AF. Relational agent's dialogue content, informed by a review of literature, focused on patient-centered domains and qualitative interviews with patients with AF, encompassed AF education, common symptoms, adherence challenges, and patient activation. We established that the content was accessible to individuals with limited health or computer literacy. Relational agent content coordinated with use of the smartphone AliveCor Kardia heart rate and rhythm monitor. Participants (N=31) were recruited as a convenience cohort from ambulatory clinical sites and instructed to use the relational agent and Kardia for 30 days. We collected demographic, social, and clinical characteristics and conducted baseline and 30-day assessments of health-related quality of life (HRQoL) with the Atrial Fibrillation Effect on Quality of life (AFEQT) measure; self-reported medication adherence with the Morisky 8-item Medication Adherence Scale (MMAS-8); and patient activation with the Patient Activation Measure (PAM).ResultsParticipants (mean age 68 [SD 11]; 39% [12/31] women) used the relational agent for an average 17.8 (SD 10.0) days. The mean number of independent log-ins was 19.6 (SD 10.7), with a median of 20 times over 30 days. The mean number of Kardia uses was 26.5 (SD 5.9), and participants using Kardia were in AF for 14.3 (SD 11.0) days. AFEQT scores improved significantly from 64.5 (SD 22.9) at baseline to 76.3 (SD 19.4) units at 30 days (P<.01). We observed marginal but statistically significant improvement in self-reported medication adherence (baseline: 7.3 [SD 0.9], 30 days: 7.7 [SD 0.5]; P=.01). Assessments of acceptability identified that most of the participants found the relational agent useful, informative, and trustworthy.ConclusionsWe piloted a 30-day smartphone-based intervention that combined a relational agent with dedicated content for AF alongside Kardia heart rate and rhythm monitoring. Pilot participants had favorable improvements in HRQoL and self-reported medication adherence, as well as positive responses to the intervention. These data will guide a larger, enhanced randomized trial implementing the smartphone relational agent and the Kardia monitor system.

Project description:BACKGROUND: There is a lack of effective and affordable technologies to address health needs in the developing world. One way to address problems of innovation and affordability is to design global health technologies to follow agreed-upon standards. This Debate article argues that we can better develop standards for global health technologies if we learn lessons from other industries. DISCUSSION: The article's Background section begins by explaining why standards are needed in global health. For example, if global health technologies can be modularized into independent interfacing parts, these parts can then interact via well-defined standards in a "plug and play" fashion. This can avoid development of mutually incompatible solutions by different organizations, speed the pace of innovation, unlock health systems from single providers and approaches, and lower barriers to entry. The Background then gives a brief primer on standards and discusses incentives for health standards. The article's Discussion section begins with brief relevant cases of standards development from other industries, including electricity, container shipping, CD standards, Universal Serial Bus (USB), and the Internet. It then explores lessons from these and other industries that suggest how to develop standards for global health technologies. The remainder of the Discussion considers intellectual property and regulatory issues and standards-based global health business models, and ends with a checklist of considerations for health standards development leaders. (The associated Additional file discusses observations from standards development for cell phones and semiconductors, as well as challenges in the standards development process itself.) Throughout the article, point-of-care diagnostics are used as an illustrative example. An initiative is already underway to explore standardized diagnostics platforms. SUMMARY: This Debate article aims to convince the reader that standards can benefit global health technologies if we learn lessons from other industries. The article draws from historical examples and the authors' experiences to suggest principles, challenges, and opportunities in developing these standards. If implemented well, standardized platforms can lower barriers to entry, improve affordability, and create a vibrant ecosystem of innovative new global health technologies.

Project description:BackgroundThe use of apps for weight management has increased over recent years; however, there is a lack of evidence regarding the efficacy and safety of these apps. The EVALAPPS project will develop and validate an assessment instrument to specifically assess the safety and efficacy of weight management apps.ObjectiveThe aim of this study was to reach a consensus among stakeholders on a comprehensive set of criteria to guide development of the EVALAPPS assessment instrument. A modified Delphi process was used in order to verify the robustness of the criteria that had been identified through a literature review and to prioritize a set of the identified criteria.MethodsStakeholders (n=31) were invited to participate in a 2-round Delphi process with 114 initial criteria that had been identified from the literature. In round 1, participants rated criteria according to relevance on a scale from 0 ("I suggest this criterion is excluded") to 5 ("This criterion is extremely relevant"). A criterion was accepted if the median rating was 4 or higher and if the relative intraquartile range was equal to 0.67 or lower. In round 2, participants were asked about criteria that had been discarded in round 1. A prioritization strategy was used to identify crucial criteria according to (1) the importance attributed by participants (criteria with a mean rating of 4.00 or higher), (2) the level of consensus (criteria with a score of 4 or 5 by at least 80% of the participants).ResultsThe response rate was 83.9% (26/31) in round 1 and 90.3% (28/31) in round 2. A total of 107 out of 114 criteria (93.9%) were accepted by consensus-105 criteria in round 1 and 2 criteria in round 2. After prioritization, 53 criteria were deemed crucial. These related mainly to the dimensions of security and privacy (13/53, 24.5%) and usability (9/53, 17.0%), followed by activity data (5/53, 9.4%), clinical effectiveness (5/53, 9.4%), and reliability (5/53, 9.4%).ConclusionsResults confirmed the robustness of the criteria that were identified, with those relating to security and privacy being deemed most relevant by stakeholders. Additionally, a specific set of criteria based on health indicators (activity data, physical state data, and personal data) was also prioritized.

Project description:BackgroundPrivacy has always been a concern, especially in the health domain. The proliferation of mobile health (mHealth) apps has led to a large amount of sensitive data being generated. Some authors have performed privacy assessments of mHealth apps. They have evaluated diverse privacy components; however, different authors have used different criteria for their assessments.ObjectiveThis scoping review aims to understand how privacy is assessed for mHealth apps, focusing on the components, scales, criteria, and scoring methods used. A simple taxonomy to categorize the privacy assessments of mHealth apps based on component evaluation is also proposed.MethodsWe followed the methodology defined by Arksey and O'Malley to conduct a scoping review. Included studies were categorized based on the privacy component, which was assessed using the proposed taxonomy.ResultsThe database searches retrieved a total of 710 citations-24 of them met the defined selection criteria, and data were extracted from them. Even though the inclusion criteria considered articles published since 2009, all the studies that were ultimately included were published from 2014 onward. Although 12 papers out of 24 (50%) analyzed only privacy, 8 (33%) analyzed both privacy and security. Moreover, 4 papers (17%) analyzed full apps, with privacy being just part of the assessment. The evaluation criteria used by authors were heterogeneous and were based on their experience, the literature, and/or existing legal frameworks. Regarding the set of items used for the assessments, each article defined a different one. Items included app permissions, analysis of the destination, analysis of the content of communications, study of the privacy policy, use of remote storage, and existence of a password to access the app, among many others. Most of the included studies provided a scoring method that enables the comparison of privacy among apps.ConclusionsThe privacy assessment of mHealth apps is a complex task, as the criteria used by different authors for their evaluations are very heterogeneous. Although some studies about privacy assessment have been conducted, a very large set of items to evaluate privacy has been used up until now. In-app information and privacy policies are primarily utilized by the scientific community to extract privacy information from mHealth apps. The creation of a scale based on more objective criteria is a desirable step forward for privacy assessment in the future.

Project description:Background and aimsTraditional apprenticeship-based surgical training presents with challenges, especially in acute scenarios. Simulation provides the current standard of facilitating surgical training in a low-risk environment but is restricted by limited accessibility and high costs. Virtual reality (VR) offers immersive three-dimensional computer-generated training scenarios and can connect users from various locations. We aimed to compare the performance of junior doctors to manage an acute surgical scenario using VR and mannequin-based simulation. We hypothesised that VR would be as effective as mannequin-based simulation in performance outcomes.MethodsThis multicentre, randomised controlled pilot study was conducted with eighteen junior doctor volunteers (Foundation and Core Trainee Year 1). Ten were randomly allocated to VR and eight to mannequin-based simulation. Participants completed questionnaires and a 15-min pneumothorax scenario. Quantitative metrics included overall score, time-to-critical decisions, and academic buoyancy scores (ABS). Qualitative metrics included participants' likes and dislikes of their allocated simulation modality.ResultsVR participants scored significantly higher than mannequin-based simulation participants in overall scores (74.30% (SD ± 5.08%) vs. 59.75% (SD ± 10.14) (p = 0.04)), and technical skills aspects (77.20% (SD ± 8.01%) vs. 65.00% (SD ± 8.21%) (p = 0.01)). Mannequin-based simulation participants initiated critical decisions faster and demonstrated a trend towards a faster mean time-to-completion (p = 0.06). ABS scores increased for both study groups, though was only significant for VR participants (p ≤ 0.01). VR participants liked how VR fostered independent learning but disliked the formulaic content and impaired communication-learning compared to mannequin-based simulation.ConclusionBoth VR and mannequin-based simulation training are effective in training junior doctors in acute surgical scenarios but present different educational benefits. Future research should recruit a larger sample size for a full comparative randomised controlled trial.

Dataset Information

Technology Evaluation and Assessment Criteria for Health Apps (TEACH-Apps): Pilot Study.

Publications

Technology Evaluation and Assessment Criteria for Health Apps (TEACH-Apps): Pilot Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets