Dataset Information

Actionable health app evaluation: translating expert frameworks into objective metrics.

ABSTRACT: As use and availability of mobile health apps have increased, so too has the need for a thorough, accessible framework for app evaluation. The American Psychiatric Association's app evaluation model has emerged as a way to critically assess an app by considering accessibility, privacy and security, clinical foundation, engagement, and interoperability; however, there is no centralized database where users can view how various health apps perform when assessed via the APA model. In this perspective, we propose and outline our effort to translate the APA's model for the evaluation of health apps into a set of objective metrics that can be published online, making the framework actionable and accessible to a broad audience. The questions from the APA model were operationalized into 105 objective questions that are either binary or numeric. These questions serve as the foundation of an online database, where app evaluation consists of answering these 105 questions and can be crowdsourced. While the database has yet to be published and crowdsourced, initial internal testing demonstrated excellent interrater reliability. The database proposed here introduces a public and interactive approach to data collection that is guided by the APA model. The published product enables users to sort through the many mobile health apps and filter them according to individual preferences and priorities, making the ever-growing health app market more navigable.

SUBMITTER: Lagan S

PROVIDER: S-EPMC7393366 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Actionable health app evaluation: translating expert frameworks into objective metrics.

Lagan Sarah S Aquino Patrick P Emerson Margaret R MR Fortuna Karen K Walker Robert R Torous John J

NPJ digital medicine 20200730

As use and availability of mobile health apps have increased, so too has the need for a thorough, accessible framework for app evaluation. The American Psychiatric Association's app evaluation model has emerged as a way to critically assess an app by considering accessibility, privacy and security, clinical foundation, engagement, and interoperability; however, there is no centralized database where users can view how various health apps perform when assessed via the APA model. In this perspecti ...[more]

PMID: 32821855

Similar Datasets

Project description:ObjectivesDespite an estimated 300 000 mobile health apps on the market, there remains no consensus around helping patients and clinicians select safe and effective apps. In 2018, our team drew on existing evaluation frameworks to identify salient categories and create a new framework endorsed by the American Psychiatric Association (APA). We have since created a more expanded and operational framework Mhealth Index and Navigation Database (MIND) that aligns with the APA categories but includes objective and auditable questions (105). We sought to survey the existing space, conducting a review of all mobile health app evaluation frameworks published since 2018, and demonstrate the comprehensiveness of this new model by comparing it to existing and emerging frameworks.DesignWe conducted a scoping review of mobile health app evaluation frameworks.Data sourcesReferences were identified through searches of PubMed, EMBASE and PsychINFO with publication date between January 2018 and October 2020.Eligibility criteriaPapers were selected for inclusion if they meet the predetermined eligibility criteria-presenting an evaluation framework for mobile health apps with patient, clinician or end user-facing questions.Data extraction and synthesisTwo reviewers screened the literature separately and applied the inclusion criteria. The data extracted from the papers included: author and dates of publication, source affiliation, country of origin, name of framework, study design, description of framework, intended audience/user and framework scoring system. We then compiled a collection of more than 1701 questions across 79 frameworks. We compared and grouped these questions using the MIND framework as a reference. We sought to identify the most common domains of evaluation while assessing the comprehensiveness and flexibility-as well as any potential gaps-of MIND.ResultsNew app evaluation frameworks continue to emerge and expand. Since our 2019 review of the app evaluation framework space, more frameworks include questions around privacy (43) and clinical foundation (57), reflecting an increased focus on issues of app security and evidence base. The majority of mapped frameworks overlapped with at least half of the MIND categories. The results of this search have informed a database (apps.digitalpsych.org) that users can access today.ConclusionAs the number of app evaluation frameworks continues to rise, it is becoming difficult for users to select both an appropriate evaluation tool and to find an appropriate health app. This review provides a comparison of what different app evaluation frameworks are offering, where the field is converging and new priorities for improving clinical guidance.

Project description:BackgroundHealth Technology Reassessment (HTR) is a process that systematically assesses technologies that are currently used in the health care system. The process results in four outputs: increase use or decrease use, no change, or de-adoption of a technology. Implementation of these outputs remains a challenge. The Knowledge Translation (KT) field enables to transfer/translate knowledge into practice. KT could help with implementation of HTR outputs. This study sought to identify which characteristics of KT theories, models, and frameworks could be useful, specifically for decreased use or de-adoption of a technology.MethodsA qualitative descriptive approach was used to ascertain the perspectives of international KT and HTR experts on the characteristics of KT theories, models, and frameworks for decreased use or de-adoption of a technology. One-to-one semi-structured interviews were conducted from September to December 2019. Interviews were audio recorded and transcribed verbatim. Themes and sub-themes were deduced from the data through framework analysis using five distinctive steps: familiarization, identifying an analytic framework, indexing, charting, mapping and interpretation. Themes and sub-themes were also mapped to existing KT theories, models, and frameworks.ResultsThirteen experts from Canada, United States, United Kingdom, Australia, Germany, Spain, and Sweden participated in the study. Three themes emerged that illustrated the ideal traits: principles that were foundational for HTR, levers of change, and steps for knowledge to action. Principles included evidence-based, high usability, patient-centered, and ability to apply to the micro, meso, macro levels. Levers of change were characterized as positive, neutral, or negative influences for changing behaviour for HTR. Steps for knowledge to action included: build the case for HTR, adapt research knowledge, assess context, select interventions, and assess impact. Of the KT theories, models, and frameworks that were mapped, the Consolidated Framework for Implementation Research had most of the characteristics, except ability to apply to micro, meso, macro levels.ConclusionsCharacteristics that need to be considered within a KT theory, model, and framework for implementing HTR outputs have been identified. Consideration of these characteristics may guide users to select relevant KT theories, models, and frameworks to apply to HTR projects.

Dataset Information

Actionable health app evaluation: translating expert frameworks into objective metrics.

Publications

Actionable health app evaluation: translating expert frameworks into objective metrics.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure