Dataset Information

Evaluation Criteria for Weight Management Apps: Validation Using a Modified Delphi Process.

ABSTRACT: BACKGROUND:The use of apps for weight management has increased over recent years; however, there is a lack of evidence regarding the efficacy and safety of these apps. The EVALAPPS project will develop and validate an assessment instrument to specifically assess the safety and efficacy of weight management apps. OBJECTIVE:The aim of this study was to reach a consensus among stakeholders on a comprehensive set of criteria to guide development of the EVALAPPS assessment instrument. A modified Delphi process was used in order to verify the robustness of the criteria that had been identified through a literature review and to prioritize a set of the identified criteria. METHODS:Stakeholders (n=31) were invited to participate in a 2-round Delphi process with 114 initial criteria that had been identified from the literature. In round 1, participants rated criteria according to relevance on a scale from 0 ("I suggest this criterion is excluded") to 5 ("This criterion is extremely relevant"). A criterion was accepted if the median rating was 4 or higher and if the relative intraquartile range was equal to 0.67 or lower. In round 2, participants were asked about criteria that had been discarded in round 1. A prioritization strategy was used to identify crucial criteria according to (1) the importance attributed by participants (criteria with a mean rating of 4.00 or higher), (2) the level of consensus (criteria with a score of 4 or 5 by at least 80% of the participants). RESULTS:The response rate was 83.9% (26/31) in round 1 and 90.3% (28/31) in round 2. A total of 107 out of 114 criteria (93.9%) were accepted by consensus-105 criteria in round 1 and 2 criteria in round 2. After prioritization, 53 criteria were deemed crucial. These related mainly to the dimensions of security and privacy (13/53, 24.5%) and usability (9/53, 17.0%), followed by activity data (5/53, 9.4%), clinical effectiveness (5/53, 9.4%), and reliability (5/53, 9.4%). CONCLUSIONS:Results confirmed the robustness of the criteria that were identified, with those relating to security and privacy being deemed most relevant by stakeholders. Additionally, a specific set of criteria based on health indicators (activity data, physical state data, and personal data) was also prioritized.

SUBMITTER: Robles N

PROVIDER: S-EPMC7407251 | biostudies-literature | 2020 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluation Criteria for Weight Management Apps: Validation Using a Modified Delphi Process.

Robles Noemí N Puigdomènech Puig Elisa E Gómez-Calderón Corpus C Saigí-Rubió Francesc F Cuatrecasas Cambra Guillem G Zamora Alberto A Moharra Montse M Paluzié Guillermo G Balfegó Mariona M Carrion Carme C

JMIR mHealth and uHealth 20200722 7

<h4>Background</h4>The use of apps for weight management has increased over recent years; however, there is a lack of evidence regarding the efficacy and safety of these apps. The EVALAPPS project will develop and validate an assessment instrument to specifically assess the safety and efficacy of weight management apps.<h4>Objective</h4>The aim of this study was to reach a consensus among stakeholders on a comprehensive set of criteria to guide development of the EVALAPPS assessment instrument. ...[more]

PMID: 32706689

Similar Datasets

Project description:BackgroundThe number of e-mental health apps is increasing rapidly. Studies have shown that the use of some apps is beneficial, whereas others are ineffective or do not meet users' privacy expectations. Individuals and organizations that curate, recommend, host, use, or pay for apps have an interest in categorizing apps according to the consensus criteria of usability and effectiveness. Others have previously published recommendations for assessing health-related apps; however, the extent to which these recommendations can be generalized across different population groups (eg, culture, gender, and language) remains unclear. This study describes an attempt by Canadian stakeholders to develop an e-mental health assessment framework that responds to the unique needs of people living in Canada in an evidence-based manner.ObjectiveThe objective of our study was to achieve consensus from a broad group of Canadian stakeholders on guiding principles and criteria for a framework to assess e-mental health apps in Canada.MethodsWe developed an initial set of guiding principles and criteria from a rapid review and environmental scan of pre-existing app assessment frameworks. The initial list was refined through a two-round modified Delphi process. Participants (N=25) included app developers and users, health care providers, mental health advocates, people with lived experience of a mental health problem or mental illness, policy makers, and researchers. Consensus on each guideline or criterion was defined a priori as at least 70% agreement. The first round of voting was conducted electronically. Prior to Round 2 voting, in-person presentations from experts and a persona empathy mapping process were used to explore the perspectives of diverse stakeholders.ResultsOf all respondents, 68% (17/25) in Round 1 and 100% (13/13) in Round 2 agreed that a framework for evaluating health apps is needed to help Canadian consumers identify high-quality apps. Consensus was reached on 9 guiding principles: evidence based, gender responsive, culturally appropriate, user centered, risk based, internationally aligned, enabling innovation, transparent and fair, and based on ethical norms. In addition, 15 informative and evaluative criteria were defined to assess the effectiveness, functionality, clinical applicability, interoperability, usability, transparency regarding security and privacy, security or privacy standards, supported platforms, targeted users, developers' transparency, funding transparency, price, user desirability, user inclusion, and meaningful inclusion of a diverse range of communities.ConclusionsCanadian mental health stakeholders reached the consensus on a framework of 9 guiding principles and 15 criteria important in assessing e-mental health apps. What differentiates the Canadian framework from other scales is explicit attention to user inclusion at all stages of the development, gender responsiveness, and cultural appropriateness. Furthermore, an empathy mapping process markedly influenced the development of the framework. This framework may be used to inform future mental health policies and programs.

Project description:BackgroundWeight management apps may provide support and management options for individuals with overweight and obesity. Research on the quality of weight management mHealth apps among the Saudi population is insufficient despite frequent use.ObjectiveThe aims of this study were to explore user perceptions of weight management apps, explore reasons for starting and stopping app use, appraise the quality of weight management apps available in the App Store, and compare the features currently available within the app market and those that are most desirable to weight management app users.MethodsA web-based survey consisted of 31 open and closed questions about sociodemographic information, general health questions, app use, app user perceptions, and discontinuation of app use. The quality of the weight management apps available on the App Store was assessed using the Mobile App Rating Scale and evidence-based strategies. We also used six sigma evaluations to ensure that the quality measured by the tools consistently meets customer expectations.ResultsData from the survey were analyzed. Of the respondents, 30.17% (324/1074) had used a weight management app, 18.16% (195/1074) used the apps and stopped, and 51.68% (555/1074) had never used a weight management app. Of apps mentioned, 23 met the inclusion criteria. The overall average Mobile App Rating Scale quality of apps was acceptable; 30% (7/23) received a quality mean score of 4 or higher (out of 5), and 30% (7/23) did not meet the acceptability score of 3 or higher. Evidence-based strategy results showed that feedback was not observed in any of the apps, and motivation strategy was observed in only 1 app. The sigma results of evidence-based strategies reflect that most of the apps fail to pass the mean.ConclusionsApp users desired a feature that allows them to communicate with a specialist, which is a missing in the available free apps. Despite the large number and accessibility of weight management apps, the quality and features of most are variable. It can be concluded from six sigma results that passing the mean does not ensure that the quality is consistently distributed through all app quality properties and Mobile App Rating Scale and evidence-based strategies do not give developers an indication of the acceptance of their apps by mobile users. This finding stresses the importance of reevaluating the passing criterion, which is ≥50% for designing an effective app.

Project description:Importance:Current practice to diagnose idiopathic orbital inflammation (IOI) is inconsistent, leading to frequent misdiagnosis of other orbital entities, including cancer. By specifying criteria, diagnosis of orbital inflammation will be improved. Objective:To define a set of criteria specific for the diagnosis of IOI. Design, Setting, and Participants:A 3-round modified Delphi process with an expert panel was conducted from June 8, 2015, to January 25, 2016. Fifty-three orbital scientist experts, identified through membership in the Orbital Society, were invited to participate in on online survey and they scored, using 5-point Likert scales, items that are eligible as diagnostic criteria from the literature and from personal experience. The items were clustered around the anatomic subtypes of IOI: idiopathic dacryoadenitis and idiopathic orbital fat inflammation (2 nonmyositic IOIs), and idiopathic orbital myositis (myositic IOI). Items with dissensus were rescored in the second round, and all items with consensus (median, ?4; interquartile range, ?1) were ranked by importance in the third round. Main Outcomes and Measures:Consensus on items to be included in the criteria. Results:Of the 53 experts invited to participate, a multinational panel of 35 (66%) individuals with a mean (SD) years of experience of 31 (11) years were included. Consensus was achieved on 7 of 14 clinical and radiologic items and 5 of 7 pathologic items related to diagnosis of nonmyositic IOI, and 11 of 14 clinical and radiologic items and 1 of 5 pathologic items for myositic IOI. There was agreement among panelists to focus on surgical tissue biopsy results in the diagnosis of nonmyositic IOI and on a trial with systemic corticosteroids in myositic IOI. Panelists agreed that a maximum number of 30 IgG4-positive plasma cells per high-power field in the orbital tissue is compatible with the diagnosis of IOI. Conclusions and Relevance:An international panel of experts endorsed consensus diagnostic criteria of IOI. These criteria define a level of exclusion suggested for diagnosis and include tissue biopsy for lesions not confined to the extraocular muscles. This consensus is a step toward developing guidelines for the management of IOI, which needs to be followed by validation studies of the criteria.

Project description:BackgroundThere is growing evidence that apps and digital interventions have a positive impact on diabetes self-management. Standard self-management for patients with diabetes could therefore be supplemented by apps and digital interventions to increase patients' skills. Several initiatives, models, and frameworks suggest how health apps and digital interventions could be evaluated, but there are few standards for this. And although there are many methods for evaluating apps and digital interventions, a more specific approach might be needed for assessing digital diabetes self-management interventions.ObjectiveThis review aims to identify which methods and criteria are used to evaluate apps and digital interventions for diabetes self-management, and to describe how patients were involved in these evaluations.MethodsWe searched CINAHL, EMBASE, MEDLINE, and Web of Science for articles published from 2015 that referred to the evaluation of apps and digital interventions for diabetes self-management and involved patients in the evaluation. We then conducted a narrative qualitative synthesis of the findings, structured around the included studies' quality, methods of evaluation, and evaluation criteria.ResultsOf 1681 articles identified, 31 fulfilled the inclusion criteria. A total of 7 articles were considered of high confidence in the evidence. Apps were the most commonly used platform for diabetes self-management (18/31, 58%), and type 2 diabetes (T2D) was the targeted health condition most studies focused on (12/31, 38%). Questionnaires, interviews, and user-group meetings were the most common methods of evaluation. Furthermore, the most evaluated criteria for apps and digital diabetes self-management interventions were cognitive impact, clinical impact, and usability. Feasibility and security and privacy were not evaluated by studies considered of high confidence in the evidence.ConclusionsThere were few studies with high confidence in the evidence that involved patients in the evaluation of apps and digital interventions for diabetes self-management. Additional evaluation criteria, such as sustainability and interoperability, should be focused on more in future studies to provide a better understanding of the effects and potential of apps and digital interventions for diabetes self-management.

Project description:BackgroundDespite the emergence of app evaluation tools, there remains no well-defined process receptive to diverse local needs, rigorous standards, and current content. The need for such a process to assist in the implementation of app evaluation across all medical fields is evident. Such a process has the potential to increase stakeholder engagement and catalyze interest and engagement with present-day app evaluation models.ObjectiveThis study aimed to develop and pilot test the Technology Evaluation and Assessment Criteria for Health apps (TEACH-apps).MethodsTailoring a well-known implementation framework, Replicating Effective Programs, we present a new process to approach the challenges faced in implementing app evaluation tools today. As a culmination of our experience implementing this process and feedback from stakeholders, we present the four-part process to aid the implementation of mobile health technology. This paper outlines the theory, evidence, and initial versions of the process.ResultsThe TEACH-apps process is designed to be broadly usable and widely applicable across all fields of health. The process comprises four parts: (1) preconditions (eg, gathering apps and considering local needs), (2) preimplementation (eg, customizing criteria and offering digital skills training), (3) implementation (eg, evaluating apps and creating educational handouts), and (4) maintenance and evolution (eg, repeating the process every 90 days and updating content). TEACH-apps has been tested internally at our hospital, and there is growing interest in partnering health care facilities to test the system at their sites.ConclusionsThis implementation framework introduces a process that equips stakeholders, clinicians, and users with the foundational tools to make informed decisions around app use and increase app evaluation engagement. The application of this process may lead to the selection of more culturally appropriate and clinically relevant tools in health care.

Project description:IntroductionThe ongoing opioid epidemic has necessitated increasing prescriptions of buprenorphine, which is an evidence-based treatment for opioid use disorder, and also shown to reduce harms associated with unsafe opioid administration. A systematic review of perioperative management strategies for patients taking buprenorphine concluded that there was little guidance for managing buprenorphine perioperatively. The aim of this project is to develop consensus guidelines on the optimal perioperative management strategies for this group of patients. In this paper, we present the design for a modified Delphi technique that will be used to gain consensus among patients and multidisciplinary experts in addiction, pain, community and perioperative medicine.Methods and analysisA national panel of experts identified by perioperative, pain and/or addiction systematic review authorship established an international profile in perioperative, pain and/or addiction research, community clinical excellence and by peer referral. A steering group will develop the first round with a list of indications to be rated by the panel of national experts, patients and allied healthcare professionals. In round 1, the expert panel will rate the appropriateness of each individual item and provide additional suggestions for revisions, additions or deletions. The definition of consensus will be set a priori. Consensus will be gauged for both appropriateness and inappropriateness of treatment strategies. Where an agreement is not reached and items are suggested for addition/deletion/modification, round 2 will take place over teleconference in order to obtain consensus.Ethics and disseminationInstitutional research ethics board provided a waiver for this modified Delphi protocol. We plan on developing a national guideline for the management of patients taking buprenorphine in the perioperative period that will be generalisable across three sets of preoperative diagnoses including opioid use disorder and/or co-occurring pain disorders. The findings will be published in peer-reviewed publications and conference presentations.

Dataset Information

Evaluation Criteria for Weight Management Apps: Validation Using a Modified Delphi Process.

Publications

Evaluation Criteria for Weight Management Apps: Validation Using a Modified Delphi Process.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets