Design and testing of a mobile health application rating tool.
Ontology highlight
ABSTRACT:
SUBMITTER: Levine DM
PROVIDER: S-EPMC7242452 | biostudies-literature | 2020
REPOSITORIES: biostudies-literature
Publications
Design and testing of a mobile health application rating tool.
Levine David M DM Co Zoe Z Newmark Lisa P LP Groisser Alissa R AR Holmgren A Jay AJ Haas Jennifer S JS Bates David W DW
NPJ digital medicine 20200521
Mobile health applications ("apps") have rapidly proliferated, yet their ability to improve outcomes for patients remains unclear. A validated tool that addresses apps' potentially important dimensions has not been available to patients and clinicians. The objective of this study was to develop and preliminarily assess a usable, valid, and open-source rating tool to objectively measure the risks and benefits of health apps. We accomplished this by using a Delphi process, where we constructed an ...[more]
Similar Datasets
Project description:BackgroundOver the last decade, mobile health applications (mHealth App) have evolved exponentially to assess and support our health and well-being.ObjectiveThis paper presents an Artificial Intelligence (AI)-enabled mHealth app rating tool, called ACCU3RATE, which takes multidimensional measures such as user star rating, user review and features declared by the developer to generate the rating of an app. However, currently, there is very little conceptual understanding on how user reviews affect app rating from a multi-dimensional perspective. This study applies AI-based text mining technique to develop more comprehensive understanding of user feedback based on several important factors, determining the mHealth app ratings.MethodBased on the literature, six variables were identified that influence the mHealth app rating scale. These factors are user star rating, user text review, user interface (UI) design, functionality, security and privacy, and clinical approval. Natural Language Toolkit package is used for interpreting text and to identify the App users' sentiment. Additional considerations were accessibility, protection and privacy, UI design for people living with physical disability. Moreover, the details of clinical approval, if exists, were taken from the developer's statement. Finally, we fused all the inputs using fuzzy logic to calculate the new app rating score.Results and conclusionsACCU3RATE concentrates on heart related Apps found in the play store and App gallery. The findings indicate the efficacy of the proposed method as opposed to the current device scale. This study has implications for both App developers and consumers who are using mHealth Apps to monitor and track their health. The performance evaluation shows that the proposed mHealth scale has shown excellent reliability as well as internal consistency of the scale, and high inter-rater reliability index. It has also been noticed that the fuzzy based rating scale, as in ACCU3RATE, matches more closely to the rating performed by experts.
| S-EPMC8675707 | biostudies-literature
Project description:BackgroundMobile health apps (MHA) have the potential to improve health care. The commercial MHA market is rapidly growing, but the content and quality of available MHA are unknown. Instruments for the assessment of the quality and content of MHA are highly needed. The Mobile Application Rating Scale (MARS) is one of the most widely used tools to evaluate the quality of MHA. Only few validation studies investigated its metric quality. No study has evaluated the construct validity and concurrent validity.ObjectiveThis study evaluates the construct validity, concurrent validity, reliability, and objectivity, of the MARS.MethodsData was pooled from 15 international app quality reviews to evaluate the metric properties of the MARS. The MARS measures app quality across four dimensions: engagement, functionality, aesthetics and information quality. Construct validity was evaluated by assessing related competing confirmatory models by confirmatory factor analysis (CFA). Non-centrality (RMSEA), incremental (CFI, TLI) and residual (SRMR) fit indices were used to evaluate the goodness of fit. As a measure of concurrent validity, the correlations to another quality assessment tool (ENLIGHT) were investigated. Reliability was determined using Omega. Objectivity was assessed by intra-class correlation.ResultsIn total, MARS ratings from 1,299 MHA covering 15 different health domains were included. Confirmatory factor analysis confirmed a bifactor model with a general factor and a factor for each dimension (RMSEA = 0.074, TLI = 0.922, CFI = 0.940, SRMR = 0.059). Reliability was good to excellent (Omega 0.79 to 0.93). Objectivity was high (ICC = 0.82). MARS correlated with ENLIGHT (ps<.05).ConclusionThe metric evaluation of the MARS demonstrated its suitability for the quality assessment. As such, the MARS could be used to make the quality of MHA transparent to health care stakeholders and patients. Future studies could extend the present findings by investigating the re-test reliability and predictive validity of the MARS.
| S-EPMC7605637 | biostudies-literature
Project description:BackgroundDespite the increasing number of mobile health applications, the validity of their content is understudied. The objective of this study was to rate the content of HIV/AIDS-related mobile applications and to determine the extent to which evidence-based medicine is being incorporated into their content using a new tool called the Evidence-based content rating tool of mobile health applications (EBCRT-mHealth).MethodsAll available HIV/AIDS-related applications in Iran from Cafe Bazaar and Google Play Store were evaluated. This study was first conducted in 2018, then after almost two years in 2021 was done again. In this study, researchers developed the EBCRT-mHealth tool to rate the content of applications based on the evidence-based medicine pyramid. Its reliability was calculated (α = 0.78), and five specialists confirmed its validity. Two reviewers independently reviewed all HIV/AIDS applications directly downloaded and installed from the Google Play Store and Cafe Bazaar.ResultsOut of 980 retrieved applications, in 2018, 85, and in 2021, 78 applications were included in the study. Only in 17 (28%) out of the 60 in 2018, and 25 (51%) in 2021 Google Play store applications the source of content information was mentioned. All Cafe Bazaar mobile applications mentioned the source of information. The mean rating of all application content in 2018 was 2.38 (SD = 0.74), and in 2021 was 2.90 (SD = 1.35) out of 5. The mean rating of the content of Cafe Bazaar applications in 2018 was 2.10 (SD = 0.49), and in 2021 was 1.94 (SD = 0.29). The mean content rating of Google Play store applications in 2018 was 2.50 (SD = 0.80) and in 2021 was 3.86 (SD = 1.18).ConclusionAfter two years, the rating of the content of HIV/AIDS-related applications available in Iran that existed in Cafe Bazaar decreased from "poor" to "inappropriate". Also, the content score of the Google Play Store applications increased from "poor" to "good". It is critical to ensure the credibility of the sources used in developing their content and removing applications with inappropriate and unreliable content from the App Stores. Also, mobile health application developers should use the highest quality information in their applications.
| S-EPMC8067376 | biostudies-literature
Project description:BACKGROUND:People living with chronic illnesses are an increasingly large group. Research indicates that care and self-management should not only focus on the illness and problem-oriented aspects of these individuals' lives but also support them in recognizing and leveraging their personal strengths in daily life. OBJECTIVE:This paper presents the design and developmental process of MyStrengths, a mobile health (mHealth) app designed to help its users (people with chronic conditions) both find and make use of their personal strengths in their daily lives. Through 4 consecutive phases, this paper presents participant- and researcher-driven activities, discussions regarding design, and development of both the MyStrengths app and its content. METHODS:During the 4 phases, we used a range of methods and activities, including (1) an idea-generating workshop aimed at creating ideas for strengths-supporting features with different stakeholders, including patients, caregivers, relatives, and designers (N=35); (2) research seminars with an international group of experts (N=6), in which the concept, theoretical background, and design ideas for the app were discussed; (3) a series of co-design workshops with people in the user group (N=22) aiming to create ideas for how to, in an engaging manner, design the app; and (4) in 4 developmental iterations, the app was evaluated by people in the user group (N=13). Content and strengths exercises were worked on and honed by the research team, the expert groups, and our internal editorial team during the entire developmental process. RESULTS:The first phase found a wide range of stakeholder requirements to, and ideas for, strengths-focused mHealth apps. From reviewing literature during the second phase, we found a dearth of research on personal strengths with respect to people living with chronic illnesses. Activities during the third phase creatively provided numerous ideas and suggestions for engaging and gameful ways to develop and design the MyStrengths app. The final phase saw the output from all the earlier phases come together. Through multiple increasingly complete iterations of user evaluations testing and developing, the final prototype of the MyStrengths app was created. CONCLUSIONS:Although research supports the use of strengths-focused mHealth tools to support people living with chronic illnesses, there is little guidance as to how these tools and their content should be designed. Through all activities, we found great support among participating users for strengths-focused apps, and we can consider such apps to be both appropriate and valuable. This paper illustrates how combining a range of user-, researcher-, literature-, and designer-based methods can contribute to creating mHealth tools to support people with chronic illnesses to find and use more of their own personal strengths.
| S-EPMC7414410 | biostudies-literature
Project description:In several nations, caries in pre-school children remain a significant oral health issue. In an outbreak period such as the Coronavirus disease 2019 (COVID-19), remote contact and education aimed at the prevention of oral diseases and the preservation of children's oral health are more relevant than ever. Currently, the amount of published applications is far higher than the published scientific studies while the problems of usability remains vulnerable. The goal of this paper was to comprehensively document the phase of development and usability testing of a mobile application for diet and oral health, namely Gigiku Sihat, which was primarily intended to be used by parents and guardians of pre-school children. The mobile application was developed using the System Development Life Cycle principle. Apart from searching for the available oral health application on Android platform, the initial requirement gathering process consisted of situational analysis, concept generation, content development, and features and functional requirement determination. The mobile application design and implementation evolved at each phase before being finalised. Gigiku Sihat was successfully developed in the Bahasa Malaysia. Finalised Gigiku Sihat was installed on mobile devices to determine the usability using translated and validated System Usability Scale questionnaire namely Skala Kebolehgunaan Aplikasi Mudah Alih (SKAMA). The mean score usability with score of 68 and above was deemed to have good usability. This study found that Gigiku Sihat mean (SD) usability score was 77.0 (14.18). The results were promising as they showed that Gigiku Sihat had a good usability. Thus, the development of this mobile application focusing on diet and oral health served as a new source of oral health education and provided a necessary foundation in developing future improved mobile application development for parents in the prevention of early childhood caries.
| S-EPMC8425540 | biostudies-literature
Project description:Although patients express an interest in using mobile health (mHealth) interventions to manage their health and chronic conditions, many current mHealth interventions are difficult to use. Usability testing is critical for the success of novel mHealth interventions. Researchers recognize the utility of using qualitative and quantitative approaches for usability testing, but many mHealth researchers lack the awareness of integration approaches from advances in mixed methods research that can add value to mHealth technology. As efficient usability testing proceeds iteratively, we introduce a novel mixed methods design developed specifically for mHealth researchers. The iterative convergent mixed methods design involves simultaneous qualitative and quantitative data collection and analysis that continues cyclically through multiple rounds of mixed methods data collection and analysis until the mHealth technology under evaluation is found to work to the satisfaction of the researcher. In cyclical iterations, early development is more qualitatively driven but progressively becomes more quantitatively driven. Using this design, mHealth researchers can leverage mixed methods integration procedures in the research question, data collection, data analysis, interpretation, and dissemination dimensions. This study demonstrates how the iterative convergent mixed methods design provides a novel framework for generating unique insights into multifaceted phenomena impacting mHealth usability. Understanding these practices can help developers and researchers leverage the strengths of an integrated mixed methods design.
| S-EPMC6658163 | biostudies-literature
Project description:BACKGROUND:Chronic rheumatic diseases need long-term treatment and professional supervision. Mobile apps promise to improve the lives of patients and physicians. In routine practice, however, rheumatology apps are largely unknown and little is known about their quality and safety. OBJECTIVE:The aim of this study was to provide an overview of mobile rheumatology apps currently available in German app stores, evaluate app quality using the Mobile Application Rating Scale (MARS), and compile brief, ready-to-use descriptions for patients and rheumatologists. METHODS:The German App Store and Google Play store were systematically searched to identify German rheumatology mobile apps for patient and physician use. MARS was used to independently assess app quality by 8 physicians, 4 using Android and 4 using iOS smartphones. Apps were randomly assigned so that 4 apps were rated by all raters and the remaining apps were rated by two Android and two iOS users. Furthermore, brief app descriptions including app developers, app categories, and features were compiled to inform potential users and developers. RESULTS:In total, 128 and 63 apps were identified in the German Google Play and App Store, respectively. After removing duplicates and only including apps that were available in both stores, 28 apps remained. Sixteen apps met the inclusion criteria, which were (1) German language, (2) availability in both app stores, (3) targeting patients or physicians as users, and (4) clearly including rheumatology or rheumatic diseases as subject matter. Exclusion criteria were (1) congress apps and (2) company apps with advertisements. Nine apps addressed patients and 7 apps addressed physicians. No clinical studies to support the effectiveness and safety of apps could be found. Pharmaceutical companies were the main developers of two apps. Rheuma Auszeit was the only app mainly developed by a patient organization. This app had the highest overall MARS score (4.19/5). Three out of 9 patient apps featured validated questionnaires. The median overall MARS score was 3.85/5, ranging from 2.81/5 to 4.19/5. One patient-targeted and one physician-targeted app had MARS scores >4/5. No significant rater gender or platform (iOS/Android) differences could be observed. The overall correlation between app store ratings and MARS scores was low and inconsistent between platforms. CONCLUSIONS:To our knowledge, this is the first study that systematically identified and evaluated mobile apps in rheumatology for patients and physicians available in German app stores. We found a lack of supporting clinical studies, use of validated questionnaires, and involvement of academic developers. Overall app quality was heterogeneous. To create high-quality apps, closer cooperation led by patients and physicians is vital.
| S-EPMC6699116 | biostudies-literature
Project description:BackgroundTo realize the potential for mobile learning in clinical skills acquisition, medical students and their teachers should be able to evaluate the value of an app to support student learning of clinical skills. To our knowledge, there is currently no rubric for evaluation of quality or value that is specific for apps to support medical student learning. Such a rubric might assist students to be more confident in using apps to support their learning.ObjectiveThe objective of this study was to develop an instrument that can be used by health professional educators to rate the value of a mobile app to support health professional student learning.MethodsUsing the literature, we developed a list of potential criteria for the evaluation of educational app value, which were then refined with a student group using a modified nominal group technique. The refined list was organized into themes, and the initial rubric, Mobile App Rubric for Learning (MARuL, version 1), was developed. iOS and Android app stores were searched for clinical skills apps that met our inclusion criteria. After the 2 reviewers were trained and the item descriptions were refined (version 2), a random sample of 10 included apps, 5 for each mobile operating system, was reviewed. Interitem and interrater analyses and discussions with the reviewers resulted in refinement of MARuL to version 3. The reviewers completed a review of 41 clinical skills mobile apps, and a second round of interitem and interrater reliability testing was performed, leading to version 4 of the MARuL.ResultsStudents identified 28 items (from an initial set of 144 possible items) during the nominal group phase, and these were then grouped into 4 themes: teaching and learning, user centered, professional, and usability. Testing and refinement with reviewers reduced the list to 26 items. Internal consistency for MARuL was excellent (α=.96), and the interrater reliability as measured by the intraclass correlation coefficient (ICC) was good (ICC=0.66).ConclusionsMARuL offers a fast and user-friendly method for teachers to select valuable apps to enhance student learning.
| S-EPMC7428912 | biostudies-literature
Project description:Background:Current continuous positive airway pressure (CPAP) devices can be monitored remotely; however, in-person visits are kept for clinical follow-up in order to promote CPAP use and resolve potential side-effects. Mobile health is a promising way to provide remote and easy clinical control for CPAP follow-up and support. We aimed to evaluate the feasibility and acceptance by obstructive sleep apnoea (OSA) patients and healthcare professionals of a newly designed mobile app (Appnea-Q) to promote clinical control through a self-monitoring tool for patients with CPAP supervised by sleep professionals. Methods:Appnea-Q incorporates a simple follow-up questionnaire with automated responses, together with frequent problems and lifestyle recommendations sections. Feasibility, acceptance and usefulness were assessed. First, an internal validation was performed during outpatient CPAP follow-up visits with sleep professionals from various sleep units. Second, an external validation was performed in a subgroup of 15 patients at home. Results:Most patients (n=75) considered the app useful and were willing to use it and recommend it (72-88%). Up to 64.87% agreed on its capacity to reduce hospital visits. Appnea-Q was rated as acceptable (79.37±19.29) by the system usability score. Sleep professionals (n=30) concurred on its usefulness for OSA patient follow-up, particularly during the first month of CPAP therapy. The external validation showed its feasibility among 11 out of 15 patients and their data were received accordingly on the professionals' web platform. Conclusions:According to our validation process, and the viewpoints of the patients and professionals, our new mobile app is a feasible and well-received tool for personal OSA management. Future clinical trials should substantiate its performance and cost-effectiveness in the clinical arena.
| S-EPMC7008136 | biostudies-literature
Project description:BACKGROUND:The continuous development of mobile apps has led to many health care professionals using them in clinical settings; however, little research is available to guide occupational therapists (OTs) in choosing quality apps for use in their respective clinical settings. OBJECTIVE:The purpose of this study was to use the user version of the Mobile Application Rating Scale (uMARS) to evaluate the quality of the most frequently noted mobile health (mHealth) apps used by OTs and to demonstrate the utility of the uMARS to assess the quality of mHealth apps. METHODS:A previous study surveying OTs' use of apps in therapy compiled a list of apps frequently noted. A total of 25 of these apps were evaluated individually by 2 trained researchers using the uMARS, a simple, multidimensional analysis tool that can be reliably used to evaluate the quality of mHealth apps. RESULTS:The top 10 apps had a total quality score of 4.3, or higher, out of 5 based on the mean scores of engagement, functionality, and aesthetics. Apps scored highest in functionality and lowest in engagement. Apps noted most frequently were not always high-quality apps; apps noted least frequently were not always low-quality apps. CONCLUSIONS:Determining the effectiveness of using apps in clinical settings must be built upon a foundation of the implementation of high-quality apps. Mobile apps should not be incorporated into clinical settings solely based on frequency of use. The uMARS should be considered as a useful tool for OTs, and other professionals, to determine app quality.
| S-EPMC6526689 | biostudies-literature