The Mobile App Development and Assessment Guide (MAG): Delphi-Based Validity Study.
Ontology highlight
ABSTRACT:
SUBMITTER: Llorens-Vernet P
PROVIDER: S-EPMC7428935 | biostudies-literature | 2020 Jul
REPOSITORIES: biostudies-literature
Publications
The Mobile App Development and Assessment Guide (MAG): Delphi-Based Validity Study.
Llorens-Vernet Pere P Miró Jordi J
JMIR mHealth and uHealth 20200731 7
<h4>Background</h4>In recent years, there has been an exponential growth of mobile health (mHealth)-related apps. This has occurred in a somewhat unsupervised manner. Therefore, having a set of criteria that could be used by all stakeholders to guide the development process and the assessment of the quality of the apps is of most importance.<h4>Objective</h4>The aim of this paper is to study the validity of the Mobile App Development and Assessment Guide (MAG), a guide recently created to help s ...[more]
Similar Datasets
Project description:BACKGROUND:Gait disorders are common among older adults. With an increase in the use of technology among older adults, a mobile phone app provides a solution for older adults to self-monitor their gait quality in daily life. OBJECTIVE:This study aimed to develop a gait-monitoring mobile phone app (Pocket Gait) and evaluate its acceptability and usability among potential older users. METHODS:The app was developed to allow older adults to track their gait quality, including step frequency, acceleration root mean square (RMS), step regularity, step symmetry, and step variability. We recruited a total of 148 community-dwelling older adults aged 60 years and older from two cities in China: Beijing and Chongqing. They walked in three ways (single task, dual task, and fast walking) using a smartphone with the gait-monitoring app installed and completed an acceptability and usability survey after the walk test. User acceptability was measured by a questionnaire including four quantitative measures: perceived ease of use, perceived usefulness, ease of learning, and intention to use. Usability was measured using the System Usability Scale (SUS). Interviews were conducted with participants to collect open-ended feedback questions. RESULTS:Task type had a significant effect on all gait parameters, namely, step frequency, RMS, step variability, step regularity, and step symmetry (all P values <.001). Age had a significant effect on step frequency (P=.01), and region had a significant effect on step regularity (P=.04). The acceptability of the gait-monitoring app was positive among older adults. Participants identified the usability of the system with an overall score of 59.7 (SD 10.7) out of 100. Older adults from Beijing scored significantly higher SUS compared with older adults from Chongqing (P<.001). The age of older adults was significantly associated with their SUS score (P=.048). Older adults identified improvements such as a larger font size, inclusion of reference values for gait parameters, and inclusion of heart rate and blood pressure monitoring. CONCLUSIONS:This mobile phone app is a health management tool for older adults to self-manage their gait quality and prevent adverse outcomes. In the future, it will be important to take factors such as age and region into consideration while designing a mobile phone-based gait assessment app. The feedback of the participants would help to design more elderly-friendly products. TRIAL REGISTRATION:
| S-EPMC7284482 | biostudies-literature
Project description:BackgroundLongitudinal studies of many health behaviors often rely on infrequent self-report assessments. The measurement of psychoactive substance use among youth is expected to improve with more frequent mobile assessments, which can reduce recall bias. Researchers have used mobile devices for longitudinal research, but studies that last years and assess youth continuously at a fine-grained, temporal level (eg, weekly) are rare. A tailored mobile app (mNCANDA [mobile National Consortium on Alcohol and Neurodevelopment in Adolescence]) and a brief assessment protocol were designed specifically to provide a feasible platform to elicit responses to health behavior assessments in longitudinal studies, including NCANDA (National Consortium on Alcohol and Neurodevelopment in Adolescence).ObjectiveThis study aimed to determine whether an acceptable mobile app system could provide repeatable and valid assessment of youth's health behaviors in different developmental stages over extended follow-up.MethodsParticipants were recruited (n=534; aged 17-28 years) from a larger longitudinal study of neurodevelopment. Participants used mNCANDA to register reports of their behaviors for up to 18 months. Response rates as a function of time measured using mNCANDA and participant age were modeled using generalized estimating equations to evaluate response rate stability and age effects. Substance use reports captured using mNCANDA were compared with responses from standardized interviews to assess concurrent validity. Reactivity was assessed by evaluating patterns of change in substance use after participants initiated weekly reports via mNCANDA. Quantitative feedback about the app was obtained from the participants. Qualitative interviews were conducted with a subset of participants who used the app for at least one month to obtain feedback on user experience, user-derived explanations of some quantitative results, and suggestions for system improvements.ResultsThe mNCANDA protocol adherence was high (mean response rate 82%, SD 27%) and stable over time across all age groups. The median time to complete each assessment was 51 s (mean response time 1.14, SD 1.03 min). Comparisons between mNCANDA and interview self-reports on recent (previous 30 days) alcohol and cannabis use days demonstrate close agreement (eg, within 1 day of reported use) for most observations. Models used to identify reactivity failed to detect changes in substance use patterns subsequent to enrolling in mNCANDA app assessments (P>.39). Most participants (64/76, 84%) across the age range reported finding the mNCANDA system acceptable. Participants provided recommendations for improving the system (eg, tailoring signaling times).ConclusionsmNCANDA provides a feasible, multi-year, continuous, fine-grained (eg, weekly) assessment of health behaviors designed to minimize respondent burden and provides acceptable regimes for long-term self-reporting of health behaviors. Fine-grained characterization of variability in behaviors over relatively long periods (eg, up to 18 months) may, through the use of mNCANDA, improve our understanding of the relationship between substance use exposure and neurocognitive development.
| S-EPMC7904399 | biostudies-literature
Project description:BackgroundThe mobile health (mHealth) app trustworthiness (mHAT) checklist was created to identify end users' opinions on the characteristics of trustworthy mHealth apps and to communicate this information to app developers. To ensure that the checklist is suited for all relevant stakeholders, it is necessary to validate its contents.ObjectiveThe purpose of this study was to assess the feasibility of the mHAT checklist by modifying its contents according to ratings and suggestions from stakeholders familiar with the process of developing, managing, or curating mHealth apps.MethodsA 44-item online survey was administered to relevant stakeholders. The survey was largely comprised of the mHAT checklist items, which respondents rated on a 5-point Likert scale, ranging from completely disagree (1) to completely agree (5).ResultsIn total, seven professional backgrounds were represented in the survey: administrators (n=6), health professionals (n=7), information technology personnel (n=6), managers (n=2), marketing personnel (n=3), researchers (n=5), and user experience researchers (n=8). Aside from one checklist item-"the app can inform end users about errors in measurements"-the combined positive ratings (ie, completely agree and agree) of the checklist items overwhelmingly exceeded the combined negative ratings (ie, completely disagree and disagree). Meanwhile, two additional items were included in the checklist: (1) business or funding model of the app and (2) details on app uninstallation statistics.ConclusionsOur results indicate that the mHAT checklist is a valuable resource for a broad range of stakeholders to develop trustworthy mHealth apps. Future studies should examine if the checklist works best for certain mHealth apps or in specific settings.
| S-EPMC7404005 | biostudies-literature
Project description:BackgroundSelf-regulation theory suggests people learn to influence their own behavior through self-monitoring, goal-setting, feedback, self-reward, and self-instruction, all of which smartphones are now capable of facilitating. Several mobile apps exist to manage asthma; however, little evidence exists about whether these apps employ user-centered design processes that adhere to government usability guidelines for mobile apps.ObjectiveBuilding upon a previous study that documented adolescent preferences for an asthma self-management app, we employed a user-centered approach to assess the usability of a high-fidelity wireframe for an asthma self-management app intended for use by adolescents with persistent asthma.MethodsIndividual interviews were conducted with adolescents (ages 11-18 years) with persistent asthma who owned a smartphone (N=8). Adolescents were asked to evaluate a PDF app wireframe consisting of 76 screen shots displaying app features, including log in and home screen, profile setup, settings and info, self-management features, and graphical displays for charting asthma control and medication. Preferences, comments, and suggestions for each set of screen shots were assessed using the audio-recorded interviews. Two coders reached consensus on adolescent evaluations of the following aspects of app features: (1) usability, (2) behavioral intentions to use, (3) confusing aspects, and (4) suggestions for improvement.ResultsThe app wireframe was generally well received, and several suggestions for improvement were recorded. Suggestions included increased customization of charts and notifications, reminders, and alerts. Participants preferred longitudinal data about asthma control and medication use to be displayed using line graphs. All participants reported that they would find an asthma management app like the one depicted in the wireframe useful for managing their asthma.ConclusionsEarly stage usability tests guided by government usability guidelines (usability.gov) revealed areas for improvement for an asthma self-management app for adolescents. Addressing these areas will be critical to developing an engaging and effective asthma self-management app that is capable of improving adolescent asthma outcomes.
| S-EPMC5311420 | biostudies-literature
Project description:BACKGROUND:Consumer food environments are increasingly being recognized as influential determinants of food purchasing and subsequent intake and health. We developed a tool to enable efficient, but relatively comprehensive, appraisal of the in-store food environment. The Store Scout mobile app facilitates the evaluation of product (availability and range), placement (visibility, accessibility, proximity to high-traffic areas, and location relative to other products), price (price promotion), and promotion (displays and advertising) across 7 categories of food products, with appraisal given immediately as scores (0-100, where a higher score is more in line with best practice). Primary end users are public health nutritionists and nutritionists employed by store organizations; however, store managers and staff are also potential end users. OBJECTIVE:This study aims to evaluate the reliability (interrater reliability and internal consistency), utility (distribution of scores), and construct validity (score by store type) of measurements using the Store Scout mobile app. METHODS:The Store Scout mobile app was used independently by 2 surveyors to evaluate the store environment in 54 stores: 34 metropolitan stores (9 small and 11 large supermarkets, 10 convenience stores, and 4 petrol stations) in Brisbane, Australia, and 20 remote stores (19 small supermarkets and 1 petrol station) in Indigenous Australian communities in Northern Australia. The agreement between surveyors in the overall and category scores was evaluated using intraclass correlation coefficients (ICCs). Interrater reliability of measurement items was assessed using percentage agreement and the Gwet agreement coefficient (AC). Internal consistency was assessed by comparing the responses of items measuring similar aspects of the store environment. We examined the distribution of score values using boxplots and differences by store type using the Kruskal-Wallis test. RESULTS:The median difference in the overall score between surveyors was 4.4 (range 0.0-11.1), with an ICC of 0.954 (95% CI 0.914-0.975). Most measurement items had very good (n=74/196, 37.8%) or good (n=81/196, 41.3%) interrater reliability using the Gwet AC. A minimal inconsistency of measurement was found. Overall scores ranged from 19.2 to 81.6. There was a significant difference in score by store type (P<.001). Large Brisbane supermarkets scored highest (median 77.4, range 53.2-81.6), whereas small Brisbane supermarkets (median 63.9, range 41.0-71.3) and small remote supermarkets (median 63.8, range 56.5-74.9) scored significantly higher than Brisbane petrol stations (median 33.1, range 19.2-37.8) and convenience stores (median 39.0, range 22.4-63.8). CONCLUSIONS:These findings suggest good reliability and internal consistency of food environment measurements using the Store Scout mobile app. We identified specific aspects that can be improved to further increase the reliability of this tool. We found a good distribution of score values and evidence that scoring could capture differences by store type in line with previous evidence, which gives an indication of construct validity. The Store Scout mobile app shows promise in its capability to measure and track the health-enabling characteristics of store environments.
| S-EPMC7407248 | biostudies-literature
Project description:BACKGROUND: The use of mobile phones, and specifically smartphones, in the last decade has become more and more prevalent. The latest mobile phones are equipped with comprehensive features that can be used in health care, such as providing rapid access to up-to-date evidence-based information, provision of instant communications, and improvements in organization. The estimated number of health care apps for mobile phones is increasing tremendously, but previous research has highlighted the lack of critical appraisal of new apps. This lack of appraisal of apps has largely been due to the lack of clinicians with technical knowledge of how to create an evidence-based app. OBJECTIVE: We discuss two freely available methodologies for developing Web-based mobile phone apps: a website builder and an app builder. With these, users can program not just a Web-based app, but also integrate multimedia features within their app, without needing to know any programming language. METHODS: We present techniques for creating a mobile Web-based app using two well-established online mobile app websites. We illustrate how to integrate text-based content within the app, as well as integration of interactive videos and rich site summary (RSS) feed information. We will also briefly discuss how to integrate a simple questionnaire survey into the mobile-based app. A questionnaire survey was administered to students to collate their perceptions towards the app. RESULTS: These two methodologies for developing apps have been used to convert an online electronic psychiatry textbook into two Web-based mobile phone apps for medical students rotating through psychiatry in Singapore. Since the inception of our mobile Web-based app, a total of 21,991 unique users have used the mobile app and online portal provided by WordPress, and another 717 users have accessed the app via a Web-based link. The user perspective survey results (n=185) showed that a high proportion of students valued the textbook and objective structured clinical examination videos featured in the app. A high proportion of students concurred that a self-designed mobile phone app would be helpful for psychiatry education. CONCLUSIONS: These methodologies can enable busy clinicians to develop simple mobile Web-based apps for academic, educational, and research purposes, without any prior knowledge of programming. This will be beneficial for both clinicians and users at large, as there will then be more evidence-based mobile phone apps, or at least apps that have been appraised by a clinician.
| S-EPMC4285745 | biostudies-other
Project description:To facilitate easily accessible screening for trauma-related symptoms, a web-based application called Smart Assessment on your Mobile (SAM) was developed. In this study, we examined whether SAM was able to accurately identify posttraumatic stress disorder (PTSD) and depression in adults. Eighty-nine referred police officers completed SAM, containing the PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 (PCL-5) and the Depression Anxiety and Stress Scale (DASS-21), on their own device prior to a diagnostic interview where the Clinician-Administered PTSD Scale for DSM-5 (CAPS-5) and Structured Clinical Interview for DSM-IV (SCID-I/P) were administered. Results showed a substantial agreement between SAM and the diagnostic interview in the assessment of PTSD and depression. An optimal trade-off between sensitivity (89%) and specificity (68%) levels was found at a cut-off score of 31 on the PTSD Checklist for DSM-5 (area under the curve = 0.845, 95% CI [0.765, 0.925], diagnostic odds ratio = 15.97). This is one of the first studies to support the validity and reliability of a mobile screener following trauma. SAM may facilitate screening for trauma-related symptoms on a large scale and could be a first step in a stepped-care model for trauma survivors to help identify individuals who need further diagnostics and care.
| S-EPMC5639363 | biostudies-literature
Project description:BACKGROUND:Diabetes is a significant public health issue. Saudi Arabia has the highest prevalence of type 2 diabetes mellitus (T2DM) in the Arab world. Currently, it affects 31.6% of the general population, and the prevalence of T2DM is predicted to rise to 45.36% by 2030. Mobile health (mHealth) offers improved and cost-effective care to people with T2DM. However, the efficiency of engagement strategies and features of this technology need to be reviewed and standardized according to stakeholder and expert perspectives. OBJECTIVE:The main objective of this study was to identify the most agreed-upon features for T2DM self-management mobile apps; the secondary objective was to identify the most agreed-upon strategies that prompt users to use these apps. METHODS:In this study, a 4-round modified Delphi method was applied by experts in the domain of diabetes care. RESULTS:In total, 11 experts with a mean age of 47.09 years (SD 11.70) consented to participate in the study. Overall, 36 app features were generated. The group of experts displayed weak agreement in their ranking of intervention components (Kendall W=0.275; P<.001). The top 5 features included insulin dose adjustment according to carbohydrate counting and blood glucose readings (5.36), alerting a caregiver of abnormal or critical readings (6.09), nutrition education (12.45), contacts for guidance if required (12.64), and offering patient-specific education tailored to the user's goals, needs, and blood glucose readings (12.90). In total, 21 engagement strategies were generated. Overall, the experts showed a moderate degree of consensus in their strategy rankings (Kendall W=0.454; P<.001). The top 5 engagement strategies included a user-friendly design (educational and age-appropriate design; 2.82), a free app (3.73), allowing the user to communicate or send information/data to a health care provider (HCP; 5.36), HCPs prescribing the mobile app in the clinic and asking about patients' app use compliance during clinical visits (6.91), and flexibility and customization (7.91). CONCLUSIONS:This is the first study in the region consisting of a local panel of experts from the diabetes field gathering together. We used an iterative process to combine the experts' opinions into a group consensus. The results of this study could thus be useful for health app developers and HCPs and inform future decision making on the topic.
| S-EPMC7519429 | biostudies-literature
Project description:BackgroundWith thousands of health apps in app stores globally, it is crucial to systemically and thoroughly evaluate the quality of these apps due to their potential influence on health decisions and outcomes. The Mobile App Rating Scale (MARS) is the only currently available tool that provides a comprehensive, multidimensional evaluation of app quality, which has been used to compare medical apps from American and European app stores in various areas, available in English, Italian, Spanish, and German. However, this tool is not available in Arabic.ObjectiveThis study aimed to translate and adapt MARS to Arabic and validate the tool with a sample of health apps aimed at managing or preventing obesity and associated disorders.MethodsWe followed a well-established and defined "universalist" process of cross-cultural adaptation using a mixed methods approach. Early translations of the tool, accompanied by confirmation of the contents by two rounds of separate discussions, were included and culminated in a final version, which was then back-translated into English. Two trained researchers piloted the MARS in Arabic (MARS-Ar) with a sample of 10 weight management apps obtained from Google Play and the App Store. Interrater reliability was established using intraclass correlation coefficients (ICCs). After reliability was ascertained, the two researchers independently evaluated a set of additional 56 apps.ResultsMARS-Ar was highly aligned with the original English version. The ICCs for MARS-Ar (0.836, 95% CI 0.817-0.853) and MARS English (0.838, 95% CI 0.819-0.855) were good. The MARS-Ar subscales were highly correlated with the original counterparts (P<.001). The lowest correlation was observed in the area of usability (r=0.685), followed by aesthetics (r=0.827), information quality (r=0.854), engagement (r=0.894), and total app quality (r=0.897). Subjective quality was also highly correlated (r=0.820).ConclusionsMARS-Ar is a valid instrument to assess app quality among trained Arabic-speaking users of health and fitness apps. Researchers and public health professionals in the Arab world can use the overall MARS score and its subscales to reliably evaluate the quality of weight management apps. Further research is necessary to test the MARS-Ar on apps addressing various health issues, such as attention or anxiety prevention, or sexual and reproductive health.
| S-EPMC7078658 | biostudies-literature
Project description:BackgroundMobile technology may serve as a cost-effective and scalable tool for delivering behavioral nutrition interventions. This research sought to iteratively develop a theory-driven mobile app, Vegethon, to increase vegetable consumption.MethodsDevelopment of Vegethon followed phases outlined by the IDEAS framework: 1) empathize with users (qualitative interviews, n = 18); 2) specify target behavior; 3) ground in behavioral theory; 4) ideate implementation strategies; 5) prototype potential products; 6) gather user feedback (qualitative interviews, n = 14; questionnaire, n = 41); 7) build minimum viable product; and 8) pilot potential efficacy and usability (pilot RCT, n = 17). Findings from each phase informed subsequent phases. The target population that informed intervention development was 18-50 years of age, had BMIs of 28-40 kg/m(2), and lived in the geographical area surrounding Stanford University. A full description of the final version of Vegethon is included in the paper.ResultsQualitative findings that shaped initial intervention conception were: participants' interests in accountability without judgment; their desire for simple and efficient dietary self-monitoring; and the importance of planning meals in advance. Qualitative findings identified during intervention refinement were the need for a focus on vegetable self-monitoring; inclusion of vegetable challenges; simplification of features; advice and inspiration for eating vegetables; reminder notifications; and peer comparison. Pilot RCT findings suggested the initial efficacy, acceptance, and feasibility of the intervention. The final version of Vegethon enabled easy self-monitoring of vegetable consumption and included a range of features designed to engage the user (e.g., surprise challenges; leaderboard; weekly reports). Vegethon was coded for its inclusion of 18 behavior change techniques (BCTs) (e.g., goal setting; feedback; social comparison; prompts/cues; framing/reframing; identity).ConclusionsVegethon is a theory-based, user-informed mobile intervention that was systematically developed using the IDEAS framework. Vegethon targets increased vegetable consumption among overweight adults and is currently being evaluated in a randomized controlled efficacy trial.Trial registrationClinical Trials.gov: NCT01826591.
| S-EPMC4977881 | biostudies-literature