Systematic Review Protocol to Assess the Effectiveness of Usability Questionnaires in mHealth App Studies.
Ontology highlight
ABSTRACT:
SUBMITTER: Zhou L
PROVIDER: S-EPMC5558042 | biostudies-other | 2017 Aug
REPOSITORIES: biostudies-other
Publications
Systematic Review Protocol to Assess the Effectiveness of Usability Questionnaires in mHealth App Studies.
Zhou Leming L Bao Jie J Parmanto Bambang B
JMIR research protocols 20170801 8
<h4>Background</h4>Usability questionnaires have a wide use in mobile health (mHealth) app usability studies. However, no systematic review has been conducted for assessing the effectiveness of these questionnaires.<h4>Objective</h4>This paper describes a protocol for conducting a systematic review of published questionnaire-based mHealth app usability studies.<h4>Methods</h4>In this systematic review, we will select recently published (2008-2017) articles from peer-reviewed journals and confere ...[more]
Similar Datasets
Project description:BACKGROUND:After a mobile health (mHealth) app is created, an important step is to evaluate the usability of the app before it is released to the public. There are multiple ways of conducting a usability study, one of which is collecting target users' feedback with a usability questionnaire. Different groups have used different questionnaires for mHealth app usability evaluation: The commonly used questionnaires are the System Usability Scale (SUS) and Post-Study System Usability Questionnaire (PSSUQ). However, the SUS and PSSUQ were not designed to evaluate the usability of mHealth apps. Self-written questionnaires are also commonly used for evaluation of mHealth app usability but they have not been validated. OBJECTIVE:The goal of this project was to develop and validate a new mHealth app usability questionnaire. METHODS:An mHealth app usability questionnaire (MAUQ) was designed by the research team based on a number of existing questionnaires used in previous mobile app usability studies, especially the well-validated questionnaires. MAUQ, SUS, and PSSUQ were then used to evaluate the usability of two mHealth apps: an interactive mHealth app and a standalone mHealth app. The reliability and validity of the new questionnaire were evaluated. The correlation coefficients among MAUQ, SUS, and PSSUQ were calculated. RESULTS:In this study, 128 study participants provided responses to the questionnaire statements. Psychometric analysis indicated that the MAUQ has three subscales and their internal consistency reliability is high. The relevant subscales correlated well with the subscales of the PSSUQ. The overall scale also strongly correlated with the PSSUQ and SUS. Four versions of the MAUQ were created in relation to the type of app (interactive or standalone) and target user of the app (patient or provider). A website has been created to make it convenient for mHealth app developers to use this new questionnaire in order to assess the usability of their mHealth apps. CONCLUSIONS:The newly created mHealth app usability questionnaire-MAUQ-has the reliability and validity required to assess mHealth app usability.
| S-EPMC6482399 | biostudies-other
Project description:BackgroundThe use of e-health can lead to several positive outcomes. However, the potential for e-health to improve healthcare is partially dependent on its ease of use. In order to determine the usability for any technology, rigorously developed and appropriate measures must be chosen.ObjectivesTo identify psychometrically tested questionnaires that measure usability of e-health tools, and to appraise their generalizability, attributes coverage, and quality.MethodsWe conducted a systematic review of studies that measured usability of e-health tools using four databases (Scopus, PubMed, CINAHL, and HAPI). Non-primary research, studies that did not report measures, studies with children or people with cognitive limitations, and studies about assistive devices or medical equipment were systematically excluded. Two authors independently extracted information including: questionnaire name, number of questions, scoring method, item generation, and psychometrics using a data extraction tool with pre-established categories and a quality appraisal scoring table.ResultsUsing a broad search strategy, 5,558 potentially relevant papers were identified. After removing duplicates and applying exclusion criteria, 35 articles remained that used 15 unique questionnaires. From the 15 questionnaires, only 5 were general enough to be used across studies. Usability attributes covered by the questionnaires were: learnability (15), efficiency (12), and satisfaction (11). Memorability (1) was the least covered attribute. Quality appraisal showed that face/content (14) and construct (7) validity were the most frequent types of validity assessed. All questionnaires reported reliability measurement. Some questionnaires scored low in the quality appraisal for the following reasons: limited validity testing (7), small sample size (3), no reporting of user centeredness (9) or feasibility estimates of time, effort, and expense (7).ConclusionsExisting questionnaires provide a foundation for research on e-health usability. However, future research is needed to broaden the coverage of the usability attributes and psychometric properties of the available questionnaires.
| S-EPMC6241759 | biostudies-literature
Project description:BackgroundMajor surgery is associated with negative postoperative outcomes such as complications and delayed or poor recovery. Multimodal prehabilitation can help to reduce the negative effects of major surgery. Offering prehabilitation by means of mobile health (mHealth) could be an effective new approach.ObjectiveThe objectives of this pilot study were to (1) evaluate the usability of the Be Prepared mHealth app prototype for people undergoing major surgery, (2) explore whether the app was capable of bringing about a change in risk behaviors, and (3) estimate a preliminary effect of the app on functional recovery after major surgery.MethodsA mixed-methods pilot randomized controlled trial was conducted in two Dutch academic hospitals. In total, 86 people undergoing major surgery participated. Participants in the intervention group received access to the Be Prepared app, a smartphone app using behavior change techniques to address risk behavior prior to surgery. Both groups received care as usual. Usability (System Usability Scale), change in risk behaviors 3 days prior to surgery, and functional recovery 30 days after discharge from hospital (Patient-Reported Outcomes Measurement Information System physical functioning 8-item short form) were assessed using online questionnaires. Quantitative data were analyzed using descriptive statistics, chi-square tests, and multivariable linear regression. Semistructured interviews about the usability of the app were conducted with 12 participants in the intervention group. Thematic analysis was used to analyze qualitative data.ResultsSeventy-nine people-40 in the intervention group and 39 in the control group-were available for further analysis. Participants had a median age of 61 (interquartile range 51.0-68.0) years. The System Usability Scale showed that patients considered the Be Prepared app to have acceptable usability (mean 68.2 [SD 18.4]). Interviews supported the usability of the app. The major point of improvement identified was further personalization of the app. Compared with the control group, the intervention group showed an increase in self-reported physical activity and muscle strengthening activities prior to surgery. Also, 2 of 2 frequent alcohol users in the intervention group versus 1 of 9 in the control group drank less alcohol in the run-up to surgery. No difference was found in change of smoking cessation. Between-group analysis showed no meaningful differences in functional recovery after correction for baseline values (β=-2.4 [95% CI -5.9 to 1.1]).ConclusionsThe Be Prepared app prototype shows potential in terms of usability and changing risk behavior prior to major surgery. No preliminary effect of the app on functional recovery was found. Points of improvement have been identified with which the app and future research can be optimized.Trial registrationNetherlands Trial Registry NL8623; https://www.trialregister.nl/trial/8623.
| S-EPMC7819776 | biostudies-literature
Project description:Pre-eclampsia is one of the leading causes of maternal death and morbidity in low-resource countries due to delays in case identification and a shortage of health workers trained to manage the disorder. Pre-eclampsia Integrated Estimate of RiSk (PIERS) on the Move (PotM) is a low cost, easy-to-use, mobile health (mHealth) platform that has been created to aid health workers in making decisions around the management of hypertensive pregnant women. PotM combines two previously successful innovations into a mHealth app: the miniPIERS risk assessment model and the Phone Oximeter.The aim of this study was to assess the usability of PotM (with mid-level health workers) for iteratively refining the system.Development of the PotM user interface involved usability testing with target end-users in South Africa. Users were asked to complete clinical scenario tasks, speaking aloud to give feedback on the interface and then complete a questionnaire. The tool was then evaluated in a pilot clinical evaluation in Tygerberg Hospital, Cape Town.After ethical approval and informed consent, 37 nurses and midwives evaluated the tool. During Study 1, major issues in the functionality of the touch-screen keyboard and date scroll wheels were identified (total errors n=212); during Study 2 major improvements in navigation of the app were suggested (total errors n=144). Overall, users felt the app was usable using the Computer Systems Usability Questionnaire; median (range) values for Study 1 = 2 (1-6) and Study 2 = 1 (1-7). To demonstrate feasibility, PotM was used by one research nurse for the pilot clinical study. In total, more than 500 evaluations were performed on more than 200 patients. The median (interquartile range) time to complete an evaluation was 4 min 55 sec (3 min 25 sec to 6 min 56 sec).By including target end-users in the design and evaluation of PotM, we have developed an app that can be easily integrated into health care settings in low- and middle-income countries. Usability problems were often related to mobile phone features (eg, scroll wheels, touch screen use). Larger scale evaluation of the clinical impact of this tool is underway.
| S-EPMC4417132 | biostudies-other
Project description:BackgroundFalls are the leading cause of injury-related death in older adults. Due to various constraints, objective fall risk screening is seldom performed in clinical settings. Smartphones offer a high potential to provide fall risk screening for older adults in home settings. However, there is limited understanding of whether smartphone technology for falls screening is usable by older adults who present age-related changes in perceptual, cognitive, and motor capabilities.ObjectiveThe aims of this study were to develop a fall risk mobile health (mHealth) app and to determine the usability of the fall risk app in healthy, older adults.MethodsA fall risk app was developed that consists of a health history questionnaire and 5 progressively challenging mobility tasks to measure individual fall risk. An iterative design-evaluation process of semistructured interviews was performed to determine the usability of the app on a smartphone and tablet. Participants also completed a Systematic Usability Scale (SUS). In the first round of interviews, 6 older adults participated, and in the second round, 5 older adults participated. Interviews were videotaped and transcribed, and the data were coded to create themes. Average SUS scores were calculated for the smartphone and tablet.ResultsThere were 2 themes identified from the first round of interviews, related to perceived ease of use and perceived usefulness. While instructions for the balance tasks were difficult to understand, participants found it beneficial to learn about their risk for falls, found the app easy to follow, and reported confidence in using the app on their own. Modifications were made to the app, and following the second round of interviews, participants reported high ease of use and usefulness in learning about their risk of falling. Few differences were reported between using a smartphone or tablet. Average SUS scores ranged from 79 to 84.ConclusionsOur fall risk app was found to be highly usable by older adults as reported from interviews and high scores on the SUS. When designing a mHealth app for older adults, developers should include clear and simple instructions and preventative strategies to improve health. Furthermore, if the design accommodates for age-related sensory changes, smartphones can be as effective as tablets. A mobile app to assess fall risk has the potential to be used in home settings by older adults.
| S-EPMC6716481 | biostudies-literature
Project description:ObjectivesThe use of interactive mobile health (mHealth) applications to monitor patient-reported postoperative pain outcomes is an emerging area in dentistry that requires further exploration. This study aimed to evaluate and improve the usability of an existing mHealth application.Materials and methodsThe usability of the application was assessed iteratively using a 3-phase approach, including a rapid cognitive walkthrough (Phase I), lab-based usability testing (Phase II), and in situ pilot testing (Phase III). The study team conducted Phase I, while providers and patients participated in Phase II and III.ResultsThe rapid cognitive walkthrough identified 23 potential issues that could negatively impact user experience, with the majority classified as system issues. The lab-based usability testing yielded 141 usability issues.; 43% encountered by patients and 57% by dentists. Usability problems encountered during pilot testing included undelivered messages due to mobile phone carrier and service-related issues, errors in patients' phone number data entry, and problems in provider training.DiscussionThrough collaborative and iterative work with the vendor, usability issues were addressed before launching a trial to assess its efficacy.ConclusionThe usability of the mHealth application for postoperative dental pain was remarkably improved by the iterative analysis and interdisciplinary collaboration.
| S-EPMC10928307 | biostudies-literature
Project description:BackgroundSuicide safety planning is an evidence-based approach used to help individuals identify strategies to keep themselves safe during a mental health crisis. This study systematically reviewed the literature focused on mobile health (mHealth) suicide safety planning apps.ObjectiveThis study aims to evaluate the extent to which apps integrated components of the safety planning intervention (SPI), and if so, how these safety planning components were integrated into the design-based features of the apps.MethodsFollowing the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, we systematically analyzed 14 peer-reviewed studies specific to mHealth apps for suicide safety planning. We conducted an analysis of the literature to evaluate how the apps incorporated SPI components and examined similarities and differences among the apps by conducting a comparative analysis of app features. An independent review of SPI components and app features was conducted by downloading the available apps.ResultsMost of the mHealth apps (5/7, 71%) integrated SPI components and provided customizable features that expanded upon traditional paper-based safety planning processes. App design features were categorized into 5 themes, including interactive features, individualized user experiences, interface design, guidance and training, and privacy and sharing. All apps included access to community supports and revisable safety plans. Fewer mHealth apps (3/7, 43%) included interactive features, such as associating coping strategies with specific stressors. Most studies (10/14, 71%) examined the usability, feasibility, and acceptability of the safety planning mHealth apps. Usability findings were generally positive, as users often found these apps easy to use and visually appealing. In terms of feasibility, users preferred using mHealth apps during times of crisis, but the continuous use of the apps outside of crisis situations received less support. Few studies (4/14, 29%) examined the effectiveness of mHealth apps for suicide-related outcomes. Positive shifts in attitudes and desire to live, improved coping strategies, enhanced emotional stability, and a decrease in suicidal thoughts or self-harm behaviors were examined in these studies.ConclusionsOur study highlights the need for researchers, clinicians, and app designers to continue to work together to align evidence-based research on mHealth suicide safety planning apps with lessons learned for how to best deliver these technologies to end users. Our review brings to light mHealth suicide safety planning strategies needing further development and testing, such as lethal means guidance, collaborative safety planning, and the opportunity to embed more interactive features that leverage the advanced capabilities of technology to improve client outcomes as well as foster sustained user engagement beyond a crisis. Although preliminary evidence shows that these apps may help to mitigate suicide risk, clinical trials with larger sample sizes and more robust research designs are needed to validate their efficacy before the widespread adoption and use.
| S-EPMC11009854 | biostudies-literature
Project description:BackgroundeHealth has the potential to improve outcomes such as physical activity or balance in older adults receiving geriatric rehabilitation. However, several challenges such as scarce evidence on effectiveness, feasibility, and usability hinder the successful implementation of eHealth in geriatric rehabilitation.ObjectiveThe aim of this systematic review was to assess evidence on the effectiveness, feasibility, and usability of eHealth interventions in older adults in geriatric rehabilitation.MethodsWe searched 7 databases for randomized controlled trials, nonrandomized studies, quantitative descriptive studies, qualitative research, and mixed methods studies that applied eHealth interventions during geriatric rehabilitation. Included studies investigated a combination of effectiveness, usability, and feasibility of eHealth in older patients who received geriatric rehabilitation, with a mean age of ≥70 years. Quality was assessed using the Mixed Methods Appraisal Tool and a narrative synthesis was conducted using a harvest plot.ResultsIn total, 40 studies were selected, with clinical heterogeneity across studies. Of 40 studies, 15 studies (38%) found eHealth was at least as effective as non-eHealth interventions (56% of the 27 studies with a control group), 11 studies (41%) found eHealth interventions were more effective than non-eHealth interventions, and 1 study (4%) reported beneficial outcomes in favor of the non-eHealth interventions. Of 17 studies, 16 (94%) concluded that eHealth was feasible. However, high exclusion rates were reported in 7 studies of 40 (18%). Of 40 studies, 4 (10%) included outcomes related to usability and indicated that there were certain aging-related barriers to cognitive ability, physical ability, or perception, which led to difficulties in using eHealth.ConclusionseHealth can potentially improve rehabilitation outcomes for older patients receiving geriatric rehabilitation. Simple eHealth interventions were more likely to be feasible for older patients receiving geriatric rehabilitation, especially, in combination with another non-eHealth intervention. However, a lack of evidence on usability might hamper the implementation of eHealth. eHealth applications in geriatric rehabilitation show promise, but more research is required, including research with a focus on usability and participation.
| S-EPMC8414304 | biostudies-literature
Project description:BackgroundMobile health (mHealth) apps play an important role in delivering education, providing advice on treatment, and monitoring patients' health. Good usability of mHealth apps is essential to achieve the objectives of mHealth apps efficiently. To date, there are questionnaires available to assess the general system usability but not explicitly tailored to precisely assess the usability of mHealth apps. Hence, the mHealth App Usability Questionnaire (MAUQ) was developed with 4 versions according to the type of app (interactive or standalone) and according to the target user (patient or provider). Standalone MAUQ for patients comprises 3 subscales, which are ease of use, interface and satisfaction, and usefulness.ObjectiveThis study aimed to translate and validate the English version of MAUQ (standalone for patients) into a Malay version of MAUQ (M-MAUQ) for mHealth app research and usage in future in Malaysia.MethodsForward and backward translation and harmonization of M-MAUQ were conducted by Malay native speakers who also spoke English as their second language. The process began with a forward translation by 2 independent translators followed by harmonization to produce an initial translated version of M-MAUQ. Next, the forward translation was continued by another 2 translators who had never seen the original MAUQ. Lastly, harmonization was conducted among the committee members to resolve any ambiguity and inconsistency in the words and sentences of the items derived with the prefinal adapted questionnaire. Subsequently, content and face validations were performed with 10 experts and 10 target users, respectively. Modified kappa statistic was used to determine the interrater agreement among the raters. The reliability of the M-MAUQ was assessed by 51 healthy young adult mobile phone users. Participants needed to install the MyFitnessPal app and use it for 2 days for familiarization before completing the designated task and answer the M-MAUQ. The MyFitnessPal app was selected because it is one among the most popular installed mHealth apps globally available for iPhone and Android users and represents a standalone mHealth app.ResultsThe content validity index for the relevancy and clarity of M-MAUQ were determined to be 0.983 and 0.944, respectively, which indicated good relevancy and clarity. The face validity index for understandability was 0.961, which indicated that users understood the M-MAUQ. The kappa statistic for every item in M-MAUQ indicated excellent agreement between the raters (κ ranging from 0.76 to 1.09). The Cronbach α for 18 items was .946, which also indicated good reliability in assessing the usability of the mHealth app.ConclusionsThe M-MAUQ fulfilled the validation criteria as it revealed good reliability and validity similar to the original version. M-MAUQ can be used to assess the usability of mHealth apps in Malay in the future.
| S-EPMC7894394 | biostudies-literature
Project description:BackgroundMultiple sclerosis (MS) is a chronic, neurodegenerative disease that causes a range of motor, sensory, and cognitive symptoms. Due to these symptoms, people with MS are at a high risk for falls, fall-related injuries, and reductions in quality of life. There is no cure for MS, and managing symptoms and disease progression is important to maintain a high quality of life. Mobile health (mHealth) apps are commonly used by people with MS to help manage their health. However, there are limited health apps for people with MS designed to evaluate fall risk. A fall risk app can increase access to fall risk assessments and improve self-management. When designing mHealth apps, a user-centered approach is critical for improving use and adoption.ObjectiveThe purpose of this study is to undergo a user-centered approach to test and refine the usability of the app through an iterative design process.MethodsThe fall risk app Steady-MS is an extension of Steady, a fall risk app for older adults. Steady-MS consists of 2 components: a 25-item questionnaire about demographics and MS symptoms and 5 standing balance tasks. Data from the questionnaire and balance tasks were inputted into an algorithm to compute a fall risk score. Two iterations of semistructured interviews (n=5 participants per iteration) were performed to evaluate usability. People with MS used Steady-MS on a smartphone, thinking out loud. Interviews were recorded, transcribed, and developed into codes and themes. People with MS also completed the System Usability Scale.ResultsA total of 3 themes were identified: intuitive navigation, efficiency of use, and perceived value. Overall, the participants found Steady-MS efficient to use and useful to learn their fall risk score. There were challenges related to cognitive overload during the balance tasks. Modifications were made, and after the second iteration, people with MS reported that the app was intuitive and efficient. Average System Usability Scale scores were 95.5 in both iterations, representing excellent usability.ConclusionsSteady-MS is the first mHealth app for people with MS to assess their overall risk of falling and is usable by a subset of people with MS. People with MS found Steady-MS to be usable and useful for understanding their risk of falling. When developing future mHealth apps for people with MS, it is important to prevent cognitive overload through simple and clear instructions and present scores that are understood and interpreted correctly through visuals and text. These findings underscore the importance of user-centered design and provide a foundation for the future development of tools to assess and prevent scalable falls for people with MS. Future steps include understanding the validity of the fall risk algorithm and evaluating the clinical utility of the app.
| S-EPMC8080269 | biostudies-literature