The Iterative Convergent Design for Mobile Health Usability Testing: Mixed Methods Approach.
Ontology highlight
ABSTRACT:
SUBMITTER: Alwashmi MF
PROVIDER: S-EPMC6658163 | biostudies-literature | 2019 Apr
REPOSITORIES: biostudies-literature
Publications
The Iterative Convergent Design for Mobile Health Usability Testing: Mixed Methods Approach.
Alwashmi Meshari F MF Hawboldt John J Davis Erin E Fetters Michael D MD
JMIR mHealth and uHealth 20190426 4
Although patients express an interest in using mobile health (mHealth) interventions to manage their health and chronic conditions, many current mHealth interventions are difficult to use. Usability testing is critical for the success of novel mHealth interventions. Researchers recognize the utility of using qualitative and quantitative approaches for usability testing, but many mHealth researchers lack the awareness of integration approaches from advances in mixed methods research that can add ...[more]
Similar Datasets
Project description:BackgroundMusculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on health care utilization. Digital technologies that improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials-the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials.ObjectiveThis study will use the iterative convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. Using this methodology, we will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. This study protocol will provide a blueprint for future usability studies of mobile health solutions.MethodsWe will collect qualitative and quantitative data from 20-30 participants aged 18 years and older for 4 months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 standard and the system usability scale, providing a usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles.ResultsThis study received approval from the Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) on June 4, 2020. At manuscript submission, study recruitment was on-going, with data collection to be completed and results published in 2021.ConclusionsThis study will provide evidence concerning mobile health DART system usability and acceptance determining system improvements required to support user adoption and minimize suboptimal system usability as a potential confounder within subsequent noninferiority clinical trials. Success should produce a safe effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This deliberately rigorous approach to mobile health innovation could be used as a guide for other developers of similar apps.International registered report identifier (irrid)DERR1-10.2196/27205.
| S-EPMC8170557 | biostudies-literature
Project description:BackgroundMusculoskeletal disorders negatively affect millions of patients worldwide, placing significant demand on health care systems. Digital technologies that improve clinical outcomes and efficiency across the care pathway are development priorities. We developed the musculoskeletal Digital Assessment Routing Tool (DART) to enable self-assessment and immediate direction to the right care.ObjectiveWe aimed to assess and resolve all serious DART usability issues to create a positive user experience and enhance system adoption before conducting randomized controlled trials for the integration of DART into musculoskeletal management pathways.MethodsAn iterative, convergent mixed methods design was used, with 22 adult participants assessing 50 different clinical presentations over 5 testing rounds across 4 DART iterations. Participants were recruited using purposive sampling, with quotas for age, habitual internet use, and English-language ability. Quantitative data collection was defined by the constructs within the International Organization for Standardization 9241-210-2019 standard, with user satisfaction measured by the System Usability Scale. Study end points were resolution of all grade 1 and 2 usability problems and a mean System Usability Scale score of ≥80 across a minimum of 3 user group sessions.ResultsAll participants (mean age 48.6, SD 15.2; range 20-77 years) completed the study. Every assessment resulted in a recommendation with no DART system errors and a mean completion time of 5.2 (SD 4.44, range 1-18) minutes. Usability problems were reduced from 12 to 0, with trust and intention to act improving during the study. The relationship between eHealth literacy and age, as explored with a scatter plot and calculation of the Pearson correlation coefficient, was performed for all participants (r=-0.2; 20/22, 91%) and repeated with a potential outlier removed (r=-0.23), with no meaningful relationships observed or found for either. The mean satisfaction for daily internet users was highest (19/22, 86%; mean 86.5, SD 4.48; 90% confidence level [CL] 1.78 or -1.78), with nonnative English speakers (6/22, 27%; mean 78.1, SD 4.60; 90% CL 3.79 or -3.79) and infrequent internet users scoring the lowest (3/22, 14%; mean 70.8, SD 5.44; 90% CL 9.17 or -9.17), although the CIs overlap. The mean score across all groups was 84.3 (SD 4.67), corresponding to an excellent system, with qualitative data from all participants confirming that DART was simple to use.ConclusionsAll serious DART usability issues were resolved, and a good level of satisfaction, trust, and willingness to act on the DART recommendation was demonstrated, thus allowing progression to randomized controlled trials that assess safety and effectiveness against usual care comparators. The iterative, convergent mixed methods design proved highly effective in fully evaluating DART from a user perspective and could provide a blueprint for other researchers of mobile health systems.International registered report identifier (irrid)RR2-10.2196/27205.
| S-EPMC9472040 | biostudies-literature
Project description:BackgroundReporting of medication errors is one of the essential mechanisms to identify risky health care systems and practices that lead to medication errors. Unreported medication errors are a real issue; one of the identified causes is a burdensome medication error reporting system. An anonymous and user-friendly mobile app for reporting medication errors could be an alternative method of reporting medication error in busy health care settings.ObjectiveThe objective of this paper is to report usability testing of the Medication Error Reporting App (MERA), a mobile app for reporting medication errors anonymously.MethodsQuantitative and qualitative methods were employed involving 45 different testers (pharmacists, doctors, and nurses) from a large tertiary hospital in Malaysia. Quantitative data was retrieved using task performance and rating of MERA and qualitative data were retrieved through focus group discussions. Three sessions, with 15 testers each session, were conducted from January to March 2018.ResultsThe majority of testers were pharmacists (23/45, 51%), female (35/45, 78%), and the mean age was 36 (SD 9) years. A total of 135 complete reports were successfully submitted by the testers (three reports per tester) and 79.2% (107/135) of the reports were correct. There was significant improvement in mean System Usability Scale scores in each session of the development process (P<.001) and mean time to report medication errors using the app was not significantly different between each session (P=.70) with an overall mean time of 6.7 (SD 2.4) minutes. Testers found the app easy to use, but doctors and nurses were unfamiliar with terms used especially medication process at which error occurred and type of error. Although, testers agreed the app can be used in the future for reporting, they were apprehensive about security, validation, and abuse of feedback featured in the app.ConclusionsMERA can be used to report medication errors easily by various health care personnel and it has the capacity to provide feedback on reporting. However, education on medication error reporting should be provided to doctors and nurses in Malaysia and the security of the app needs to be established to boost reporting by this method.
| S-EPMC6320434 | biostudies-literature
Project description:BackgroundPregnant women are often unaware of the potential risks that working conditions can cause to them and their unborn child. A mobile health (mHealth) app, the Pregnancy and Work (P and W) app, developed by a multidisciplinary team and based on an evidence-based guideline for occupational physicians, aims to provide advice on work adjustment during pregnancy.ObjectiveThis study evaluates the usability of the mHealth P and W app and the perceived usefulness of the work advice, the main goal of the app, by potential end users.MethodsA total of 12 working pregnant women participated in think aloud usability sessions and performed 9 tasks. All think aloud sessions were recorded, transcribed, and coanalyzed. The usability problems were rated for their severity in accordance with Nielsen severity scale. The completion rates and time taken for completion of tasks were registered. In addition, participants were questioned on demographics and user characteristics and were asked to evaluate the value of the app by filling in the Intrinsic Motivation Inventory (IMI) score and the System Usability Scale (SUS) questionnaire.ResultsIn total, 82 usability problems with a severity ≥1 were identified, of which 40 had severity ≥3. The main usability problems concerned the interpretation of terminology used in the app's questionnaires and difficulties in finding and understanding the work advice. Furthermore, 10 out of 12 participants were able to open the work advice page in the app. Only 7 out of these 10 participants understood and intended to follow the work advice. The overall mean IMI score was relatively high (5 out of 7), indicating that the participants did indeed value the use of the app. This IMI score corresponded to the overall mean SUS score (68 out of 100) and the mean grade given to the P and W app (7 out of 10).ConclusionsThis think aloud usability study showed that the information provided in the P and W app was considered valuable by the end users, working pregnant women, and it meets their needs; however, usability issues severely impacted the perceived usefulness of the work advice given in the app.
| S-EPMC6532337 | biostudies-literature
Project description:Multiple sclerosis (MS) is a chronic neurological condition affecting around 2.2 million people worldwide. The illness includes a range of symptoms, with fatigue considered to be one of the most disabling. This paper describes how a pragmatic and iterative approach, supported by usability and resonance testing, was used to build a minimum viable product of MS Energize-or MS Energise in UK English regions. MS Energise is a mobile application focused on self-management of fatigue for people with MS. The iterative approach included various stages of testing, during which user feedback including comments about interface, navigation and content, was sought to inform incremental app development and continual improvement. Usability testing was conducted with 11 people with longstanding multiple sclerosis in New Zealand and the United Kingdom, and focused on particular sections of the app as well as the accessibility of the app to users with MS. Two participants contributed to further resonance testing post-release to ensure the app was perceived as relevant and useful to the user. The usability testing and resonance testing phases suggested that user experience of MS Energise was mostly positive. Participants provided a number of suggestions for improvements to aspects of content and design; some of which we implemented during our app development process. Findings will also contribute to future planning and design iteration to enhance the user experience. The next step is further improvement of MS Energise prior to a trial of its clinical and cost effectiveness.
| S-EPMC7878181 | biostudies-literature
Project description:BackgroundCholera poses a significant global health burden. In Bangladesh, cholera is endemic and causes more than 100,000 cases each year. Established environmental reservoirs leave millions at risk of infection through the consumption of contaminated water. The Global Task Force for Cholera Control has called for increased environmental surveillance to detect contaminated water sources prior to human infection in an effort to reduce cases and deaths. The OmniVis rapid cholera detection device uses loop-mediated isothermal amplification and particle diffusometry detection methods integrated into a handheld hardware device that attaches to an iPhone 6 to identify and map contaminated water sources.ObjectiveThe aim of this study was to evaluate the usability of the OmniVis device with targeted end users to advance the iterative prototyping process and ultimately design a device that easily integrates into users' workflow.MethodsWater quality workers were trained to use the device and subsequently completed an independent device trial and usability questionnaire. Pretraining and posttraining knowledge assessments were administered to ensure training quality did not confound trial and questionnaire.ResultsDevice trials identified common user errors and device malfunctions including incorrect test kit insertion and device powering issues. We did not observe meaningful differences in user errors or device malfunctions accumulated per participant across demographic groups. Over 25 trials, the mean time to complete a test was 47 minutes, a significant reduction compared with laboratory protocols, which take approximately 3 days. Overall, participants found the device easy to use and expressed confidence and comfort in using the device independently.ConclusionsThese results are used to advance the iterative prototyping process of the OmniVis rapid cholera detection device so it can achieve user uptake, workflow integration, and scale to ultimately impact cholera control and elimination strategies. We hope this methodology will promote robust usability evaluations of rapid pathogen detection technologies in device development.
| S-EPMC8156127 | biostudies-literature
Project description:We introduce a mixed-methods approach for determining how people weigh tradeoffs in values related to health and technologies for health self-management. Our approach combines interviews with Q-methodology, a method from psychology uniquely suited to quantifying opinions. We derive the framework for structured data collection and analysis for the Q-methodology from theories of self-management of chronic illness and technology adoption. To illustrate the power of this new approach, we used it in a field study of nine older adults with type 2 diabetes, and nine mothers of children with asthma. Our mixed-methods approach provides three key advantages for health design science in HCI: (1) it provides a structured health sciences theoretical framework to guide data collection and analysis; (2) it enhances the coding of unstructured data with statistical patterns of polarizing and consensus views; and (3) it empowers participants to actively weigh competing values that are most personally significant to them.
| S-EPMC5554065 | biostudies-other
Project description:BackgroundBlack young adults suffer from psychological distress at either similar or greater rates than that of White Americans, yet they are seven times less likely to have access to or receive effective treatments. Fortunately, mobile-health (mHealth) technology may transform mental health services and address disparities in mental healthcare. The current study utilized focus groups of Black young adults to inform the development of culturally-adapted mHealth using quantitative and qualitative approaches.MethodsThe study utilized a mixed-methods approach, in that qualitative (i.e., mini focus groups, n=11) and quantitative methods (i.e., self-report surveys) were used to explore the research questions. Participants included African American young adults (n=38, Mage =21). Participants completed self-report questionnaires prior to focus group facilitation. Correlational analyses were used to answer the quantitative research questions, and thematic analysis was used to answer the qualitative research questions.ResultsThe qualitative findings highlighted that sociocultural experiences impact mental health and treatment seeking attitudes. Despite these findings, participants highlighted a variety of desired features and content that should be incorporated into future culturally-adapted mHealth interventions. Participants also highlighted both positive and negative aspects of current mHealth technologies for mental health. Finally, the study found that on average, participants had positive attitudes towards mental health, mental health treatments, and utilizing mHealth for mental health. Participants also had strong desires for culturally-adapted mHealth interventions. Bivariate correlations also revealed significant associations between vicarious online racial discrimination and mHealth attitudes, as well as racial identity and mHealth attitudes.ConclusionsIn summary, the current study highlights that there is an urgent need for mHealth technology for mental health symptoms for African American young adults and presents a variety of features, content, and design/development considerations for future researchers.
| S-EPMC9902238 | biostudies-literature
Project description:BACKGROUND:Mobile interventions hold promise as an intervention modality to engage children in improving diabetes self-management education, attitudes, and behaviors. OBJECTIVE:This pilot study aimed to explore the usability, acceptability, and feasibility of delivering a mobile diabetes educational tool to parent-child pairs in a clinical setting. METHODS:This mixed methods pilot study comprised two concurrent phases with differing study participants. Phase 1 used user testing interviews to collect qualitative data on the usability and acceptability of the tool. Phase 2 used a single-arm pre- and poststudy design to quantitatively evaluate the feasibility and preliminary efficacy of the intervention. Study participants (English-speaking families with youth aged 5-14 years with insulin-dependent diabetes) were recruited from an urban hospital in Massachusetts, United States. In phase 1, parent-child pairs were invited to complete the intervention together and participate in 90-min user testing interviews assessing the tool's usability and acceptability. Interview transcripts were analyzed using a directed content analysis approach. In phase 2, parent-child pairs were invited to complete the intervention together in the clinical setting. Measures included parental and child knowledge, attitudes, and behaviors related to diabetes management (self-report surveys) and child hemoglobin A1c levels (medical record extractions); data were collected at baseline and 1-month follow-up. Pre- and postoutcomes were compared using paired t tests and the Fisher exact test. RESULTS:A total of 11 parent-child pairs (N=22) participated in phase 1 of the study, and 10 parent-child pairs (N=20) participated in phase 2 of the study. Participants viewed the mobile educational tool as acceptable (high engagement and satisfaction with the layout, activities, and videos) and identified the areas of improvement for tool usability (duration, directions, and animation). CONCLUSIONS:The findings from this pilot study suggest that the mobile educational tool is an informative, engaging, and feasible way to deliver diabetes self-management education to parents and children in an urban hospital setting. Data will inform future iterations of this mobile diabetes educational intervention to improve usability and test intervention efficacy.
| S-EPMC7229529 | biostudies-literature
Project description:BackgroundThe prepandemic period saw a rise in web-based teaching. However, web-based tools for teaching the essential clinical skill of cognitive empathy (also known as perspective taking) remain limited. More of these tools are needed and require testing for ease of use and understanding by students.ObjectiveThis study aimed to evaluate the usability of the In Your Shoes web-based empathy training portal application for students using quantitative and qualitative methods.MethodsThis 3-phase formative usability study used a mixed methods design. In mid-2021, we conducted a remote observation of student participants interacting with our portal application. Their qualitative reflections were captured, followed by data analysis and iterative design refinements of the application. Overall, 8 third- and fourth-year nursing students from an undergraduate baccalaureate program at a Canadian university, in the western province of Manitoba, were included in this study. Participants in phases 1 and 2 were remotely observed by 3 research personnel while engaged in predefined tasks. In phase 3, two student participants were asked to use the application as they liked in their own environments, after which a video-recorded exit interview with a think-aloud process was conducted as participants responded to the System Usability Scale. We calculated descriptive statistics and performed content analysis to analyze the results.ResultsThis small study included 8 students with a range of technology skills. Usability themes were based on participants' comments on the application's appearance, content, navigation, and functionality. The biggest issues that participants experienced were with navigating the application's "tagging" features during video analysis and the length of educational material. We also observed variations in 2 participants' system usability scores in phase 3. This may be because of their different comfort levels with technology; however, additional research is required. We made iterative refinements to our prototype application (eg, added pop-up messages and provided a narrated video on the application's "tagging" function) based on participant feedback.ConclusionsWith increasing engagement in web-based teaching, technology has become an essential medium for receiving health care education. We developed a novel prototype application as a supplemental classroom tool to foster students' self-directed learning of empathy. This study provided direction for refinements to optimize the usability of and satisfaction with this innovative application. Qualitative feedback revealed favorable input toward learning perspective taking place on the web and helpful recommendations for improving user experiences with the application. We could not fully assess the application's key functions owing to the COVID-19 protocols. Thus, our next step is to obtain feedback from a larger sample of student users, whose experiences performing "live" video capture, annotation, and analysis will be more authentic and wholesome with the refined application. We discuss our findings in relation to research on nursing education, perspective taking, and adaptive e-learning.
| S-EPMC10131903 | biostudies-literature