Dataset Information

Usability Study of Mainstream Wearable Fitness Devices: Feature Analysis and System Usability Scale Evaluation.

ABSTRACT:

Background

Wearable devices have the potential to promote a healthy lifestyle because of their real-time data monitoring capabilities. However, device usability is a critical factor that determines whether they will be adopted on a large scale. Usability studies on wearable devices are still scarce.

Objective

This study aims to compare the functions and attributes of seven mainstream wearable devices and to evaluate their usability.

Methods

The wearable devices selected were the Apple Watch, Samsung Gear S, Fitbit Surge, Jawbone Up3, Mi Band, Huawei Honor B2, and Misfit Shine. A mixed method of feature comparison and a System Usability Scale (SUS) evaluation based on 388 participants was applied; the higher the SUS score, the better the usability of the product.

Results

For features, all devices had step counting, an activity timer, and distance recording functions. The Samsung Gear S had a unique sports track recording feature and the Huawei Honor B2 had a unique wireless earphone. The Apple Watch, Samsung Gear S, Jawbone Up3, and Fitbit Surge could measure heart rate. All the devices were able to monitor sleep, except the Apple Watch. For product characteristics, including attributes such as weight, battery life, price, and 22 functions such as step counting, activity time, activity type identification, sleep monitoring, and expandable new features, we found a very weak negative correlation between the SUS scores and price (r=-.10, P=.03) and devices that support expandable new features (r=-.11, P=.02), and a very weak positive correlation between the SUS scores and devices that support the activity type identification function (r=.11, P=.02). The Huawei Honor B2 received the highest score of mean 67.6 (SD 16.1); the lowest Apple Watch score was only 61.4 (SD 14.7). No significant difference was observed among brands. The SUS score had a moderate positive correlation with the user's experience (length of time the device was used) (r=.32, P<.001); participants in the medical and health care industries gave a significantly higher score (mean 61.1, SD 17.9 vs mean 68.7, SD 14.5, P=.03).

Conclusions

The functions of wearable devices tend to be homogeneous and usability is similar across various brands. Overall, Mi Band had the lowest price and the lightest weight. Misfit Shine had the longest battery life and most functions, and participants in the medical and health care industries had the best evaluation of wearable devices. The perceived usability of mainstream wearable devices is unsatisfactory and customer loyalty is not high. A consumer's SUS rating for a wearable device is related to their personal situation instead of the device brand. Device manufacturers should put more effort into developing innovative functions and improving the usability of their products by integrating more cognitive behavior change techniques.

SUBMITTER: Liang J

PROVIDER: S-EPMC6250954 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThere are many problems with fitness trackers, such as device usability, which limit their large-scale application, and relevant studies are limited in terms of their sample size and evaluation methods. The purpose of the study was to evaluate the perceived usability of various mainstream fitness trackers on the market, and to learn about user feedback on feature preferences for each device.MethodsTrial use of seven mainstream fitness trackers (two smart watches and five smart wristbands) followed by a survey study were applied. The questionnaire was specifically developed for this study, which included two parts (user preferences and device usability in five dimensions). We recruited users to test the devices for at least 30 days and asked experienced users to provide feedback in order to evaluate each device, including the rating and user preference of each device.ResultsWe received 388 valid questionnaires, in which users rated their responses on a five-point Likert scale. (1) User preference: the average user satisfaction was 3.50-3.86 (points), and the rating for willingness to buy averaged between 3.36 and 3.59. More users were willing to wear (58.3-81.3%) and purchase (56.8-83.0%) the devices than were not. The top three general feature preferences were daily activity tracking, heart health monitoring, and professional fitness tracking. The top three health-related feature preferences were heart rate monitoring, daily pedometer, and professional fitness tracking. (2) Usability evaluation: product design was rated from 3.57 to 4.00; durability, 3.63-4.26; ease of use, 3.70-3.90; added features, 3.30-3.83; and user-rated accuracy, 3.44-3.78. A significant difference was observed in the rating of product design and durability among the different devices (p < 0.05) score.ConclusionsUsers generally had positive subjective intent regarding fitness trackers but were less satisfied with their cost effectiveness. The users preferred health related features such as heart health monitoring, and professional fitness tracking. The rating of most of the current mainstream fitness trackers was fair with some significant differences among the devices. Thus, further improvement is needed.

Project description:BackgroundUser-centered design approaches have gained attention over the past decade, aiming to tackle the technology acceptance issues of wearable robotic devices to assist, support or augment human capabilities. While there is a consensus that usability is key to user-centered design, dedicated usability evaluation studies are scarce and clear evaluation guidelines are missing. However, the careful consideration and integration of user needs appears to be essential to successfully develop an effective, efficient, and satisfactory human-robot interaction. It is primarily the responsibility of the developer, to ensure that this users involvement takes place throughout the design process.MethodsThrough an online survey for developers of wearable robotics, we wanted to understand how the design and evaluation in actual daily practice compares to what is reported in literature. With a total of 31 questions, we analyzed the most common wearable robotic device applications and their technology maturity, and how these influence usability evaluation practices.ResultsA total of 158 responses from a heterogeneous population were collected and analyzed. The dataset representing contexts of use for augmentation (16.5%), assistance (38.0%), therapy (39.8%), as well as few other specific applications (5.7%), allowed for an insightful analysis of the influence of technology maturity on user involvement and usability evaluation. We identified functionality, ease of use, and performance as the most evaluated usability attributes and could specify which measures are used to assess them. Also, we could underline the frequent use of qualitative measures alongside the expected high prevalence of performance-metrics. In conclusion of the analysis, we derived evaluation recommendations to foster user-centered design and usability evaluation.ConclusionThis analysis might serve as state-of-the-art comparison and recommendation for usability studies in wearable robotics. We believe that by motivating for more balanced, comparable and user-oriented evaluation practices, we may support the wearable robotics field in tackling the technology acceptance limitations.

Project description:BackgroundThe System Usability Scale (SUS) is a widely used scale that has been used to quantify the usability of many software and hardware products. However, the SUS was not specifically designed to evaluate mobile apps, or in particular digital health apps (DHAs).ObjectiveThe aim of this study was to examine whether the widely used SUS distribution for benchmarking (mean 68, SD 12.5) can be used to reliably assess the usability of DHAs.MethodsA search of the literature was performed using the ACM Digital Library, IEEE Xplore, CORE, PubMed, and Google Scholar databases to identify SUS scores related to the usability of DHAs for meta-analysis. This study included papers that published the SUS scores of the evaluated DHAs from 2011 to 2021 to get a 10-year representation. In total, 117 SUS scores for 114 DHAs were identified. R Studio and the R programming language were used to model the DHA SUS distribution, with a 1-sample, 2-tailed t test used to compare this distribution with the standard SUS distribution.ResultsThe mean SUS score when all the collected apps were included was 76.64 (SD 15.12); however, this distribution exhibited asymmetrical skewness (-0.52) and was not normally distributed according to Shapiro-Wilk test (P=.002). The mean SUS score for "physical activity" apps was 83.28 (SD 12.39) and drove the skewness. Hence, the mean SUS score for all collected apps excluding "physical activity" apps was 68.05 (SD 14.05). A 1-sample, 2-tailed t test indicated that this health app SUS distribution was not statistically significantly different from the standard SUS distribution (P=.98).ConclusionsThis study concludes that the SUS and the widely accepted benchmark of a mean SUS score of 68 (SD 12.5) are suitable for evaluating the usability of DHAs. We speculate as to why physical activity apps received higher SUS scores than expected. A template for reporting mean SUS scores to facilitate meta-analysis is proposed, together with future work that could be done to further examine the SUS benchmark scores for DHAs.

Project description:BACKGROUND:There is growing interest in using wearable devices to remotely monitor patient behaviors. However, there has been little evaluation of how often these technologies are used to monitor sleep patterns over longer term periods, particularly among more high-risk patients. OBJECTIVE:The goal of the research was to evaluate the proportion of time that patients with ischemic heart disease used wearable devices to monitor their sleep and identify differences in characteristics of patients with higher versus lower use. METHODS:We evaluated wearable device data from a previously conducted clinical trial testing the use of wearable devices with personalized goal-setting and financial incentives. Patients with ischemic heart disease established a sleep baseline and were then followed for 24 weeks. The proportion of days that sleep data was collected was compared over the 24 weeks and by study arm. Characteristics of patients were compared to groups with high, low, or no sleep data. RESULTS:The sample comprised 99 patients with ischemic heart disease, among which 79% (78/99) used the wearable device to track their sleep. During the 6-month trial, sleep data were collected on 60% (10,024/16,632) of patient-days. These rates declined over time from 77% (4292/5544) in months 1 and 2 to 58% (3188/5544) in months 3 and 4 to 46% (2544/5544) in months 5 and 6. Sleep data were collected at higher rates among the intervention group compared with control (67% vs 55%, P<.001). In the main intervention period (months 3 and 4), patients with higher rates of sleep data were on average older (P=.03), had a history of smoking (P=.007), and had higher rates of commercial health insurance (P=.03). CONCLUSIONS:Among patients with ischemic heart disease in a physical activity trial, a high proportion used wearable devices to track their sleep; however, rates declined over time. Future research should consider larger evaluations coupled with behavioral interventions. TRIAL REGISTRATION:ClinicalTrials.gov NCT02531022; https://clinicaltrials.gov/ct2/show/NCT02531022.

Project description:BackgroundThe World Health Organisation's global strategy for digital health emphasises the importance of patient involvement. Understanding the usability and acceptability of wearable devices is a core component of this. However, usability assessments to date have focused predominantly on healthy adults. There is a need to understand the patient perspective of wearable devices in participants with chronic health conditions.MethodsA systematic review was conducted to identify any study design that included a usability assessment of wearable devices to measure mobility, through gait and physical activity, within five cohorts with chronic conditions (Parkinson's disease [PD], multiple sclerosis [MS], congestive heart failure, [CHF], chronic obstructive pulmonary disorder [COPD], and proximal femoral fracture [PFF]).ResultsThirty-seven studies were identified. Substantial heterogeneity in the quality of reporting, the methods used to assess usability, the devices used, and the aims of the studies precluded any meaningful comparisons. Questionnaires were used in the majority of studies (70.3%; n = 26) with a reliance on intervention specific measures (n = 16; 61.5%). For those who used interviews (n = 17; 45.9%), no topic guides were provided, while methods of analysis were not reported in over a third of studies (n = 6; 35.3%).ConclusionUsability of wearable devices is a poorly measured and reported variable in chronic health conditions. Although the heterogeneity in how these devices are implemented implies acceptance, the patient voice should not be assumed. In the absence of being able to make specific usability conclusions, the results of this review instead recommends that future research needs to: (1) Conduct usability assessments as standard, irrespective of the cohort under investigation or the type of study undertaken. (2) Adhere to basic reporting standards (e.g. COREQ) including the basic details of the study. Full copies of any questionnaires and interview guides should be supplied through supplemental files. (3) Utilise mixed methods research to gather a more comprehensive understanding of usability than either qualitative or quantitative research alone will provide. (4) Use previously validated questionnaires alongside any intervention specific measures.