An analysis of usability evaluation practices and contexts of use in wearable robotics.
Ontology highlight
ABSTRACT: Background
User-centered design approaches have gained attention over the past decade, aiming to tackle the technology acceptance issues of wearable robotic devices to assist, support or augment human capabilities. While there is a consensus that usability is key to user-centered design, dedicated usability evaluation studies are scarce and clear evaluation guidelines are missing. However, the careful consideration and integration of user needs appears to be essential to successfully develop an effective, efficient, and satisfactory human-robot interaction. It is primarily the responsibility of the developer, to ensure that this users involvement takes place throughout the design process.Methods
Through an online survey for developers of wearable robotics, we wanted to understand how the design and evaluation in actual daily practice compares to what is reported in literature. With a total of 31 questions, we analyzed the most common wearable robotic device applications and their technology maturity, and how these influence usability evaluation practices.Results
A total of 158 responses from a heterogeneous population were collected and analyzed. The dataset representing contexts of use for augmentation (16.5%), assistance (38.0%), therapy (39.8%), as well as few other specific applications (5.7%), allowed for an insightful analysis of the influence of technology maturity on user involvement and usability evaluation. We identified functionality, ease of use, and performance as the most evaluated usability attributes and could specify which measures are used to assess them. Also, we could underline the frequent use of qualitative measures alongside the expected high prevalence of performance-metrics. In conclusion of the analysis, we derived evaluation recommendations to foster user-centered design and usability evaluation.Conclusion
This analysis might serve as state-of-the-art comparison and recommendation for usability studies in wearable robotics. We believe that by motivating for more balanced, comparable and user-oriented evaluation practices, we may support the wearable robotics field in tackling the technology acceptance limitations.
SUBMITTER: Meyer JT
PROVIDER: S-EPMC8656061 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundWearable devices have the potential to promote a healthy lifestyle because of their real-time data monitoring capabilities. However, device usability is a critical factor that determines whether they will be adopted on a large scale. Usability studies on wearable devices are still scarce.ObjectiveThis study aims to compare the functions and attributes of seven mainstream wearable devices and to evaluate their usability.MethodsThe wearable devices selected were the Apple Watch, Samsung Gear S, Fitbit Surge, Jawbone Up3, Mi Band, Huawei Honor B2, and Misfit Shine. A mixed method of feature comparison and a System Usability Scale (SUS) evaluation based on 388 participants was applied; the higher the SUS score, the better the usability of the product.ResultsFor features, all devices had step counting, an activity timer, and distance recording functions. The Samsung Gear S had a unique sports track recording feature and the Huawei Honor B2 had a unique wireless earphone. The Apple Watch, Samsung Gear S, Jawbone Up3, and Fitbit Surge could measure heart rate. All the devices were able to monitor sleep, except the Apple Watch. For product characteristics, including attributes such as weight, battery life, price, and 22 functions such as step counting, activity time, activity type identification, sleep monitoring, and expandable new features, we found a very weak negative correlation between the SUS scores and price (r=-.10, P=.03) and devices that support expandable new features (r=-.11, P=.02), and a very weak positive correlation between the SUS scores and devices that support the activity type identification function (r=.11, P=.02). The Huawei Honor B2 received the highest score of mean 67.6 (SD 16.1); the lowest Apple Watch score was only 61.4 (SD 14.7). No significant difference was observed among brands. The SUS score had a moderate positive correlation with the user's experience (length of time the device was used) (r=.32, P<.001); participants in the medical and health care industries gave a significantly higher score (mean 61.1, SD 17.9 vs mean 68.7, SD 14.5, P=.03).ConclusionsThe functions of wearable devices tend to be homogeneous and usability is similar across various brands. Overall, Mi Band had the lowest price and the lightest weight. Misfit Shine had the longest battery life and most functions, and participants in the medical and health care industries had the best evaluation of wearable devices. The perceived usability of mainstream wearable devices is unsatisfactory and customer loyalty is not high. A consumer's SUS rating for a wearable device is related to their personal situation instead of the device brand. Device manufacturers should put more effort into developing innovative functions and improving the usability of their products by integrating more cognitive behavior change techniques.
| S-EPMC6250954 | biostudies-literature
Project description:BackgroundThere are many problems with fitness trackers, such as device usability, which limit their large-scale application, and relevant studies are limited in terms of their sample size and evaluation methods. The purpose of the study was to evaluate the perceived usability of various mainstream fitness trackers on the market, and to learn about user feedback on feature preferences for each device.MethodsTrial use of seven mainstream fitness trackers (two smart watches and five smart wristbands) followed by a survey study were applied. The questionnaire was specifically developed for this study, which included two parts (user preferences and device usability in five dimensions). We recruited users to test the devices for at least 30 days and asked experienced users to provide feedback in order to evaluate each device, including the rating and user preference of each device.ResultsWe received 388 valid questionnaires, in which users rated their responses on a five-point Likert scale. (1) User preference: the average user satisfaction was 3.50-3.86 (points), and the rating for willingness to buy averaged between 3.36 and 3.59. More users were willing to wear (58.3-81.3%) and purchase (56.8-83.0%) the devices than were not. The top three general feature preferences were daily activity tracking, heart health monitoring, and professional fitness tracking. The top three health-related feature preferences were heart rate monitoring, daily pedometer, and professional fitness tracking. (2) Usability evaluation: product design was rated from 3.57 to 4.00; durability, 3.63-4.26; ease of use, 3.70-3.90; added features, 3.30-3.83; and user-rated accuracy, 3.44-3.78. A significant difference was observed in the rating of product design and durability among the different devices (p < 0.05) score.ConclusionsUsers generally had positive subjective intent regarding fitness trackers but were less satisfied with their cost effectiveness. The users preferred health related features such as heart health monitoring, and professional fitness tracking. The rating of most of the current mainstream fitness trackers was fair with some significant differences among the devices. Thus, further improvement is needed.
| S-EPMC6064199 | biostudies-literature
Project description:BackgroundDespite technical advances in the field of wearable robotic devices (WRD), there is still limited user acceptance of these technologies. While usability often comes as a key factor influencing acceptance, there is a scattered landscape of definitions and scopes for the term. To advance usability evaluation, and to integrate usability features as design requirements during technology development, there is a need for benchmarks and shared terminology. These should be easily accessible and implementable by developers.MethodsAn initial set of usability attributes (UA) was extracted from a literature survey on usability evaluation in WRD. The initial set of attributes was enriched and locally validated with seven developers of WRD through an online survey and a focus group. The locally validated glossary was then externally validated through a globally distributed online survey.ResultsThe result is the Robotics Usability Glossary (RUG), a comprehensive glossary of 41 UA validated by 70 WRD developers from 17 countries, ensuring its generalizability. 31 of the UA had high agreement scores among respondents and 27 were considered highly relevant in the field, but only 11 of them had been included as design criteria by the respondents.ConclusionsMultiple UA ought to be considered for a comprehensive usability assessment. Usability remains inadequately incorporated into device development, indicating a need for increased awareness and end-user perspective. The RUG can be readily accessed through an online platform, the Interactive Usability Toolbox (IUT), developed to provide context-specific outcome measures and usability evaluation methods. Overall, this effort is an important step towards improving and promoting usability evaluation practices within WRD. It has the potential to pave the way for establishing usability evaluation benchmarks that further endorse the acceptance of WRD.
| S-EPMC10900611 | biostudies-literature
Project description:Stretchable and flexible multifunctional electronic components, including sensors and actuators, have received increasing attention in robotics, electronics, wearable, and healthcare applications. Despite advances, it has remained challenging to design analogs of many electronic components to be highly stretchable, to be efficient to fabricate, and to provide control over electronic performance. Here, we describe highly elastic sensors and interconnects formed from thin, twisted conductive microtubules. These devices consist of twisted assemblies of thin, highly stretchable (>400%) elastomer tubules filled with liquid conductor (eutectic gallium indium, EGaIn), and fabricated using a simple roller coating process. As we demonstrate, these devices can operate as multimodal sensors for strain, rotation, contact force, or contact location. We also show that, through twisting, it is possible to control their mechanical performance and electronic sensitivity. In extensive experiments, we have evaluated the capabilities of these devices, and have prototyped an array of applications in several domains of stretchable and wearable electronics. These devices provide a novel, low cost solution for high performance stretchable electronics with broad applications in industry, healthcare, and consumer electronics, to emerging product categories of high potential economic and societal significance.
| S-EPMC5431990 | biostudies-literature
Project description:Wearable assistive, rehabilitative, and augmentative devices currently require bulky power supplies, often making these tools more of a burden than an asset. This work introduces a soft, low-profile, textile-based pneumatic energy harvesting system that extracts power directly from the foot strike of a user during walking. Energy is harvested with a textile pump integrated into the insole of the user's shoe and stored in a wearable textile bladder to operate pneumatic actuators on demand, with system performance optimized based on a mechano-fluidic model. The system recovered a maximum average power of nearly 3 W with over 20% conversion efficiency-outperforming electromagnetic, piezoelectric, and triboelectric alternatives-and was used to power a wearable arm-lift device that assists shoulder motion and a supernumerary robotic arm, demonstrating its capability as a lightweight, low-cost, and comfortable solution to support adults with upper body functional limitations in activities of daily living.
| S-EPMC9401630 | biostudies-literature
Project description:BackgroundRobotic hand orthoses (RHO) aim to provide grasp assistance for people with sensorimotor hand impairment during daily tasks. Many of such devices have been shown to bring a functional benefit to the user. However, assessing functional benefit is not sufficient to evaluate the usability of such technologies for daily life application. A comprehensive and structured evaluation of device usability not only focusing on effectiveness but also efficiency and satisfaction is required, yet often falls short in existing literature. Mixed methods evaluations, i.e., assessing a combination of quantitative and qualitative measures, allow to obtain a more holistic picture of all relevant aspects of device usability. Considering these aspects already in early development stages allows to identify design issues and generate generalizable benchmarks for future developments.MethodsWe evaluated the short-term usability of the RELab tenoexo, a RHO for hand function assistance, in 15 users with tetraplegia after a spinal cord injury through a comprehensive mixed methods approach. We collected quantitative data using the Action Research Arm Test (ARAT), the System Usability Scale (SUS), and timed tasks such as the donning process. In addition, qualitative data were collected through semi-structured interviews and user observations, and analyzed with a thematic analysis to enhance the usability evaluation. All insights were attributed and discussed in relation to specifically defined usability attributes such as comfort, ease of use, functional benefit, and safety.ResultsThe RELab tenoexo provided an immediate functional benefit to the users, resulting in a mean improvement of the ARAT score by 5.8 points and peaking at 15 points improvement for one user (clinically important difference: 5.7 points). The mean SUS rating of 60.6 represents an adequate usability, however, indicating that especially the RHO donning (average task time = 295 s) was perceived as too long and cumbersome. The participants were generally very satisfied with the ergonomics (size, dimensions, fit) of the RHO. Enhancing the ease of use, specifically in donning, increasing the provided grasping force, as well as the availability of tailoring options and customization were identified as main improvement areas to promote RHO usability.ConclusionThe short-term usability of the RELab tenoexo was thoroughly evaluated with a mixed methods approach, which generated valuable data to improve the RHO in future iterations. In addition, learnings that might be transferable to the evaluation and design of other RHO were generated, which have the potential to increase the daily life applicability and acceptance of similar technologies.
| S-EPMC10693050 | biostudies-literature
Project description:BackgroundThe design complexity of critical care ventilators (CCVs) can lead to use errors and patient harm. In this study, we present the results of a comparison of four CCVs from market leaders, using a rigorous methodology for the evaluation of use safety and user experience of medical devices.MethodsWe carried out a comparative usability study of four CCVs: Hamilton G5, Puritan Bennett 980, Maquet SERVO-U, and Dräger Evita V500. Forty-eight critical care respiratory therapists participated in this fully counterbalanced, repeated measures study. Participants completed seven clinical scenarios composed of 16 tasks on each ventilator. Use safety was measured by percentage of tasks with use errors or close calls (UE/CCs). User experience was measured by system usability and workload metrics, using the Post-Study System Usability Questionnaire (PSSUQ) and the National Aeronautics and Space Administration Task Load Index (NASA-TLX).ResultsNine of 18 post hoc contrasts between pairs of ventilators were significant after Bonferroni correction, with effect sizes between 0.4 and 1.09 (Cohen's d). There were significantly fewer UE/CCs with SERVO-U when compared to G5 (p?=?0.044) and V500 (p?=?0.020). Participants reported higher system usability for G5 when compared to PB980 (p?=?0.035) and higher system usability for SERVO-U when compared to G5 (p?<?0.001), PB980 (p?<?0.001), and V500 (p?<?0.001). Participants reported lower workload for G5 when compared to PB980 (p?<?0.001) and lower workload for SERVO-U when compared to PB980 (p?<?0.001) and V500 (p?<?0.001). G5 scored better on two of nine possible comparisons; SERVO-U scored better on seven of nine possible comparisons. Aspects influencing participants' performance and perception include the low sensitivity of G5's touchscreen and the positive effect from the quality of SERVO-U's user interface design.ConclusionsThis study provides empirical evidence of how four ventilators from market leaders compare and highlights the importance of medical technology design. Within the boundaries of this study, we can infer that SERVO-U demonstrated the highest levels of use safety and user experience, followed by G5. Based on qualitative data, differences in outcomes could be explained by interaction design, quality of hardware components used in manufacturing, and influence of consumer product technology on users' expectations.
| S-EPMC4992292 | biostudies-literature
Project description:ObjectiveThe objective of this study was to systematically evaluate commercially available medication adherence apps for the inclusion of behavior change techniques (BCTs) and to conduct a usability analysis on a subset of apps with adolescents and young adults living with a chronic illness.MethodsMedication adherence apps were identified via a search of iTunes app store in August 2016. Seventy-five apps meeting initial inclusion criteria were independently coded by two researchers for the presence/absence of 26 BCTs. Twenty adolescents and young adults (ages: 13-20 years) with inflammatory bowel disease conducted usability testing on a subset of apps (n = 4).ResultsAcross 75 apps coded for presence/absence of 26 BCTs, only 7 unique BCTs were identified. The number of BCTs per app ranged from 2 to 6, with an average of 3.3 BCTs. In usability testing, quality ratings varied across apps. Medisafe received the highest average scores on engagement, functionality, aesthetics, and information subscales. Medisafe and MyTherapy ranked first and second, respectively, on overall quality and perceived impact ratings.ConclusionContent evaluation revealed only a limited number of BCTs that have been translated to medication adherence apps. Among apps with comparable content, clear user preferences emerged based on perceived quality and usability. Greater collaboration is needed between psychologists and health technologists to incorporate more evidence-based BCTs in apps. Findings also indicate a need for app developers to consider and incorporate the preferences of younger end users to improve app quality and engagement for pediatric populations.
| S-EPMC6415658 | biostudies-literature
Project description:BackgroundJapanese nurses are increasingly required to read published international research in clinical, educational, and research settings. Language barriers are a significant obstacle, and online machine translation (MT) is a tool that can be used to address this issue. We examined the quality of Google Translate® (English to Japanese and Korean to Japanese), which is a representative online MT, using a previously verified evaluation method. We also examined the perceived usability and current use of online MT among Japanese nurses.FindingsRandomly selected nursing abstracts were translated and then evaluated for intelligibility and usability by 28 participants, including assistants and research associates from nursing universities throughout Japan. They answered a questionnaire about their online MT use. From simple comparison of mean scores between two language pairs, translation quality was significantly better, with respect to both intelligibility and usability, for Korean-Japanese than for English-Japanese. Most respondents perceived a language barrier. Online MT had been used by 61% of the respondents and was perceived as not useful enough.ConclusionNursing articles translated from Korean into Japanese by an online MT system could be read at an acceptable level of comprehension, but the same could not be said for English-Japanese translations. Respondents with experience using online MT used it largely to grasp the overall meanings of the original text. Enrichment in technical terms appeared to be the key to better usability. Users will be better able to use MT outputs if they improve their foreign language proficiency as much as possible. Further research is being conducted with a larger sample size and detailed analysis.
| S-EPMC3576310 | biostudies-literature
Project description:BackgroundThe use of wearable biosensor devices for monitoring and coaching in forensic psychiatric settings yields high expectations for improved self-regulation of emotions and behavior in clients and staff members. More so, if clients have mild intellectual disabilities (IQ 50-85), they might benefit from these biosensors as they are easy to use in everyday life, which ensures that clients can practice with the devices in multiple stress and arousal-inducing situations. However, research on (continuous) use and acceptance of biosensors in forensic psychiatry for clients with mild intellectual disabilities and their caretakers is scarce. Although wearable biosensors show promise for health care, recent research showed that the acceptance and continuous use of wearable devices in consumers is not as was anticipated, probably due to low expectations.ObjectiveThe main goal of this study was to investigate the associations between and determinants of the expectation of usability, the actual experienced usability, and the intention for continuous use of biosensors.MethodsA total of 77 participants (31 forensic clients with mild intellectual disabilities and 46 forensic staff members) participated in a 1-week trial. Preceding the study, we selected 4 devices thought to benefit the participants in domains of self-regulation, physical health, or sleep. Qualitative and quantitative questionnaires were used that explored the determinants of usability, acceptance, and continuous use of biosensors. Questionnaires consisted of the System Usability Scale, the Technology Acceptance Model questionnaire, and the extended expectation confirmation model questionnaire.ResultsOnly the experienced usability of the devices was associated with intended continuous use. Forensic clients scored higher on acceptance and intention for continuous use than staff members. Moderate associations were found between usability with acceptance and continuous use. Staff members showed stronger associations between usability and acceptance (r=.80, P<.001) and usability and continuous use (r=.79, P<.001) than clients, who showed more moderate correlations between usability and acceptance (r=.46, P=.01) and usability and continuous use (r=.52, P=.003). The qualitative questionnaires in general indicated that the devices were easy to use and gave clear information.ConclusionsContrary to expectations, it was the actual perceived usability of wearing a biosensor that was associated with continuous use and to a much lesser extent the expectancy of usability. Clients scored higher on acceptance and intention for continuous use, but associations between usability and both acceptance and continuous use were markedly stronger in staff members. This study provides clear directions on how to further investigate these associations. For example, whether this is a true effect or due to a social desirability bias in the client group must be investigated. Clients with mild intellectual disabilities might benefit from the ease of use of these devices and their continuing monitoring and coaching apps. For these clients, it is especially important to develop easy-to-use biosensors with a minimum requirement on cognitive capacity to increase usability, acceptance, and continuous use.
| S-EPMC8145084 | biostudies-literature