Dataset Information

The usability of ventilators: a comparative evaluation of use safety and user experience.

ABSTRACT:

Background

The design complexity of critical care ventilators (CCVs) can lead to use errors and patient harm. In this study, we present the results of a comparison of four CCVs from market leaders, using a rigorous methodology for the evaluation of use safety and user experience of medical devices.

Methods

We carried out a comparative usability study of four CCVs: Hamilton G5, Puritan Bennett 980, Maquet SERVO-U, and Dräger Evita V500. Forty-eight critical care respiratory therapists participated in this fully counterbalanced, repeated measures study. Participants completed seven clinical scenarios composed of 16 tasks on each ventilator. Use safety was measured by percentage of tasks with use errors or close calls (UE/CCs). User experience was measured by system usability and workload metrics, using the Post-Study System Usability Questionnaire (PSSUQ) and the National Aeronautics and Space Administration Task Load Index (NASA-TLX).

Results

Nine of 18 post hoc contrasts between pairs of ventilators were significant after Bonferroni correction, with effect sizes between 0.4 and 1.09 (Cohen's d). There were significantly fewer UE/CCs with SERVO-U when compared to G5 (p?=?0.044) and V500 (p?=?0.020). Participants reported higher system usability for G5 when compared to PB980 (p?=?0.035) and higher system usability for SERVO-U when compared to G5 (p?ConclusionsThis study provides empirical evidence of how four ventilators from market leaders compare and highlights the importance of medical technology design. Within the boundaries of this study, we can infer that SERVO-U demonstrated the highest levels of use safety and user experience, followed by G5. Based on qualitative data, differences in outcomes could be explained by interaction design, quality of hardware components used in manufacturing, and influence of consumer product technology on users' expectations.

SUBMITTER: Morita PP

PROVIDER: S-EPMC4992292 | biostudies-literature | 2016 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The usability of ventilators: a comparative evaluation of use safety and user experience.

Morita Plinio P PP Weinstein Peter B PB Flewwelling Christopher J CJ Bañez Carleene A CA Chiu Tabitha A TA Iannuzzi Mario M Patel Aastha H AH Shier Ashleigh P AP Cafazzo Joseph A JA

Critical care (London, England) 20160820

<h4>Background</h4>The design complexity of critical care ventilators (CCVs) can lead to use errors and patient harm. In this study, we present the results of a comparison of four CCVs from market leaders, using a rigorous methodology for the evaluation of use safety and user experience of medical devices.<h4>Methods</h4>We carried out a comparative usability study of four CCVs: Hamilton G5, Puritan Bennett 980, Maquet SERVO-U, and Dräger Evita V500. Forty-eight critical care respiratory therapi ...[more]

PMID: 27542352

Similar Datasets

Project description:BackgroundMany essential walking activities in daily life, such as crossing a street, are challenging to practice in conventional therapeutic settings. Virtual environments (VEs) delivered through a virtual reality (VR) head-mounted display (HMD) would allow training such activities in a safe and attractive environment. Furthermore, the game-like character and high degree of immersion in these applications might help maintain or increase children's motivation and active participation during the rehabilitation process.ObjectiveThis study aimed to investigate the usability, user experience, and acceptability of an immersive VE experienced through a VR HMD to train everyday life walking activities in pediatric neurorehabilitation.MethodsIn a cross-sectional study, 21 youths (median age 12.1 years; range 6.8-17.7 years) with a neuromotor impairment undergoing inpatient or outpatient neurorehabilitation tested a VE experienced through the VR HMD Oculus Quest. The participants, accompanied by their physiotherapists, moved freely around a 4.4 by 10-meter VE, displaying a magical forest and featuring various gamified everyday activities in different game designs. Using their hands, represented in the VE, the participants could interact with the virtual objects placed throughout the VE and trigger visual and auditory feedback. Symptoms of cybersickness were checked, and usability, user experience, and acceptability were evaluated using customized questionnaires with a visual analog scale for youths and a 5-point Likert scale for their therapists.ResultsNone of the participants reported any signs of cybersickness after 20 minutes of VR HMD exposure time. They rated comfort (median 10/10) and movement ability (median 10/10) with the VR HMD as high. The VE was perceived as being really there by the majority (median 8/10), and the participants had a strong feeling of spatial presence in the VE (median 9.5/10). They enjoyed exploring the virtual world (median 10/10) and liked this new therapy approach (median 10/10). Therapists' acceptance of the VR HMD was high (4/5). There were 5 patients that needed more support than usual, mainly for supervision, when moving around with the VR HMD. Otherwise, therapists felt that the VR HMD hardly affected their patients' movement behavior (median 4.75/5), whereas it seemed to increase their level of therapy engagement (median 4/5) compared to conventional physiotherapy sessions.ConclusionsThis study demonstrates the usability of an immersive VE delivered through a VR HMD to engage youths in the training of everyday walking activities. The participants' and therapists' positive ratings on user experience and acceptance further support the promising application of this technology as a future therapeutic tool in pediatric neurorehabilitation.

Project description:IntroductionElectronic audit and feedback (e-A&F) systems are used worldwide for care quality improvement. They measure health professionals' performance against clinical guidelines, and some systems suggest improvement actions. However, little is known about optimal interface designs for e-A&F, in particular how to present suggested actions for improvement. We developed a novel theory-informed system for primary care (the Performance Improvement plaN GeneratoR; PINGR) that covers the four principal interface components: clinical performance summaries; patient lists; detailed patient-level information; and suggested actions. As far as we are aware, this is the first report of an e-A&F system with all four interface components.Objectives(1) Use a combination of quantitative and qualitative methods to evaluate the usability of PINGR with target end-users; (2) refine existing design recommendations for e-A&F systems; (3) determine the implications of these recommendations for patient safety.MethodsWe recruited seven primary care physicians to perform seven tasks with PINGR, during which we measured on-screen behaviour and eye movements. Participants subsequently completed usability questionnaires, and were interviewed in-depth. Data were integrated to: gain a more complete understanding of usability issues; enhance and explain each other's findings; and triangulate results to increase validity.ResultsParticipants committed a median of 10 errors (range 8-21) when using PINGR's interface, and completed a median of five out of seven tasks (range 4-7). Errors violated six usability heuristics: clear response options; perceptual grouping and data relationships; representational formats; unambiguous description; visually distinct screens for confusable items; and workflow integration. Eye movement analysis revealed the integration of components largely supported effective user workflow, although the modular design of clinical performance summaries unnecessarily increased cognitive load. Interviews and questionnaires revealed PINGR is user-friendly, and that improved information prioritisation could further promote useful user action.ConclusionsComparing our results with the wider usability literature we refine a previously published set of interface design recommendations for e-A&F. The implications for patient safety are significant regarding: user engagement; actionability; and information prioritisation. Our results also support adopting multi-method approaches in usability studies to maximise issue discovery and the credibility of findings.

Project description:ImportanceDespite the broad adoption and optimization of electronic health record (EHR) systems across the continuum of care, serious usability and safety problems persist.ObjectiveTo assess whether EHR safety performance is associated with EHR frontline user experience in a national sample of hospitals.Design, setting, and participantsThis cross-sectional study included all US adult hospitals that used the National Quality Forum Leapfrog Health IT Safety Measure and also used the ARCH Collaborative EHR User experience survey from January 1, 2017, to January 1, 2019. Data analysis was performed from September 2020 to November 2022.Main outcomes and measuresThe primary outcomes were hospital performance on the Leapfrog Health IT Safety measure (overall and 10 subcomponents) and the ARCH collaborative frontline user experience scores (overall and 8 subcomponents). Ordinary least squares models with survey responses clustered by hospital were used to assess associations between the overall measures and their subcomponents.ResultsThere were 112 hospitals and 5689 frontline user surveys included in the study. Hospitals scored a mean of 0.673 (range, 0.297-0.973) on the Leapfrog Health IT safety measure; the mean ARCH EHR user experience score was 3.377 (range, 1 [best] to 5 [worst]). The adjusted β coefficient between the overall safety score and overall user experience score was 0.011 (95% CI, 0.006-0.016). The ARCH overall score was also significantly associated with 10 subcategory scores of the Leapfrog Health IT safety score, and the overall Leapfrog score was associated with the 8 subcategory scores of the ARCH user experience score.Conclusions and relevanceThis cross-sectional study found a positive association between frontline user-rated EHR usability and EHR safety performance. This finding suggests that improving EHR usability, which is a current well-known pain point for EHR users, could have direct benefits in terms of improved EHR safety.

Project description:BACKGROUND:Registration of brain activity has become increasingly popular and offers a way to identify the mental state of the user, prevent inappropriate workload, and control other devices by means of brain-computer interfaces. However, electroencephalography (EEG) is often related to user acceptance issues regarding the measuring technique. Meanwhile, emerging mobile EEG technology offers the possibility of gel-free signal acquisition and wireless signal transmission. Nonetheless, user experience research about the new devices is lacking. OBJECTIVE:This study aimed to evaluate user experience aspects of emerging mobile EEG devices and, in particular, to investigate wearing comfort and issues related to emotional design. METHODS:We considered 7 mobile EEG devices and compared them for their wearing comfort, type of electrodes, visual appearance, and subjects' preference for daily use. A total of 24 subjects participated in our study and tested every device independently of the others. The devices were selected in a randomized order and worn on consecutive day sessions of 60-min duration. At the end of each session, subjects rated the devices by means of questionnaires. RESULTS:Results indicated a highly significant change in maximal possible wearing duration among the EEG devices (χ26=40.2, n=24; P<.001). Regarding the visual perception of devices' headset design, results indicated a significant change in the subjects' ratings (χ26=78.7, n=24; P<.001). Results of the subjects' ratings regarding the practicability of the devices indicated highly significant differences among the EEG devices (χ26=83.2, n=24; P<.001). Ranking order and posthoc tests offered more insight and indicated that pin electrodes had the lowest wearing comfort, in particular, when coupled with a rigid, heavy headset. Finally, multiple linear regression for each device separately revealed that users were not willing to accept less comfort for a more attractive headset design. CONCLUSIONS:The study offers a differentiated look at emerging mobile and gel-free EEG technology and the relation between user experience aspects and device preference. Our research could be seen as a precondition for the development of usable applications with wearables and contributes to consumer health informatics and health-enabling technologies. Furthermore, our results provided guidance for the technological development direction of new EEG devices related to the aspects of emotional design.

Project description:BackgroundThere are many problems with fitness trackers, such as device usability, which limit their large-scale application, and relevant studies are limited in terms of their sample size and evaluation methods. The purpose of the study was to evaluate the perceived usability of various mainstream fitness trackers on the market, and to learn about user feedback on feature preferences for each device.MethodsTrial use of seven mainstream fitness trackers (two smart watches and five smart wristbands) followed by a survey study were applied. The questionnaire was specifically developed for this study, which included two parts (user preferences and device usability in five dimensions). We recruited users to test the devices for at least 30 days and asked experienced users to provide feedback in order to evaluate each device, including the rating and user preference of each device.ResultsWe received 388 valid questionnaires, in which users rated their responses on a five-point Likert scale. (1) User preference: the average user satisfaction was 3.50-3.86 (points), and the rating for willingness to buy averaged between 3.36 and 3.59. More users were willing to wear (58.3-81.3%) and purchase (56.8-83.0%) the devices than were not. The top three general feature preferences were daily activity tracking, heart health monitoring, and professional fitness tracking. The top three health-related feature preferences were heart rate monitoring, daily pedometer, and professional fitness tracking. (2) Usability evaluation: product design was rated from 3.57 to 4.00; durability, 3.63-4.26; ease of use, 3.70-3.90; added features, 3.30-3.83; and user-rated accuracy, 3.44-3.78. A significant difference was observed in the rating of product design and durability among the different devices (p < 0.05) score.ConclusionsUsers generally had positive subjective intent regarding fitness trackers but were less satisfied with their cost effectiveness. The users preferred health related features such as heart health monitoring, and professional fitness tracking. The rating of most of the current mainstream fitness trackers was fair with some significant differences among the devices. Thus, further improvement is needed.

Project description:BackgroundSerious games (SGs) are used as complementary approaches to stimulate patients with dementia. However, many of the SGs use out-of-the-shelf technologies that may not always be suitable for such populations, as they can lead to negative behaviors, such as anxiety, fatigue, and even cybersickness.ObjectiveThis study aims to evaluate how patients with dementia interact and accept 5 out-of-the-shelf technologies while completing 10 virtual reality tasks.MethodsA total of 12 participants diagnosed with dementia (mean age 75.08 [SD 8.07] years, mean Mini-Mental State Examination score 17.33 [SD 5.79], and mean schooling 5.55 [SD 3.30]) at a health care center in Portugal were invited to participate in this study. A within-subject experimental design was used to allow all participants to interact with all technologies, such as HTC VIVE, head-mounted display (HMD), tablet, mouse, augmented reality (AR), leap motion (LM), and a combination of HMD with LM. Participants' performance was quantified through behavioral and verbal responses, which were captured through video recordings and written notes.ResultsThe findings of this study revealed that the user experience using technology was dependent on the patient profile; the patients had a better user experience when they use technologies with direct interaction configuration as opposed to indirect interaction configuration in terms of assistance required (P=.01) and comprehension (P=.01); the participants did not trigger any emotional responses when using any of the technologies; the participants' performance was task-dependent; the most cost-effective technology was the mouse, whereas the least cost-effective was AR; and all the technologies, except for one (HMD with LM), were not exposed to external hazards.ConclusionsMost participants were able to perform tasks using out-of-the-shelf technologies. However, there is no perfect technology, as they are not explicitly designed to address the needs and skills of people with dementia. Here, we propose a set of guidelines that aim to help health professionals and engineers maximize user experience when using such technologies for the population with dementia.

Dataset Information

The usability of ventilators: a comparative evaluation of use safety and user experience.

Background

Methods

Results

Publications

The usability of ventilators: a comparative evaluation of use safety and user experience.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets