Usability of a Co-designed eHealth Prototype for Caregivers: Combination Study of Three Frameworks.
Ontology highlight
ABSTRACT: Background
Co-design (or the participation of users) has shown great potential in the eHealth domain, demonstrating positive results. Nevertheless, the co-design approach cannot guarantee the usability of the system designed, and usability assessment is a complex analysis to perform, as evaluation criteria will differ depending on the usability framework (or set of criteria) used. ISO (International Organization for Standardization) on usability (ISO 9241-210), Nielsen heuristic, and Garrett element of user experience inform different yet complementary aspects of usability.Objective
This study aims to assess the usability and user experience of a co-design prototype by combining 3 complementary frameworks.Methods
To help caregivers provide care for functionally impaired older people, an eHealth tool was co-designed with caregivers, health and social service professionals, and community workers assisting caregivers. The prototype was a website that aims to support the help-seeking process for caregivers (finding resources) and allow service providers to advertise their services (offering resources). We chose an exploratory study method to assess usability in terms of each objective. The first step was to assess users' first impressions of the website. The second was a task scenario with a think-aloud protocol. The final step was a semistructured interview. All steps were performed individually (with a moderator) in a single session. The data were analyzed using 3 frameworks.Results
A total of 10 participants were recruited, 5 for each objective of the website. We were able to identify several usability problems, most of which were located in the information design and interface design dimensions (Garrett framework). Problems in both dimensions were mainly coded as effectiveness and efficiency (ISO framework) and error prevention and match between the systemand the real world (Nielsen heuristic).Conclusions
Our study provided a novel contribution about usability analysis by combining the 3 different models to classify the problems found. This combination provided a holistic understanding of the usability improvements needed. It can also be used to analyze other eHealth products.International registered report identifier (irrid)
RR2-10.2196/11634.
SUBMITTER: Tremblay M
PROVIDER: S-EPMC8411328 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Digital games aimed at improving cognitive and/or motor-sensory skills need to be carefully designed to take into account the characteristics and needs of particular categories of users. Several novel mini-games explicitly aimed at children with visual impairment (VI) were co-designed by a multidisciplinary team which involved computer engineers and a therapy team from the Robert Hollman Foundation (Padova, Italy). These games are played by children moving within a large-scale interactive environment - i.e., a floor portion placed under a motion capture system capable of tracking one or more people - with the game linking the players movements to the audio and visual output to produce meaningful interactions. We report on a pilot study of the usability of the system involving 11 children with VI. The results allowed us to improve the system and to define a set of guidelines useful for designers and developers of similar systems.Supplementary informationThe online version contains supplementary material available at 10.1007/s11042-022-13665-7.
| S-EPMC9461408 | biostudies-literature
Project description:BackgroundHarm from medications is a major patient safety challenge. Most adverse drug events arise when a medication is prescribed or reevaluated. Therefore, interventions in this area may improve patient safety. A medication plan, that is, a plan for continued treatment with medications, may support patient safety. Participation of patients in the design of health care products or services may improve patient safety. Co-design, as in the Double Diamond framework from the Design Council, England, can emphasize patient involvement. As the COVID-19 pandemic brought restrictions to face-to-face co-design approaches, interest in remote approaches increased. However, it is uncertain how best to perform remote co-design. Therefore, we explored a remote approach, which brought together older persons and health care professionals to co-design a medication plan prototype in the electronic health record, aiming to support patient safety.ObjectiveThis study aimed to describe how remote co-design was applied to create a medication plan prototype and to explore participants' experiences with this approach.MethodsWithin a case study design, we explored the experiences of a remote co-design initiative with 14 participants in a regional health care system in southern Sweden. Using descriptive statistics, quantitative data from questionnaires and web-based workshop timestamps were analyzed. A thematic analysis of the qualitative data gathered from workshops, interviews, and free-text responses to the survey questions was performed. Qualitative and quantitative data were compared side by side in the discussion.ResultsThe analysis of the questionnaires revealed that the participants rated the experiences of the co-design initiative very high. In addition, the balance between how much involved persons expressed their wishes and were listened to was considered very good. Marked timestamps from audio recordings showed that the workshops proceeded according to the plan. The thematic analysis yielded the following main themes: Everyone's perspective matters, Learning by sharing, and Mastering a digital space. The themes encompassed what helped to establish a permissive environment that allowed the participants to be involved and share viewpoints. There was a dynamic process of learning and understanding, realizing that despite different backgrounds, there was consensus about the requirements for a medication plan. The remote co-design process seemed appealing, by balancing opportunities and challenges and building an inviting, creative, and tolerant environment.ConclusionsParticipants experienced that the remote co-design initiative was inclusive of their perspectives and facilitated learning by sharing experiences. The Double Diamond framework was applicable in a digital context and supported the co-design process of the medication plan prototype. Remote co-design is still novel, but with attentiveness to power relations between all involved, this approach may increase opportunities for older persons and health care professionals to collaboratively design products or services that can improve patient safety.
| S-EPMC10131987 | biostudies-literature
Project description:INTRODUCTION:Reimbursement decisions on orphan drugs carry significant uncertainty, and as the amount increases, so does the risk of making a wrong decision, where harms outweigh benefits. Consequently, patients often face limited access to orphan drugs. Managed access programmes (MAPs) are a mechanism for managing risk while enabling access to potentially beneficial drugs. Patients and their caregivers have expressed support for these programmes and see patient input as critical to successful implementation. However, they have yet to be systematically involved in their design. OBJECTIVE:The aim of this study was to co-design with patients and caregivers a tool for the development of managed access programmes. METHODS:Building upon established relationships with the Canadian Organization for Rare Disorders, the project team collaborated with patients and caregivers using the principles of participatory action research. Data were collected at two workshops and analysed using a thematic network approach. RESULTS:Patients and caregivers co-designed a checklist comprised of six aspects of an ideal MAP relating to accountability (programme goals); governance (MAP-specific committee oversight, patient input, international collaboration); and evidence collection (outcome measures and continuation criteria, on-going monitoring and registries). They recognized that health-care resources are finite and considered disease or drug eligibility criteria for deciding when to use a MAP (eg drugs treating diseases for which there are no other legitimate alternatives). CONCLUSIONS:A patient and caregiver-designed checklist was created, which emphasized patient involvement and transparency. Further research is needed to examine the feasibility of this checklist and roles for other stakeholders.
| S-EPMC6250858 | biostudies-other
Project description:BACKGROUND:Informal caregivers are family members or close friends who provide unpaid help to individuals with acute or chronic health conditions so that they can manage daily life tasks. The greatest source of health information is the internet for meeting the needs of caregivers. However, information on the internet may not be scientifically valid, it may be written in language that is difficult to read, and is often in very large doses. 90Second Caregiver is a health letter whose aim is to disseminate knowledge to caregivers in a user-friendly, weekly format, in order to improve their wellbeing. OBJECTIVE:The main objective was to test a sample of 90Second Caregiver health letters in order to assess their usability and to optimize the design and content of the health letters. METHODS:Usability research themes were assessed using semi-structured phone interviews, incorporating the Think Aloud method with retrospective questioning. RESULTS:Usability was assessed in the context of five main themes: understandability and learnability, completeness, relevance, and quality and credibility of the health letter content, as well as design and format. Caregivers generally provided positive feedback regarding the usability of the letters. The usability feedback was used to refine 90Second Caregiver in order to improve the design and content of the series. Based on the results of this study, it may be of maximum benefit to target the series towards individuals who are new to caregiving or part-time caregivers, given that these caregivers of the sample found the letters more useful and relevant and had the most positive usability experiences. CONCLUSIONS:The findings assisted in the improvement of the 90Second Caregiver template, which will be used to create future health letters and refine the letters that have already been created. The findings have implications for who the 90Second Caregiver series should be targeting (ie, newer or part-time caregivers) in order to be maximally impactful in improving mental health and wellbeing-related outcomes for caregivers, such as self-efficacy and caregiving knowledge. The results of this study may be generalizable to the examination of other electronic health information formats, making them valuable to future researchers testing the usability of health information products. In addition, the methods used in this study are useful for usability hypothesis generation. Lastly, our 90Second delivery approach can generate information useful for a set of similar products (eg, weekly health letters targeted towards other conditions/populations).
| S-EPMC7055838 | biostudies-literature
Project description:Purpose: This study investigates the usability of a stepping exergame in older adults with major neurocognitive disorder (MNCD) residing in a long-term care facility. Materials and Methods: A mixed methods study was conducted. Participants played exergames for 30 min on one try-out session. During the exergames, the think aloud method was used, and field notes were taken by the facilitator. Following the exergames, participants completed the System Usability Scale (SUS) and a semi-structured in-depth interview about usability including their personal experiences. Audio files were transcribed and a thematic content analysis of the think aloud data, field notes and interviews were performed using NVivo 12. Results: Twenty-two participants with MNCD were included [mean age = 84.3 ± 5.5 (70-95) years; 81.8% women; Short Physical Performance Battery score = 7.5 ± 3.2 (1-12), Montreal Cognitive Assessment score = 11.9 ± 4.4 (2-19)]. System usability was rated "ok to good" with a mean SUS score of 57.8 (SD = 12.3) with scores ranging from 37.5 to 90.0. Five main themes emerged from the thematic content analysis: (1) perceived user friendliness and acceptability of the exergames; (2) interactional experience; (3) motivational factors; (4) training modalities; and (5) risks. There were no adverse events nor dropouts. Conclusion: Participants evaluated the usability of the exergames positively. The results indicate that the stepping exergame is usable in older adults with MNCD.
| S-EPMC8600328 | biostudies-literature
Project description:BackgroundMobile eHealth apps have been used as a complementary treatment to increase the quality of life of patients and provide new opportunities for the management of rheumatic diseases. Telemedicine, particularly in the areas of prevention, diagnostics, and therapy, has become an essential cornerstone in the care of patients with rheumatic diseases.ObjectiveThis study aims to improve the design and technology of YogiTherapy and evaluate its usability and quality.MethodsWe newly implemented the mobile eHealth app YogiTherapy with a modern design, the option to change language, and easy navigation to improve the app's usability and quality for patients. After refinement, we evaluated the app by conducting a study with 16 patients with AS (4 female and 12 male; mean age 48.1, SD 16.8 y). We assessed the usability of YogiTherapy with a task performance test (TPT) with a think-aloud protocol and the quality with the German version of the Mobile App Rating Scale (MARS).ResultsIn the TPT, the participants had to solve 6 tasks that should be performed on the app. The overall task completion rate in the TPT was high (84/96, 88% completed tasks). Filtering for videos and navigating to perform an assessment test caused the largest issues during the TPT, while registering in the app and watching a yoga video were highly intuitive. Additionally, 12 (75%) of the 16 participants completed the German version of MARS. The quality of YogiTherapy was rated with an average MARS score of 3.79 (SD 0.51) from a maximum score of 5. Furthermore, results from the MARS questionnaire demonstrated a positive evaluation regarding functionality and aesthetics.ConclusionsThe refined and tested YogiTherapy app showed promising results among most participants. In the future, the app could serve its function as a complementary treatment for patients with AS. For this purpose, surveys with a larger number of patients should still be conducted. As a substantial advancement, we made the app free and openly available on the iOS App and Google Play stores.
| S-EPMC10751630 | biostudies-literature
Project description:Objective Few community pharmacies have access to health information exchange (HIE) data. We conducted a first-of-its-kind usability evaluation of an HIE interface prototype (referred to throughout as the "HIE-Pioneer mock-up") developed with pharmacists and pharmacy technicians to aid future implementation in community pharmacies.Methods Community pharmacists and pharmacy technicians were recruited to complete usability evaluations with the HIE-Pioneer mock-up. Each usability evaluation lasted up to 60 minutes. System usability scale (SUS) scores were collected from each participant following each usability evaluation session and summarized with descriptive statistics. Usability evaluation videos were reviewed for common usability attributes, such as the impact of identified usability problems, learnability, and efficiency. Time on task, task success rates, and prototype utilization were also recorded.Results Sixteen total participants completed usability testing across three community pharmacies. The average SUS score was 69.7 (scale 0-100, where 100 is the best), with pharmacists on average reporting higher satisfaction than technicians (74.1 vs. 65.3, respectively). Altogether, we identified 23 distinct usability problems. Key problems identified included needed clarification in tool label names and accessibility of HIE links within the existing workflow. Overall, the usability of the HIE-Pioneer mock-up generally fostered pharmacy professionals' ease of learning and efficiency.Conclusion Our study identified key areas, and potential solutions, to improve the usability of the HIE-Pioneer mock-up. Overall, pharmacy professionals viewed the HIE-Pioneer mock-up positively, with good satisfaction ratings. The HIE-Pioneer mock-up provides a blueprint for future HIE implementation in community pharmacy settings, which would increase community pharmacy teams' access to HIE data nationwide. Community pharmacy access to bi-directional HIE is expected to improve communication among more health care professionals involved in patient care and equip pharmacy professionals with needed information for improved clinical decision-making.
| S-EPMC10567139 | biostudies-literature
Project description:BackgroundeHealth has the potential to improve outcomes such as physical activity or balance in older adults receiving geriatric rehabilitation. However, several challenges such as scarce evidence on effectiveness, feasibility, and usability hinder the successful implementation of eHealth in geriatric rehabilitation.ObjectiveThe aim of this systematic review was to assess evidence on the effectiveness, feasibility, and usability of eHealth interventions in older adults in geriatric rehabilitation.MethodsWe searched 7 databases for randomized controlled trials, nonrandomized studies, quantitative descriptive studies, qualitative research, and mixed methods studies that applied eHealth interventions during geriatric rehabilitation. Included studies investigated a combination of effectiveness, usability, and feasibility of eHealth in older patients who received geriatric rehabilitation, with a mean age of ≥70 years. Quality was assessed using the Mixed Methods Appraisal Tool and a narrative synthesis was conducted using a harvest plot.ResultsIn total, 40 studies were selected, with clinical heterogeneity across studies. Of 40 studies, 15 studies (38%) found eHealth was at least as effective as non-eHealth interventions (56% of the 27 studies with a control group), 11 studies (41%) found eHealth interventions were more effective than non-eHealth interventions, and 1 study (4%) reported beneficial outcomes in favor of the non-eHealth interventions. Of 17 studies, 16 (94%) concluded that eHealth was feasible. However, high exclusion rates were reported in 7 studies of 40 (18%). Of 40 studies, 4 (10%) included outcomes related to usability and indicated that there were certain aging-related barriers to cognitive ability, physical ability, or perception, which led to difficulties in using eHealth.ConclusionseHealth can potentially improve rehabilitation outcomes for older patients receiving geriatric rehabilitation. Simple eHealth interventions were more likely to be feasible for older patients receiving geriatric rehabilitation, especially, in combination with another non-eHealth intervention. However, a lack of evidence on usability might hamper the implementation of eHealth. eHealth applications in geriatric rehabilitation show promise, but more research is required, including research with a focus on usability and participation.
| S-EPMC8414304 | biostudies-literature
Project description:BackgroundThe use of patient health and treatment information captured in structured and unstructured formats in computerized electronic health record (EHR) repositories could potentially augment the detection of safety signals for drug products regulated by the US Food and Drug Administration (FDA). Natural language processing and other artificial intelligence (AI) techniques provide novel methodologies that could be leveraged to extract clinically useful information from EHR resources.ObjectiveOur aim is to develop a novel AI-enabled software prototype to identify adverse drug event (ADE) safety signals from free-text discharge summaries in EHRs to enhance opioid drug safety and research activities at the FDA.MethodsWe developed a prototype for web-based software that leverages keyword and trigger-phrase searching with rule-based algorithms and deep learning to extract candidate ADEs for specific opioid drugs from discharge summaries in the Medical Information Mart for Intensive Care III (MIMIC III) database. The prototype uses MedSpacy components to identify relevant sections of discharge summaries and a pretrained natural language processing (NLP) model, Spark NLP for Healthcare, for named entity recognition. Fifteen FDA staff members provided feedback on the prototype's features and functionalities.ResultsUsing the prototype, we were able to identify known, labeled, opioid-related adverse drug reactions from text in EHRs. The AI-enabled model achieved accuracy, recall, precision, and F1-scores of 0.66, 0.69, 0.64, and 0.67, respectively. FDA participants assessed the prototype as highly desirable in user satisfaction, visualizations, and in the potential to support drug safety signal detection for opioid drugs from EHR data while saving time and manual effort. Actionable design recommendations included (1) enlarging the tabs and visualizations; (2) enabling more flexibility and customizations to fit end users' individual needs; (3) providing additional instructional resources; (4) adding multiple graph export functionality; and (5) adding project summaries.ConclusionsThe novel prototype uses innovative AI-based techniques to automate searching for, extracting, and analyzing clinically useful information captured in unstructured text in EHRs. It increases efficiency in harnessing real-world data for opioid drug safety and increases the usability of the data to support regulatory review while decreasing the manual research burden.
| S-EPMC10538589 | biostudies-literature
Project description:BackgroundSupporting patients to self-manage their chronic kidney disease (CKD) has been identified as a research priority by patients with CKD and those who care for them. Self-management has been shown to slow CKD progression and improve the quality of life of individuals living with the disease. Previous work has identified a need for a person-centered, theory-informed, web-based tool for CKD self-management that can be individualized to a patient's unique situation, priorities, and preferences. We addressed this gap using an integrated knowledge translation method and patient engagement principles.ObjectiveThe aim of this study is to conduct systematic co-design and usability testing of a web-based self-management prototype for adults with CKD (nondialysis and nontransplant) and their caregivers to enhance self-management support.MethodsA multistep, iterative system development cycle was used to co-design and test the My Kidneys My Health prototype. The 3-step process included creating website features and content using 2 sequential focus groups with patients with CKD and caregivers, heuristic testing using the 10 heuristic principles by Nielsen, and usability testing through in-person 60-minute interviews with patients with CKD and their caregivers. Patients with CKD, caregivers, clinicians, researchers, software developers, graphic designers, and policy makers were involved in all steps of this study.ResultsIn step 1, 18 participants (14 patients and 4 caregivers) attended one of the 2 sequential focus groups. The participants provided specific suggestions for simplifying navigation as well as suggestions to incorporate video, text, audio, interactive components, and visuals to convey information. A total of 5 reviewers completed the heuristic analysis (step 2), identifying items mainly related to navigation and functionality. Furthermore, 5 participants completed usability testing (step 3) and provided feedback on video production, navigation, features and functionality, and branding. Participants reported visiting the website repeatedly for the following features: personalized food tool, my health care provider question list, symptom guidance based on CKD severity, and medication advice. Usability was high, with a mean system usability score of 90 out of 100.ConclusionsThe My Kidneys My Health prototype is a systematically developed, multifaceted, web-based CKD self-management support tool guided by the theory and preferences of patients with CKD and their caregivers. The website is user friendly and provides features that improve user experience by tailoring the content and resources to their needs. A feasibility study will provide insights into the acceptability of and engagement with the prototype and identify preliminary patient-reported outcomes (eg, self-efficacy) as well as potential factors related to implementation. This work is relevant given the shift to virtual care during the current pandemic times and provides patients with support when in-person care is restricted.
| S-EPMC7902181 | biostudies-literature