Dataset Information

Improving the User Experience with Discount Site-Specific User Testing.

ABSTRACT:

Objectives

Poor electronic health record (EHR) usability is associated with patient safety concerns, user dissatisfaction, and provider burnout. EHR certification requires vendors to perform user testing. However, there are no such requirements for site-specific implementations. Health care organizations customize EHR implementations, potentially introducing usability problems. Site-specific usability evaluations may help to identify these concerns, and "discount" usability methods afford health systems a means of doing so even without dedicated usability specialists. This report characterizes a site-specific discount user testing program launched at an academic medical center. We describe lessons learned and highlight three of the EHR features in detail to demonstrate the impact of testing on implementation decisions and on users.

Methods

Thirteen new EHR features which had already undergone heuristic evaluation and iterative design were evaluated over the course of three user test events. Each event included five to six users. Participants used think aloud technique. Measures of user efficiency, effectiveness, and satisfaction were collected. Usability concerns were characterized by the type of usability heuristic violated and by correctability.

Results

Usability concerns occurred at a rate of 2.5 per feature tested. Seventy percent of the usability concerns were deemed correctable prior to implementation. The first highlighted feature was moved to production despite low single ease question (SEQ) scores which may have predicted its subsequent withdrawal from production based on post implementation feedback. Another feature was rebuilt based on usability findings, and a new version was retested and moved to production. A third feature highlights an easily correctable usability concern identified in user testing. Quantitative usability metrics generally reinforced qualitative findings.

Conclusion

Simplified user testing with a limited number of participants identifies correctable usability concerns, even after heuristic evaluation. Our discount usability approach to site-specific usability has a role in implementations and may improve the usability of the EHR for the end user.

SUBMITTER: Pierce RP

PROVIDER: S-EPMC9629979 | biostudies-literature | 2022 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Improving the User Experience with Discount Site-Specific User Testing.

Pierce Robert P RP Eskridge Bernie R BR Ross Brandi B Day Margaret A MA Dean Brooke B Belden Jeffery L JL

Applied clinical informatics 20221001 5

<h4>Objectives</h4>Poor electronic health record (EHR) usability is associated with patient safety concerns, user dissatisfaction, and provider burnout. EHR certification requires vendors to perform user testing. However, there are no such requirements for site-specific implementations. Health care organizations customize EHR implementations, potentially introducing usability problems. Site-specific usability evaluations may help to identify these concerns, and "discount" usability methods affor ...[more]

PMID: 36323335

Similar Datasets

Project description:BackgroundGuideline producers are increasingly producing versions of guidelines for the public, and knowledge of what the public want from them is also increasing. The aim of this study was to user test a patient version of a SIGN clinical guideline that was designed based on preliminary work for the DECIDE project. SIGN is the leading national guideline producer in Scotland.MethodsPeople with a diagnosis of glaucoma and non-professional carers were recruited from across Scotland. User testing was conducted using a think-aloud protocol method. Each session was conducted by one interviewer and one observer. All sessions were recorded and transcribed. The data was analysed, problems with the guideline were identified and resolved and key findings were themed using a user experience model.ResultsThirteen user testing sessions with people with glaucoma and one with a carer were conducted. Key facilitators of desirability and usability identified include clear branding as a patient version and a clearly described purpose, audience and contents page. Other facilitators include a "friendly" tone which is achieved by the use of colour, quotes, icons, simple language and charts, and brief chunked text. The value and usefulness of the patient guideline was influenced by its ability to: inform the public, link information to actions, and empower people in their interaction with healthcare professionals. Participants were disappointed by the lack of information on treatment in the patient version, which was outside its scope. Information on the evidence based guideline production process and the involvement of appropriately skilled professionals was key to the credibility of the guideline. Lack of awareness of guidelines and guideline producing bodies, is a potentially serious threat to findability/accessibility.ConclusionsIt is important for guideline producers to maximise the user experience of the public when they access patient versions of guidelines, particularly given the current low level of access and awareness. One size does not fit all and guideline producers need to strike a balance between keeping the patient version simple and providing sufficient information to facilitate shared decision making and empower the public. Guideline producers may find the results of this study useful in designing their own patient versions.

Project description:BackgroundVirtual assistants can be used to deliver innovative health programs that provide appealing, personalized, and convenient health advice and support at scale and low cost. Design characteristics that influence the look and feel of the virtual assistant, such as visual appearance or language features, may significantly influence users' experience and engagement with the assistant.ObjectiveThis scoping review aims to provide an overview of the experimental research examining how design characteristics of virtual health assistants affect user experience, summarize research findings of experimental research examining how design characteristics of virtual health assistants affect user experience, and provide recommendations for the design of virtual health assistants if sufficient evidence exists.MethodsWe searched 5 electronic databases (Web of Science, MEDLINE, Embase, PsycINFO, and ACM Digital Library) to identify the studies that used an experimental design to compare the effects of design characteristics between 2 or more versions of an interactive virtual health assistant on user experience among adults. Data were synthesized descriptively. Health domains, design characteristics, and outcomes were categorized, and descriptive statistics were used to summarize the body of research. Results for each study were categorized as positive, negative, or no effect, and a matrix of the design characteristics and outcome categories was constructed to summarize the findings.ResultsThe database searches identified 6879 articles after the removal of duplicates. We included 48 articles representing 45 unique studies in the review. The most common health domains were mental health and physical activity. Studies most commonly examined design characteristics in the categories of visual design or conversational style and relational behavior and assessed outcomes in the categories of personality, satisfaction, relationship, or use intention. Over half of the design characteristics were examined by only 1 study. Results suggest that empathy and relational behavior and self-disclosure are related to more positive user experience. Results also suggest that if a human-like avatar is used, realistic rendering and medical attire may potentially be related to more positive user experience; however, more research is needed to confirm this.ConclusionsThere is a growing body of scientific evidence examining the impact of virtual health assistants' design characteristics on user experience. Taken together, data suggest that the look and feel of a virtual health assistant does affect user experience. Virtual health assistants that show empathy, display nonverbal relational behaviors, and disclose personal information about themselves achieve better user experience. At present, the evidence base is broad, and the studies are typically small in scale and highly heterogeneous. Further research, particularly using longitudinal research designs with repeated user interactions, is needed to inform the optimal design of virtual health assistants.

Project description:Essential medicine lists (EMLs) are important medicine prioritization tools used by the World Health Organization (WHO) EML and over 130 countries. The criteria used by WHO's Expert Committee on the Selection and Use of Essential Medicines has parallels to the GRADE Evidence-to-Decision (EtD) frameworks. In this study, we explored the EtD frameworks and a visual abstract as adjunctive tools to strengthen the integrate evidence and improve the transparency of decisions of EML applications. We conducted user-experience testing interviews of key EML stakeholders using Morville's honeycomb model. Interviews explored multifaceted dimensions (e.g., usability) on two EML applications for the 2021 WHO EML-long-acting insulin analogues for diabetes and immune checkpoint inhibitors for lung cancer. Using a pre-determined coding framework and thematic analysis we iteratively improved both the EtD framework and the visual abstract. We coded the transcripts of 17 interviews with 13 respondents in 103 locations of the interview texts across all dimensions of the user-experience honeycomb. Respondents felt the EtD framework and visual abstract presented complementary useful and findable adjuncts to the traditional EML application. They felt this would increase transparency and efficiency in evidence assessed by EML committees. As EtD frameworks are also used in health practice guidelines, including those by the WHO, respondents articulated that the adoption of the EtD by EML applications represents a tangible mechanism to align EMLs and guidelines, decrease duplication of work and improve coordination. Improvements were made to clarify instructions for the EtD and visual abstract, and to refine the design and content included. 'Availability' was added as an additional criterion for EML applications to highlight this criterion in alignment with WHO EML criteria. EtD frameworks and visual abstracts present additional important tools to communicate evidence and support decision-criteria in EML applications, which have global health impact. Access to essential medicines is important for achieving universal health coverage, and the development of essential medicine lists should be as evidence-based and trustworthy as possible.

Project description:BackgroundPolygenic scores-which quantify inherited risk by integrating information from many common sites of DNA variation-may enable a tailored approach to clinical medicine. However, alongside considerable enthusiasm, we and others have highlighted a lack of standardized approaches for score disclosure. Here, we review the landscape of polygenic score reporting and describe a generalizable approach for development of a polygenic score disclosure tool for coronary artery disease.MethodsWe assembled a working group of clinicians, geneticists, data visualization specialists, and software developers. The group reviewed existing polygenic score reports and then designed a two-page mock report for coronary artery disease. We then conducted a qualitative user-experience study with this report using an interview guide focused on comprehension, experience, and attitudes. Interviews were transcribed and analyzed for themes identification to inform report revision.ResultsReview of nine existing polygenic score reports from commercial and academic groups demonstrated significant heterogeneity, reinforcing the need for additional efforts to study and standardize score disclosure. Using a newly developed mock score report, we conducted interviews with ten adult individuals (50% females, 70% without prior genetic testing experience, age range 20-70 years) recruited via an online platform. We identified three themes from interviews: (1) visual elements, such as color and simple graphics, enable participants to interpret, relate to, and contextualize their polygenic score, (2) word-based descriptions of risk and polygenic scores presented as percentiles were the best recognized and understood, (3) participants had varying levels of interest in understanding complex genomic information and therefore would benefit from additional resources that can adapt to their individual needs in real time. In response to user feedback, colors used for communicating risk were modified to minimize unintended color associations and odds ratios were removed. All 10 participants expressed interest in receiving a polygenic score report based on their personal genomic information.ConclusionsOur findings describe a generalizable approach to develop a polygenic score report understandable by potential patients. Although additional studies are needed across a wider spectrum of patient populations, these results are likely to inform ongoing efforts related to polygenic score disclosure within clinical practice.

Project description:BackgroundDepression, as one of the most prevalent mental disorders, is expected to become a leading cause of disability. While evidence-based treatments are not always easily accessible, Internet-based information and self-help appears as a promising approach to improve the strained supply situation by avoiding barriers of traditional offline treatment. User experience in the domain of mental problems therefore emerges as an important research topic. The aim of our study is to investigate the impact of depressive symptoms on subjective and objective measures of web user experience.MethodIn this two-part online study (Ntotal = 721) we investigate the relationship between depressive symptoms of web users and basic website characteristics (i.e., content, subjective and objective usability, aesthetics). Participants completed search and memory tasks on different fully-functional websites. In addition, they were asked to evaluate the given websites with standardized measures and were screened for symptoms of depression using the PHQ-9. We used structural equation modeling (SEM) to determine whether depression severity affects users' perception of and performance in using information websites.ResultsWe found significant associations between depressive symptoms and subjective user experience, specifically of website content, usability, and aesthetics, as well as an effect of content perception on the overall appraisal of a website in terms of the intention to visit it again. Small yet significant negative effects of depression severity on all named subjective website evaluations were revealed, leading to an indirect negative effect on the intention to revisit a website via impaired content perceptions. However, objective task performance was not influenced by depressiveness of users.DiscussionDepression emerges as capable of altering the subjective perception of a website to some extend with respect to the main features content, usability, and aesthetics. The user experience of a website is crucial, especially as it facilitates revisiting a website and thus might be relevant in avoiding drop-out in online interventions. Thus, the biased impression of persons affected by symptoms of depression and resulting needs of those users should be considered when designing and evaluating E-(Mental)-Health-platforms. The high prevalence of some mental disorders such as depression in the general population stresses the need for further investigations of the found effects.

Dataset Information

Improving the User Experience with Discount Site-Specific User Testing.

Objectives

Methods

Results

Conclusion

Publications

Improving the User Experience with Discount Site-Specific User Testing.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets