Dataset Information

Usability Evaluation of Laboratory Information Systems.

ABSTRACT: Background:Numerous studies have revealed widespread clinician frustration with the usability of electronic health records (EHRs) that is counterproductive to adoption of EHR systems to meet the aims of health-care reform. With poor system usability comes increased risk of negative unintended consequences. Usability issues could lead to user error and workarounds that have the potential to compromise patient safety and negatively impact the quality of care.[1] While there is ample research on EHR usability, there is little information on the usability of laboratory information systems (LISs). Yet, LISs facilitate the timely provision of a great deal of the information needed by physicians to make patient care decisions.[2] Medical and technical advances in genomics that require processing of an increased volume of complex laboratory data further underscore the importance of developing user-friendly LISs. This study aims to add to the body of knowledge on LIS usability. Methods:A survey was distributed among LIS users at hospitals across the United States. The survey consisted of the ten-item System Usability Scale (SUS). In addition, participants were asked to rate the ease of performing 24 common tasks with a LIS. Finally, respondents provided comments on what they liked and disliked about using the LIS to provide diagnostic insight into LIS perceived usability. Results:The overall mean SUS score of 59.7 for the LIS evaluated is significantly lower than the benchmark of 68 (P < 0.001). All LISs evaluated received mean SUS scores below 68 except for Orchard Harvest (78.7). While the years of experience using the LIS was found to be a statistically significant influence on mean SUS scores, the combined effect of years of experience and LIS used did not account for the statistically significant difference in the mean SUS score between Orchard Harvest and each of the other LISs evaluated. Conclusions:The results of this study indicate that overall usability of LISs is poor. Usability lags that of systems evaluated across 446 usability surveys.

SUBMITTER: Mathews A

PROVIDER: S-EPMC5653961 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionElectronic audit and feedback (e-A&F) systems are used worldwide for care quality improvement. They measure health professionals' performance against clinical guidelines, and some systems suggest improvement actions. However, little is known about optimal interface designs for e-A&F, in particular how to present suggested actions for improvement. We developed a novel theory-informed system for primary care (the Performance Improvement plaN GeneratoR; PINGR) that covers the four principal interface components: clinical performance summaries; patient lists; detailed patient-level information; and suggested actions. As far as we are aware, this is the first report of an e-A&F system with all four interface components.Objectives(1) Use a combination of quantitative and qualitative methods to evaluate the usability of PINGR with target end-users; (2) refine existing design recommendations for e-A&F systems; (3) determine the implications of these recommendations for patient safety.MethodsWe recruited seven primary care physicians to perform seven tasks with PINGR, during which we measured on-screen behaviour and eye movements. Participants subsequently completed usability questionnaires, and were interviewed in-depth. Data were integrated to: gain a more complete understanding of usability issues; enhance and explain each other's findings; and triangulate results to increase validity.ResultsParticipants committed a median of 10 errors (range 8-21) when using PINGR's interface, and completed a median of five out of seven tasks (range 4-7). Errors violated six usability heuristics: clear response options; perceptual grouping and data relationships; representational formats; unambiguous description; visually distinct screens for confusable items; and workflow integration. Eye movement analysis revealed the integration of components largely supported effective user workflow, although the modular design of clinical performance summaries unnecessarily increased cognitive load. Interviews and questionnaires revealed PINGR is user-friendly, and that improved information prioritisation could further promote useful user action.ConclusionsComparing our results with the wider usability literature we refine a previously published set of interface design recommendations for e-A&F. The implications for patient safety are significant regarding: user engagement; actionability; and information prioritisation. Our results also support adopting multi-method approaches in usability studies to maximise issue discovery and the credibility of findings.

Project description:BackgroundMedication is the most common intervention in health care, and the number of online consumer information systems within the pharmaceutical sector is increasing. However, online consumer information systems can be a barrier for users, imposing information asymmetries between stakeholders.ObjectiveThe objective of this study was to quantify and compare the usability of an online consumer medication information system (OCMIS) against a reference implementation based on an interoperable information model for patients, physicians, and pharmacists.MethodsQuantitative and qualitative data were acquired from patients, physicians, and pharmacists in this online usability study. We administered 3 use cases and a post hoc questionnaire per user. Quantitative usability data including effectiveness (task success), efficiency (task time), and user satisfaction (system usability scale [SUS]) was complemented by qualitative and demographic data. Users evaluated 6 existing systems and 1 reference implementation of an OCMIS.ResultsA total of 137 patients, 81 physicians, and 68 pharmacists participated in this study. Task success varied from 84% to 92% in patients, 66% to 100% in physicians, and 50% to 91% in pharmacists. Task completion time decreased over the course of the study for all but 2 OCMIS within the patient group. Due to an assumed nonnormal distribution of SUS scores, within-group comparison was done using the Kruskal-Wallis test. Patients showed differences in SUS scores (P=.02) and task time (P=.03), while physicians did not have significant differences in SUS scores (P=.83) and task time (P=.72). For pharmacists, a significant difference in SUS scores (P<.001) and task time (P=.007) was detected.ConclusionsThe vendor-neutral reference implementation based on an interoperable information model was proven to be a promising approach that was not inferior to existing solutions for patients and physicians. For pharmacists, it exceeded user satisfaction scores compared to other OCMIS. This data-driven approach based on an interoperable information model enables the development of more user-tailored features to increase usability. This fosters data democratization and empowers stakeholders within the pharmaceutical sector.

Project description:Objective Few community pharmacies have access to health information exchange (HIE) data. We conducted a first-of-its-kind usability evaluation of an HIE interface prototype (referred to throughout as the "HIE-Pioneer mock-up") developed with pharmacists and pharmacy technicians to aid future implementation in community pharmacies.Methods Community pharmacists and pharmacy technicians were recruited to complete usability evaluations with the HIE-Pioneer mock-up. Each usability evaluation lasted up to 60 minutes. System usability scale (SUS) scores were collected from each participant following each usability evaluation session and summarized with descriptive statistics. Usability evaluation videos were reviewed for common usability attributes, such as the impact of identified usability problems, learnability, and efficiency. Time on task, task success rates, and prototype utilization were also recorded.Results Sixteen total participants completed usability testing across three community pharmacies. The average SUS score was 69.7 (scale 0-100, where 100 is the best), with pharmacists on average reporting higher satisfaction than technicians (74.1 vs. 65.3, respectively). Altogether, we identified 23 distinct usability problems. Key problems identified included needed clarification in tool label names and accessibility of HIE links within the existing workflow. Overall, the usability of the HIE-Pioneer mock-up generally fostered pharmacy professionals' ease of learning and efficiency.Conclusion Our study identified key areas, and potential solutions, to improve the usability of the HIE-Pioneer mock-up. Overall, pharmacy professionals viewed the HIE-Pioneer mock-up positively, with good satisfaction ratings. The HIE-Pioneer mock-up provides a blueprint for future HIE implementation in community pharmacy settings, which would increase community pharmacy teams' access to HIE data nationwide. Community pharmacy access to bi-directional HIE is expected to improve communication among more health care professionals involved in patient care and equip pharmacy professionals with needed information for improved clinical decision-making.

Project description:AIMS AND OBJECTIVES:To evaluate online information on dysmenorrhoea, including readability, credibility, quality and usability. BACKGROUND:Menstrual pain impacts 45%-95% of women of reproductive age globally and is the leading cause of school and work absences among women. Women often seek online information on dysmenorrhoea; however, little is known about the information quality. DESIGN:This was a descriptive study to evaluate online information on dysmenorrhoea. METHODS:We imitated search strategies of the general public. Specifically, we employed the three most popular search engines worldwide-Google, Yahoo and Bing-and used lay search terms, "period pain" and "menstrual cramps." We screened 60 web pages. Following removal of duplicates and irrelevant web pages, 25 met the eligibility criteria. Two team members independently evaluated the included web pages using standardised tools. Readability was evaluated with the Flesch-Kincaid Reading Ease and Flesch-Kincaid Grade formulas; credibility, quality and usability were evaluated with established tools. We followed the STROBE checklist for reporting this study. RESULTS:For readability, the mean Flesch-Kincaid level was 10th grade. For credibility, 8% of web pages referenced scientific literature and 28% stated the author's name and qualifications. For quality, no web page employed user-driven content production; 8% of web pages referenced evidence-based guidelines, 32% of web pages had accurate content, and 4% of web pages recommended shared decision-making. Most web pages were interactive and included nontextual information. Some nontextual information was inaccurate. CONCLUSION:Online information on dysmenorrhoea has generally low readability, mixed credibility and variable quality. RELEVANCE TO CLINICAL PRACTICE:Strategies to improve health information on dysmenorrhoea include avoiding complex terms, incorporating visual aids, presenting evidence-based information and developing a decision aid to support shared decision-making. Healthcare providers should be aware of the problematic health information that individuals are exposed to and provide education about how to navigate online health information.

Project description:BackgroundIncreasingly, older adults and their informal caregivers are using the Internet to search for health-related information. There is a proliferation of health information online, but the quality of this information varies, often based on exaggerated or dramatic findings, and not easily comprehended by consumers. The McMaster Optimal Aging Portal (Portal) was developed to provide Internet users with high-quality evidence about aging and address some of these current limitations of health information posted online. The Portal includes content for health professionals coming from three best-in-class resources (MacPLUS, Health Evidence, and Health Systems Evidence) and four types of content specifically prepared for the general public (Evidence Summaries, Web Resource Ratings, Blog Posts, and Twitter messages).ObjectiveOur objectives were to share the findings of the usability evaluation of the Portal with particular focus on the content features for the general public and to inform designers of health information websites and online resources for older adults about key usability themes.MethodsData analysis included task performance during usability testing and qualitative content analyses of both the usability sessions and interviews to identify core themes.ResultsA total of 37 participants took part in 33 usability testing sessions and 21 focused interviews. Qualitative analysis revealed common themes regarding the Portal's strengths and challenges to usability. The strengths of the website were related to credibility, applicability, browsing function, design, and accessibility. The usability challenges included reluctance to register, process of registering, searching, terminology, and technical features.ConclusionsThe study reinforced the importance of including end users during the development of this unique, dynamic, evidence-based health information website. The feedback was applied to iteratively improve website usability. Our findings can be applied by designers of health-related websites.

Dataset Information

Usability Evaluation of Laboratory Information Systems.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets