A Methodology to Compare the Usability of Information Systems
Ontology highlight
ABSTRACT:
SUBMITTER: Hattingh M
PROVIDER: S-EPMC7134260 | biostudies-literature | 2020 Mar
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundNumerous studies have revealed widespread clinician frustration with the usability of electronic health records (EHRs) that is counterproductive to adoption of EHR systems to meet the aims of health-care reform. With poor system usability comes increased risk of negative unintended consequences. Usability issues could lead to user error and workarounds that have the potential to compromise patient safety and negatively impact the quality of care.[1] While there is ample research on EHR usability, there is little information on the usability of laboratory information systems (LISs). Yet, LISs facilitate the timely provision of a great deal of the information needed by physicians to make patient care decisions.[2] Medical and technical advances in genomics that require processing of an increased volume of complex laboratory data further underscore the importance of developing user-friendly LISs. This study aims to add to the body of knowledge on LIS usability.MethodsA survey was distributed among LIS users at hospitals across the United States. The survey consisted of the ten-item System Usability Scale (SUS). In addition, participants were asked to rate the ease of performing 24 common tasks with a LIS. Finally, respondents provided comments on what they liked and disliked about using the LIS to provide diagnostic insight into LIS perceived usability.ResultsThe overall mean SUS score of 59.7 for the LIS evaluated is significantly lower than the benchmark of 68 (P < 0.001). All LISs evaluated received mean SUS scores below 68 except for Orchard Harvest (78.7). While the years of experience using the LIS was found to be a statistically significant influence on mean SUS scores, the combined effect of years of experience and LIS used did not account for the statistically significant difference in the mean SUS score between Orchard Harvest and each of the other LISs evaluated.ConclusionsThe results of this study indicate that overall usability of LISs is poor. Usability lags that of systems evaluated across 446 usability surveys.
| S-EPMC5653961 | biostudies-literature
Project description:BackgroundMedication is the most common intervention in health care, and the number of online consumer information systems within the pharmaceutical sector is increasing. However, online consumer information systems can be a barrier for users, imposing information asymmetries between stakeholders.ObjectiveThe objective of this study was to quantify and compare the usability of an online consumer medication information system (OCMIS) against a reference implementation based on an interoperable information model for patients, physicians, and pharmacists.MethodsQuantitative and qualitative data were acquired from patients, physicians, and pharmacists in this online usability study. We administered 3 use cases and a post hoc questionnaire per user. Quantitative usability data including effectiveness (task success), efficiency (task time), and user satisfaction (system usability scale [SUS]) was complemented by qualitative and demographic data. Users evaluated 6 existing systems and 1 reference implementation of an OCMIS.ResultsA total of 137 patients, 81 physicians, and 68 pharmacists participated in this study. Task success varied from 84% to 92% in patients, 66% to 100% in physicians, and 50% to 91% in pharmacists. Task completion time decreased over the course of the study for all but 2 OCMIS within the patient group. Due to an assumed nonnormal distribution of SUS scores, within-group comparison was done using the Kruskal-Wallis test. Patients showed differences in SUS scores (P=.02) and task time (P=.03), while physicians did not have significant differences in SUS scores (P=.83) and task time (P=.72). For pharmacists, a significant difference in SUS scores (P<.001) and task time (P=.007) was detected.ConclusionsThe vendor-neutral reference implementation based on an interoperable information model was proven to be a promising approach that was not inferior to existing solutions for patients and physicians. For pharmacists, it exceeded user satisfaction scores compared to other OCMIS. This data-driven approach based on an interoperable information model enables the development of more user-tailored features to increase usability. This fosters data democratization and empowers stakeholders within the pharmaceutical sector.
| S-EPMC7301258 | biostudies-literature
Project description:Health information technologies (HIT) have become nearly ubiquitous in the contemporary healthcare landscape, but information about HIT development, functionality, and implementation readiness is frequently siloed. Theory-driven methods of compiling, evaluating, and integrating information from the academic and commercial sectors are necessary to guide stakeholder decision-making surrounding HIT adoption and to develop pragmatic HIT research agendas. This article presents the Health Information Technologies-Academic and Commercial Evaluation (HIT-ACE) methodology, a structured, theory-driven method for compiling and evaluating information from multiple sectors. As an example demonstration of the methodology, we apply HIT-ACE to mental and behavioral health measurement feedback systems (MFS). MFS are a specific class of HIT that support the implementation of routine outcome monitoring, an evidence-based practice.HIT-ACE is guided by theories and frameworks related to user-centered design and implementation science. The methodology involves four phases: (1) coding academic and commercial materials, (2) developer/purveyor interviews, (3) linking putative implementation mechanisms to hit capabilities, and (4) experimental testing of capabilities and mechanisms. In the current demonstration, phase 1 included a systematic process to identify MFS in mental and behavioral health using academic literature and commercial websites. Using user-centered design, implementation science, and feedback frameworks, the HIT-ACE coding system was developed, piloted, and used to review each identified system for the presence of 38 capabilities and 18 additional characteristics via a consensus coding process. Bibliometic data were also collected to examine the representation of the systems in the scientific literature. As an example, results are presented for the application of HIT-ACE phase 1 to MFS wherein 49 separate MFS were identified, reflecting a diverse array of characteristics and capabilities.Preliminary findings demonstrate the utility of HIT-ACE to represent the scope and diversity of a given class of HIT beyond what can be identified in the academic literature. Phase 2 data collection is expected to confirm and expand the information presented and phases 3 and 4 will provide more nuanced information about the impact of specific HIT capabilities. In all, HIT-ACE is expected to support adoption decisions and additional HIT development and implementation research.
| S-EPMC5034443 | biostudies-other
Project description:BackgroundThe contribution of usability flaws to patient safety issues is acknowledged but not well-investigated. Free-text descriptions of incident reports may provide useful data to identify the connection between health information technology (HIT) usability flaws and patient safety.ObjectivesThis article examines the feasibility of using incident reports about HIT to learn about the usability flaws that affect patient safety. We posed three questions: (1) To what extent can we gain knowledge about usability issues from incident reports? (2) What types of usability flaws, related usage problems, and negative outcomes are reported in incidents reports? (3) What are the reported usability issues that give rise to patient safety issues?MethodsA sample of 359 reports from the U.S. Food and Drug Administration Manufacturer and User Facility Device Experience database was examined. Descriptions of usability flaws, usage problems, and negative outcomes were extracted and categorized. A supplementary analysis was performed on the incidents which contained the full chain going from a usability flaw up to a patient safety issue to identify the usability issues that gave rise to patient safety incidents.ResultsA total of 249 reports were included. We found that incident reports can provide knowledge about usability flaws, usage problems, and negative outcomes. Thirty-six incidents report how usability flaws affected patient safety (ranging from incidents without consequence, to death) involving electronic patient scales, imaging systems, and HIT for medication management. The most significant class of involved usability flaws is related to the reliability, the understandability, and the availability of the clinical information.ConclusionIncidents reports involving HIT are an exploitable source of information to learn about usability flaws and their effects on patient safety. Results can be used to convince all stakeholders involved in the HIT system lifecycle that usability should be considered seriously to prevent patient safety incidents.
| S-EPMC6561760 | biostudies-literature
Project description:Usability factors are a major obstacle to health information technology (IT) adoption. The purpose of this paper is to review and categorize health IT usability study methods and to provide practical guidance on health IT usability evaluation. 2025 references were initially retrieved from the Medline database from 2003 to 2009 that evaluated health IT used by clinicians. Titles and abstracts were first reviewed for inclusion. Full-text articles were then examined to identify final eligibility studies. 629 studies were categorized into the five stages of an integrated usability specification and evaluation framework that was based on a usability model and the system development life cycle (SDLC)-associated stages of evaluation. Theoretical and methodological aspects of 319 studies were extracted in greater detail and studies that focused on system validation (SDLC stage 2) were not assessed further. The number of studies by stage was: stage 1, task-based or user-task interaction, n=42; stage 2, system-task interaction, n=310; stage 3, user-task-system interaction, n=69; stage 4, user-task-system-environment interaction, n=54; and stage 5, user-task-system-environment interaction in routine use, n=199. The studies applied a variety of quantitative and qualitative approaches. Methodological issues included lack of theoretical framework/model, lack of details regarding qualitative study approaches, single evaluation focus, environmental factors not evaluated in the early stages, and guideline adherence as the primary outcome for decision support system evaluations. Based on the findings, a three-level stratified view of health IT usability evaluation is proposed and methodological guidance is offered based upon the type of interaction that is of primary interest in the evaluation.
| S-EPMC3341772 | biostudies-literature
Project description:Omics data sharing is crucial to the biological research community, and the last decade or two has seen a huge rise in collaborative analysis systems, databases, and knowledge bases for omics and other systems biology data. We assessed the "FAIRness" of NASA's GeneLab Data Systems (GLDS) along with four similar kinds of systems in the research omics data domain, using 14 FAIRness metrics. The range of overall FAIRness scores was 6-12 (out of 14), average 10.1, and standard deviation 2.4. The range of Pass ratings for the metrics was 29-79%, Partial Pass 0-21%, and Fail 7-50%. The systems we evaluated performed the best in the areas of data findability and accessibility, and worst in the area of data interoperability. Reusability of metadata, in particular, was frequently not well supported. We relate our experiences implementing semantic integration of omics data from some of the assessed systems for federated querying and retrieval functions, given their shortcomings in data interoperability. Finally, we propose two new principles that Big Data system developers, in particular, should consider for maximizing data accessibility.
| S-EPMC6371294 | biostudies-other
Project description:ObjectiveThe Health Information Technology Usability Evaluation Scale (Health-ITUES) is a validated and reliable instrument to evaluate usability of information technology (IT) tools. In this study, we aimed to establish the optimal cut-point of the Health-ITUES to identify usability of IT tools.MethodsAdult participants were recruited to a trial evaluating a mobile app for self-managing HIV. Participants completed the Health-ITUES at the 3- and 6-month follow-up. Health-ITUES is a 20-item questionnaire that assesses four subscales: impact, perceived usefulness, perceived ease of use, and user control. The total score ranged from 1 to 5 and a higher score indicates greater usability. App use was defined as the proportion of activities completed by participants in both study arms. The selection of an optimal cut-point involved a series of multiple linear regression models with 500 bootstrap replications to examine the relationship between the Health-ITUES total score and app use, controlling for potential covariates.ResultsWe included 158 participants; mean age was 49.7 years (SD 10.3), 71% were African American/Black, and 72% were non-Hispanic. Mean Health-ITUES total scores at 3 and 6 months were 4.39 (SD 0.75) and 4.43 (SD 0.75), respectively. App use completedby participants from baseline to the 3-month follow-up visits was 0.61 (SD 0.36, range 0-1.72) and from 3-month to the 6-month follow-up visits was 0.51 (SD 0.37). Participants who reported greater Health-ITUES total score completed more activities [β = 0.18, 95% Confidence Interval (CI) 0.10-0.27]. The optimal cut-point of 4.32 (95% CI: 4.25-4.56) yielded the lowest p-value to identify usability of IT tools.ConclusionsIn this study of adults with HIV, we identified an optimal cut-point of 4.32 on the Health-ITUES total score to define usability. Further studies are needed to validate this cut-point.
| S-EPMC8903058 | biostudies-literature
Project description:Metamaterials have great capabilities and flexibilities in controlling electromagnetic (EM) waves because their subwavelength meta-atoms can be designed and tailored in desired ways. However, once the structure-only metamaterials (i.e., passive metamaterials) are fabricated, their functions will be fixed. To control the EM waves dynamically, active devices are integrated into the meta-atoms, yielding active metamaterials. Traditionally, the active metamaterials include tunable metamaterials and reconfigurable metamaterials, which have either small-range tunability or a few numbers of reconfigurability. Recently, a special kind of active metamaterials, digital coding and programmable metamaterials, have been presented, which can realize a large number of distinct functionalities and switch them in real time with the aid of field programmable gate array (FPGA). More importantly, the digital coding representations of metamaterials make it possible to bridge the digital world and physical world using the metamaterial platform and make the metamaterials process digital information directly, resulting in information metamaterials. In this review article, we firstly introduce the evolution of metamaterials and then present the concepts and basic principles of digital coding metamaterials and information metamaterials. With more details, we discuss a series of information metamaterial systems, including the programmable metamaterial systems, software metamaterial systems, intelligent metamaterial systems, and space-time-coding metamaterial systems. Finally, we introduce the current progress and predict the future trends of information metamaterials.
| S-EPMC7415848 | biostudies-literature
Project description:In order to evaluate cell- and disease-specific changes in the interacting strength of chromatin targets, ChIP-seq signal across multiple conditions must undergo robust normalization. However, this is not possible using the standard ChIP-seq scheme, which lacks a reference for the control of biological and experimental variabilities. While several studies have recently proposed different solutions to circumvent this problem, substantial analytical differences among methodologies could hamper the experimental reproducibility and quantitative accuracy. Here, we propose a computational method to accurately compare ChIP-seq experiments, with exogenous spike-in chromatin, across samples in a genome-wide manner by using a local regression strategy (spikChIP). In contrast to the previous methodologies, spikChIP reduces the influence of sequencing noise of spike-in material during ChIP-seq normalization, while minimizes the overcorrection of non-occupied genomic regions in the experimental ChIP-seq. We demonstrate the utility of spikChIP with both histone and non-histone chromatin protein, allowing us to monitor for experimental reproducibility and the accurate ChIP-seq comparison of distinct experimental schemes. spikChIP software is available on GitHub (https://github.com/eblancoga/spikChIP).
| S-EPMC8315120 | biostudies-literature
Project description:Different experimental technologies measure different aspects of a system and to differing depth and breadth. High-throughput assays have inherently high false-positive and false-negative rates. Moreover, each technology includes systematic biases of a different nature. These differences make network reconstruction from multiple data sets difficult and error-prone. Additionally, because of the rapid rate of progress in biotechnology, there is usually no curated exemplar data set from which one might estimate data integration parameters. To address these concerns, we have developed data integration methods that can handle multiple data sets differing in statistical power, type, size, and network coverage without requiring a curated training data set. Our methodology is general in purpose and may be applied to integrate data from any existing and future technologies. Here we outline our methods and then demonstrate their performance by applying them to simulated data sets. The results show that these methods select true-positive data elements much more accurately than classical approaches. In an accompanying companion paper, we demonstrate the applicability of our approach to biological data. We have integrated our methodology into a free open source software package named POINTILLIST.
| S-EPMC1297682 | biostudies-literature