Dataset Information

Detection and characterization of usability problems in structured data entry interfaces in dentistry.

ABSTRACT:

Background

Poor usability is one of the major barriers for optimally using electronic health records (EHRs). Dentists are increasingly adopting EHRs, and are using structured data entry interfaces to enter data such that the data can be easily retrieved and exchanged. Until recently, dentists have lacked a standardized terminology to consistently represent oral health diagnoses.

Objectives

In this study we evaluated the usability of a widely used EHR interface that allow the entry of diagnostic terms, using multi-faceted methods to identify problems and work with the vendor to correct them using an iterative design method.

Methods

Fieldwork was undertaken at two clinical sites, and dental providers as subjects participated in user testing (n=32), interviews (n=36) and observations (n=24).

Results

User testing revealed that only 22-41% of users were able to successfully complete a simple task of entering one diagnosis, while no user was able to complete a more complex task. We identified and characterized 24 high-level usability problems reducing efficiency and causing user errors. Interface-related problems included unexpected approaches for displaying diagnosis, lack of visibility, and inconsistent use of UI widgets. Terminology related issues included missing and mis-categorized concepts. Work domain issues involved both absent and superfluous functions. In collaboration with the vendor, each usability problem was prioritized and a timeline set to resolve the concerns.

Discussion

Mixed methods evaluations identified a number of critical usability issues relating to the user interface, underlying terminology of the work domain. The usability challenges were found to prevent most users from successfully completing the tasks. Our further work we will determine if changes to the interface, terminology and work domain do result in improved usability.

SUBMITTER: Walji MF

PROVIDER: S-EPMC3492538 | biostudies-literature | 2013 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Detection and characterization of usability problems in structured data entry interfaces in dentistry.

Walji Muhammad F MF Kalenderian Elsbeth E Tran Duong D Kookal Krishna K KK Nguyen Vickie V Tokede Oluwabunmi O White Joel M JM Vaderhobli Ram R Ramoni Rachel R Stark Paul C PC Kimmes Nicole S NS Schoonheim-Klein Meta E ME Patel Vimla L VL

International journal of medical informatics 20120629 2

<h4>Background</h4>Poor usability is one of the major barriers for optimally using electronic health records (EHRs). Dentists are increasingly adopting EHRs, and are using structured data entry interfaces to enter data such that the data can be easily retrieved and exchanged. Until recently, dentists have lacked a standardized terminology to consistently represent oral health diagnoses.<h4>Objectives</h4>In this study we evaluated the usability of a widely used EHR interface that allow the entry ...[more]

PMID: 22749840

Similar Datasets

Project description:There is an increased demand in hospitals for tools, such as dedicated mobile device apps, that enable the recording of clinical information in an electronic format at the patient's bedside. Although the human-machine interface design on mobile devices strongly influences the accuracy and effectiveness of data recording, there is still a lack of evidence as to which interface design offers the best guarantee for ease of use and quality of recording. Therefore, interfaces need to be assessed both for usability and reliability because recording errors can seriously impact the overall level of quality of the data and affect the care provided.In this randomized crossover trial, we formally compared 6 handheld device interfaces for both speed of data entry and accuracy of recorded information. Three types of numerical data commonly recorded at the patient's bedside were used to evaluate the interfaces.In total, 150 health care professionals from the University Hospitals of Geneva volunteered to record a series of randomly generated data on each of the 6 interfaces provided on a smartphone. The interfaces were presented in a randomized order as part of fully automated data entry scenarios. During the data entry process, accuracy and effectiveness were automatically recorded by the software.Various types of errors occurred, which ranged from 0.7% for the most reliable design to 18.5% for the least reliable one. The length of time needed for data recording ranged from 2.81 sec to 14.68 sec, depending on the interface. The numeric keyboard interface delivered the best performance for pulse data entry with a mean time of 3.08 sec (SD 0.06) and an accuracy of 99.3%.Our study highlights the critical impact the choice of an interface can have on the quality of recorded data. Selecting an interface should be driven less by the needs of specific end-user groups or the necessity to facilitate the developer's task (eg, by opting for default solutions provided by commercial platforms) than by the level of speed and accuracy an interface can provide for recording information. An important effort must be made to properly validate mobile device interfaces intended for use in the clinical setting. In this regard, our study identified the numeric keyboard, among the proposed designs, as the most accurate interface for entering specific numerical values. This is an important step toward providing clearer guidelines on which interface to choose for the appropriate use of handheld device interfaces in the health care setting.

Project description:BackgroundDementia-related impairments can cause complex barriers to access, use, and adopt digital health technologies (DHTs). These barriers can contribute to digital health inequities. Therefore, literature-based design principles called DEMIGNED have been developed to support the design and evaluation of DHTs for this rapidly increasing population.ObjectiveThis study aims to apply the DEMIGNED principles in usability evaluation methods to (1) capture usability problems on a mobile website providing information resources for people visiting a memory clinic, including those living with subjective cognitive decline (SCD), mild cognitive impairment (MCI), or dementia, and (2) investigate the realness of usability problems captured by the DEMIGNED principles in expert testing, specifically for mobile websites that act as a means of providing DHTs.MethodsFirst, a heuristic evaluation was conducted, with the DEMIGNED principles serving as domain-specific guidelines, with 3 double experts (experienced in both usability and dementia) and 2 usability engineering experts. Second, think-aloud sessions were conducted with patients visiting a memory clinic who were living with SCD, MCI, or dementia.ResultsThe heuristic evaluation resulted in 36 unique usability problems. A representative sample of 7 people visiting a memory clinic participated in a think-aloud session, including 4 (57%) with SCD, 1 (14%) with MCI, and 2 (29%) with dementia. The analysis of the think-aloud sessions revealed 181 encounters with usability problems. Of these encounters, 144 (79.6%) could be mapped to 18 usability problems identified in the heuristic evaluation. The remaining 37 (20.4%) encounters from the user testing revealed another 10 unique usability problems. Usability problems frequently described in the think-aloud sessions encompassed difficulties with using the search function, discrepancies between the user's expectations and the content organization, the need for scrolling, information overload, and unclear system feedback.ConclusionsBy applying the DEMIGNED principles in expert testing, evaluators were able to capture 79.6% (144/181) of all usability problem encounters in the user testing of a mobile website for people visiting a memory clinic, including people living with dementia. Regarding unique usability problems, 50% (18/36) of the unique usability problems identified during the heuristic evaluation were captured by the user-testing sessions. Future research should look into the applicability of the DEMIGNED principles to other digital health functionalities to increase the accessibility of digital health and decrease digital health inequity for this complex and rapidly increasing population.

Project description:BackgroundUsability testing of medical devices are mandatory for market access. The testings' goal is to identify usability problems that could cause harm to the user or limit the device's effectiveness. In practice, human factor engineers study participants under actual conditions of use and list the problems encountered. This results in a binary discovery matrix in which each row corresponds to a participant, and each column corresponds to a usability problem. One of the main challenges in usability testing is estimating the total number of problems, in order to assess the completeness of the discovery process. Today's margin-based methods fit the column sums to a binomial model of problem detection. However, the discovery matrix actually observed is truncated because of undiscovered problems, which corresponds to fitting the marginal sums without the zeros. Margin-based methods fail to overcome the bias related to truncation of the matrix. The objective of the present study was to develop and test a matrix-based method for estimating the total number of usability problems.MethodsThe matrix-based model was based on the full discovery matrix (including unobserved columns) and not solely on a summary of the data (e.g. the margins). This model also circumvents a drawback of margin-based methods by simultaneously estimating the model's parameters and the total number of problems. Furthermore, the matrix-based method takes account of a heterogeneous probability of detection, which reflects a real-life setting. As suggested in the usability literature, we assumed that the probability of detection had a logit-normal distribution.ResultsWe assessed the matrix-based method's performance in a range of settings reflecting real-life usability testing and with heterogeneous probabilities of problem detection. In our simulations, the matrix-based method improved the estimation of the number of problems (in terms of bias, consistency, and coverage probability) in a wide range of settings. We also applied our method to five real datasets from usability testing.ConclusionsEstimation models (and particularly matrix-based models) are of value in estimating and monitoring the detection process during usability testing. Matrix-based models have a solid mathematical grounding and, with a view to facilitating the decision-making process for both regulators and device manufacturers, should be incorporated into current standards.

Dataset Information

Detection and characterization of usability problems in structured data entry interfaces in dentistry.

Background

Objectives

Methods

Results

Discussion

Publications

Detection and characterization of usability problems in structured data entry interfaces in dentistry.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets