Dataset Information

Evidence-based usability design principles for medication alerting systems.

ABSTRACT:

Background

Usability flaws in medication alerting systems may have a negative impact on clinical use and patient safety. In order to prevent the release of alerting systems that contain such flaws, it is necessary to provide designers and evaluators with evidence-based usability design principles. The objective of the present study was to develop a comprehensive, structured list of evidence-based usability design principles for medication alerting systems.

Methods

Nine sets of design principles for medication alerting systems were analyzed, summarized, and structured. We then matched the summarized principles with a list of usability flaws in order to determine the level of underlying evidence.

Results

Fifty-eight principles were summarized from the literature and two additional principles were defined, so that each flaw was matched with a principle. We organized the 60 summarized usability design principles into 6 meta-principles, 38 principles, and 16 sub-principles. Only 15 principles were not matched with a usability flaw. The 6 meta-principles respectively covered the improvement of the signal-to-noise ratio, the support for collaborative working, the fit with a clinician's workflow, the data display, the transparency of the alerting system, and the actionable tools to be provided within an alert.

Conclusions

It is possible to develop an evidence-based, structured, comprehensive list of usability design principles that are specific to medication alerting systems and are illustrated by the corresponding usability flaws. This list represents an improvement over the current literature. Each principle is now associated with the best available evidence of its violation. This knowledge may help to improve the usability of medication alerting systems and, ultimately, decrease the harmful consequences of the systems' usability flaws.

SUBMITTER: Marcilly R

PROVIDER: S-EPMC6057098 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundIt is well known that recommendations from electronic medication alerts are seldom accepted or acted on by users. Key factors affecting the effectiveness of medication alerts include system usability and alert design. Thus, human factors principles that apply knowledge of human capabilities and limitations are increasingly used in the design of health technology to improve the usability of systems.ObjectiveThis study aims to evaluate a newly developed evidence-based self-assessment tool that allows the valid and reliable evaluation of computerized medication alerting systems. This tool was developed to be used by hospital staff with detailed knowledge of their hospital's computerized provider order entry system and alerts to identify and address potential system deficiencies. In this initial assessment, we aim to determine whether the items in the tool can measure compliance of medication alerting systems with human factors principles of design, the tool can be consistently used by multiple users to assess the same system, and the items are easy to understand and perceived to be useful for assessing medication alerting systems.MethodsThe Tool for Evaluating Medication Alerting Systems (TEMAS) was developed based on human factors design principles and consisted of 66 items. In total, 18 staff members recruited across 6 hospitals used the TEMAS to assess their medication alerting systems. Data collected from participant assessments were used to evaluate the validity, reliability, and usability of the TEMAS. Validity was assessed by comparing the results of the TEMAS with those of prior in-house evaluations. Reliability was measured using Krippendorff α to determine agreement among assessors. A 7-item survey was used to determine usability.ResultsThe participants reported mostly negative (n=8) and neutral (n=7) perceptions of alerts in their medication alerting system. However, the validity of the TEMAS could not be directly tested, as participants were unaware of any results from prior in-house evaluations. The reliability of the TEMAS, as measured by Krippendorff α, was low to moderate (range 0.26-0.46); however, participant feedback suggests that individuals' knowledge of the system varied according to their professional background. In terms of usability, 61% (11/18) of participants reported that the TEMAS items were generally easy to understand; however, participants suggested the revision of 22 items to improve clarity.ConclusionsThis initial assessment of the TEMAS allowed the identification of its components that required modification to improve usability and usefulness. It also revealed that for the TEMAS to be effective in facilitating a comprehensive assessment of a medication alerting system, it should be completed by a multidisciplinary team of hospital staff from both clinical and technical backgrounds to maximize their knowledge of systems.

Project description:BackgroundMedication is the most common intervention in health care, and the number of online consumer information systems within the pharmaceutical sector is increasing. However, online consumer information systems can be a barrier for users, imposing information asymmetries between stakeholders.ObjectiveThe objective of this study was to quantify and compare the usability of an online consumer medication information system (OCMIS) against a reference implementation based on an interoperable information model for patients, physicians, and pharmacists.MethodsQuantitative and qualitative data were acquired from patients, physicians, and pharmacists in this online usability study. We administered 3 use cases and a post hoc questionnaire per user. Quantitative usability data including effectiveness (task success), efficiency (task time), and user satisfaction (system usability scale [SUS]) was complemented by qualitative and demographic data. Users evaluated 6 existing systems and 1 reference implementation of an OCMIS.ResultsA total of 137 patients, 81 physicians, and 68 pharmacists participated in this study. Task success varied from 84% to 92% in patients, 66% to 100% in physicians, and 50% to 91% in pharmacists. Task completion time decreased over the course of the study for all but 2 OCMIS within the patient group. Due to an assumed nonnormal distribution of SUS scores, within-group comparison was done using the Kruskal-Wallis test. Patients showed differences in SUS scores (P=.02) and task time (P=.03), while physicians did not have significant differences in SUS scores (P=.83) and task time (P=.72). For pharmacists, a significant difference in SUS scores (P<.001) and task time (P=.007) was detected.ConclusionsThe vendor-neutral reference implementation based on an interoperable information model was proven to be a promising approach that was not inferior to existing solutions for patients and physicians. For pharmacists, it exceeded user satisfaction scores compared to other OCMIS. This data-driven approach based on an interoperable information model enables the development of more user-tailored features to increase usability. This fosters data democratization and empowers stakeholders within the pharmaceutical sector.

Project description:BackgroundMedication guides are required documents to be distributed to patients in order to convey serious risks associated with certain prescribed medicines. Little is known about the effectiveness of this information to adequately inform patients on safe use.ObjectiveTo examine the readability, suitability, and comprehensibility of medication guides, particularly for those with limited literacy.DesignAssessments of suitability and readability of 185 medication guides, and a sub-study examining change in suitability and readability from 2006 to 2010 among 32 of the medication guides (Study 1); 'open book' comprehension assessment of medication guides (Study 2).SettingTwo general internal medicine clinics in Chicago, IL.PatientsFour hundred and forty-nine adults seeking primary care services, ages 18-85.MeasurementsFor Study 1, the Suitability Assessment of Materials (SAM) and Lexile score for readability. For Study 2, a tailored comprehension assessment of content found in three representative medication guides.ResultsThe 185 analyzed medication guides were on average 1923 words (SD = 1022), with a mean reading level of 10-11th grade. Only one medication guide was deemed suitable in SAM analyses. None provided summaries or reviews, or framed the context first, while very few were rated as having made the purpose evident (8 %), or limited the scope of content (22 %). For Study 2, participants' comprehension of medication guides was poor (M = 52.7 % correct responses, SD = 22.6). In multivariable analysis, low and marginal literacy were independently associated with poorer understanding (β = -14.3, 95 % CI -18.0 - -10.6, p < 0.001; low: β = -23.7, 95 % CI -28.3 - -19.0, p < 0.001).ConclusionCurrent medication guides are of little value to patients, as they are too complex and difficult to understand especially for individuals with limited literacy. Explicit guidance is offered for improving these print materials.