Dataset Information

Does user-centred design affect the efficiency, usability and safety of CPOE order sets?

ABSTRACT:

Background

Application of user-centred design principles to Computerized provider order entry (CPOE) systems may improve task efficiency, usability or safety, but there is limited evaluative research of its impact on CPOE systems.

Objective

We evaluated the task efficiency, usability, and safety of three order set formats: our hospital's planned CPOE order sets (CPOE Test), computer order sets based on user-centred design principles (User Centred Design), and existing pre-printed paper order sets (Paper).

Participants

27 staff physicians, residents and medical students.

Setting

Sunnybrook Health Sciences Centre, an academic hospital in Toronto, Canada. Methods Participants completed four simulated order set tasks with three order set formats (two CPOE Test tasks, one User Centred Design, and one Paper). Order of presentation of order set formats and tasks was randomized. Users received individual training for the CPOE Test format only.

Main measures

Completion time (efficiency), requests for assistance (usability), and errors in the submitted orders (safety).

Results

27 study participants completed 108 order sets. Mean task times were: User Centred Design format 273 s, Paper format 293 s (p=0.73 compared to UCD format), and CPOE Test format 637 s (p<0.0001 compared to UCD format). Users requested assistance in 31% of the CPOE Test format tasks, whereas no assistance was needed for the other formats (p<0.01). There were no significant differences in number of errors between formats.

Conclusions

The User Centred Design format was more efficient and usable than the CPOE Test format even though training was provided for the latter. We conclude that application of user-centred design principles can enhance task efficiency and usability, increasing the likelihood of successful implementation.

SUBMITTER: Chan J

PROVIDER: S-EPMC3078654 | biostudies-literature | 2011 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Does user-centred design affect the efficiency, usability and safety of CPOE order sets?

Chan Julie J Shojania Kaveh G KG Easty Anthony C AC Etchells Edward E EE

Journal of the American Medical Informatics Association : JAMIA 20110501 3

<h4>Background</h4>Application of user-centred design principles to Computerized provider order entry (CPOE) systems may improve task efficiency, usability or safety, but there is limited evaluative research of its impact on CPOE systems.<h4>Objective</h4>We evaluated the task efficiency, usability, and safety of three order set formats: our hospital's planned CPOE order sets (CPOE Test), computer order sets based on user-centred design principles (User Centred Design), and existing pre-printed ...[more]

PMID: 21486886

Similar Datasets

Project description:BackgroundMobile health (mHealth) has the potential to improve access to healthcare, especially in developing countries. The proliferation of mHealth has not been accompanied by a corresponding growth in design guidelines for mHealth applications. This paper proposes a framework for mHealth application design that combines the Information Systems Research (ISR) framework and design thinking. We demonstrate a use case for the proposed framework in the form of an app to read the result of the tuberculin skin test (TST), which is used to screen for latent tuberculosis infection. The framework was used in the re-design of the TST reading app but could also be used in earlier stages of mHealth app design.MethodsThe ISR framework and design thinking were merged based on how the modes of design thinking integrate with the cycles of the ISR framework. Using the combined framework, we re-designed an mHealth app for TST reading, intended to be used primarily in a developing context by healthcare workers. Using the proposed framework, the app was iterated upon and developed with the aid of personas, observations, prototyping and questionnaires.ResultThe combined framework was applied through engagement with end-users, namely ten healthcare workers and ten graduate students. Through review of the literature and iterations of the app prototype, we identified various usability requirements and limitations. These included challenges related to image capture and a misunderstanding of instructions. These insights influenced the development and improvement of the app.ConclusionThe combined framework allowed for engagement with end-users and for low-cost, rapid development of the app while addressing contextual challenges and needs. The integration of design thinking modes with the ISR cycles was effective in achieving the objectives of each approach. The combined framework acknowledges the importance of engaging users when implementing mHealth technologies, especially in developing and under-resourced contexts. Findings from this study support the use of this framework as a guide in the design of user-centred mHealth interventions.

Project description:ObjectiveAuthors evaluated whether displaying context sensitive links to infrequently accessed educational materials and patient information via the user interface of an inpatient computerized care provider order entry (CPOE) system would affect access rates to the materials.DesignThe CPOE of Vanderbilt University Hospital (VUH) included "baseline" clinical decision support advice for safety and quality. Authors augmented this with seven new primarily educational decision support features. A prospective, randomized, controlled trial compared clinicians' utilization rates for the new materials via two interfaces. Control subjects could access study-related decision support from a menu in the standard CPOE interface. Intervention subjects received active notification when study-related decision support was available through context sensitive, visibly highlighted, selectable hyperlinks.MeasurementsRates of opportunities to access and utilization of study-related decision support materials from April 1999 through March 2000 on seven VUH Internal Medicine wards.ResultsDuring 4,466 intervention subject-days, there were 240,504 (53.9/subject-day) opportunities for study-related decision support, while during 3,397 control subject-days, there were 178,235 (52.5/subject-day) opportunities for such decision support, respectively (p = 0.11). Individual intervention subjects accessed the decision support features at least once on 3.8% of subject-days logged on (278 responses); controls accessed it at least once on 0.6% of subject-days (18 responses), with a response rate ratio adjusted for decision support frequency of 9.17 (95% confidence interval 4.6-18, p < 0.0005). On average, intervention subjects accessed study-related decision support materials once every 16 days individually and once every 1.26 days in aggregate.ConclusionHighlighting availability of context-sensitive educational materials and patient information through visible hyperlinks significantly increased utilization rates for study-related decision support when compared to "standard" VUH CPOE methods, although absolute response rates were low.

Project description:Background:Isolation precautions are intended to prevent transmission of infectious agents, yet healthcare provider (HCP) adherence remains suboptimal. This may be due to ambiguity regarding the required precautions or to cognitive overload of HCPs. In response to the challenge of changing HCP behaviour, increasing attention should be paid to the role of engineering controls and facility design that incorporate human factors elements. In the current study, we aimed to develop an isolation precaution signage system that provides visual cues, serves as a cognitive aid at the point of care, and removes ambiguity regarding which precautions are necessary (e.g. masks, gowns, gloves, single rooms) when caring for isolated patients. Methods:We employed a user-centred, participatory design approach in which HCPs were actively involved in generating an isolation precaution signage system based on human factors design principles. HCPs were purposefully sampled for each design phase to include a representative sample of potential system users. We conducted front-end analysis through interviews and observations to identify challenges related to the existing signage and to establish design requirements for new signage. This was followed by the creation of user personas, design thinking workshops, and prototyping, which then underwent iterative cycles of evaluation. Graphical symbols were developed and tested for comprehensibility. Results:Front-end analysis revealed several barriers to use of the current signage system such as unclear target audience, low signal-to-noise ratio, and ambiguity regarding the applicable precautions. A comprehensive list of design requirements was generated. The project ultimately resulted in a collection of validated, comprehensible symbols and signs for contact, droplet, and airborne isolation, as well as the identification of several systems-level solutions for work re-organisation to improve compliance with isolation precautions. Conclusions:The introduction of visual cues in the form of signage offers a promising opportunity to make guidelines available directly at the frontline. Anecdotal evidence based on observations and interviews with HCP have shown that the current solution is superior to previous isolation signage. User-centred participatory design was a useful approach that holds potential for further improving design in healthcare settings.

Project description:BackgroundThe design complexity of critical care ventilators (CCVs) can lead to use errors and patient harm. In this study, we present the results of a comparison of four CCVs from market leaders, using a rigorous methodology for the evaluation of use safety and user experience of medical devices.MethodsWe carried out a comparative usability study of four CCVs: Hamilton G5, Puritan Bennett 980, Maquet SERVO-U, and Dräger Evita V500. Forty-eight critical care respiratory therapists participated in this fully counterbalanced, repeated measures study. Participants completed seven clinical scenarios composed of 16 tasks on each ventilator. Use safety was measured by percentage of tasks with use errors or close calls (UE/CCs). User experience was measured by system usability and workload metrics, using the Post-Study System Usability Questionnaire (PSSUQ) and the National Aeronautics and Space Administration Task Load Index (NASA-TLX).ResultsNine of 18 post hoc contrasts between pairs of ventilators were significant after Bonferroni correction, with effect sizes between 0.4 and 1.09 (Cohen's d). There were significantly fewer UE/CCs with SERVO-U when compared to G5 (p?=?0.044) and V500 (p?=?0.020). Participants reported higher system usability for G5 when compared to PB980 (p?=?0.035) and higher system usability for SERVO-U when compared to G5 (p?<?0.001), PB980 (p?<?0.001), and V500 (p?<?0.001). Participants reported lower workload for G5 when compared to PB980 (p?<?0.001) and lower workload for SERVO-U when compared to PB980 (p?<?0.001) and V500 (p?<?0.001). G5 scored better on two of nine possible comparisons; SERVO-U scored better on seven of nine possible comparisons. Aspects influencing participants' performance and perception include the low sensitivity of G5's touchscreen and the positive effect from the quality of SERVO-U's user interface design.ConclusionsThis study provides empirical evidence of how four ventilators from market leaders compare and highlights the importance of medical technology design. Within the boundaries of this study, we can infer that SERVO-U demonstrated the highest levels of use safety and user experience, followed by G5. Based on qualitative data, differences in outcomes could be explained by interaction design, quality of hardware components used in manufacturing, and influence of consumer product technology on users' expectations.

Project description:ObjectivesWe aimed to develop a video animation knowledge translation (KT) resource to explain the purpose, use and importance of evidence synthesis to the public regarding healthcare decision-making.MethodsWe drew on a user-centred design approach to develop a spoken animated video (SAV) by conducting two cycles of idea generation, prototyping, user testing, analysis, and refinement. Six researchers identified the initial key messages of the SAV and informed the first draft of the storyboard and script. Seven members of the public provided input on this draft and the key messages through think-aloud interviews, which we used to develop an SAV prototype. Seven additional members of the public participated in think-aloud interviews while watching the video prototype. All members of the public also completed a questionnaire on perceived usefulness, desirability, clarity and credibility. We subsequently synthesised all data to develop the final SAV.ResultsResearchers identified the initial key messages as 1) the importance of evidence synthesis, 2) what an evidence synthesis is and 3) how evidence synthesis can impact healthcare decision-making. Members of the public rated the initial video prototype as 9/10 for usefulness, 8/10 for desirability, 8/10 for clarity and 9/10 for credibility. Using their guidance and feedback, we produced a three-and-a-half-minute video animation. The video was uploaded on YouTube, has since been translated into two languages, and viewed over 12,000 times to date.ConclusionsDrawing on user-centred design methods provided a structured and transparent approach to the development of our SAV. Involving members of the public enhanced the credibility and usefulness of the resource. Future work could explore involving the public from the outset to identify key messages in developing KT resources explaining methodological topics. This study describes the systematic development of a KT resource with limited resources and provides transferrable learnings for others wishing to do similar.

Dataset Information

Does user-centred design affect the efficiency, usability and safety of CPOE order sets?

Background

Objective

Participants

Setting

Main measures

Results

Conclusions

Publications

Does user-centred design affect the efficiency, usability and safety of CPOE order sets?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets