Dataset Information

User Testing to Improve Retrieval and Comprehension of Information in Guidelines to Improve Medicines Safety.

ABSTRACT:

Objective

The aim of the study was to investigate the effectiveness of user testing for improving healthcare professionals' retrieval and comprehension of information in medicines guidelines.

Methods

The United Kingdom's Injectable Medicines Guide was selected as a case study. This gives guidance to nurses on preparing and administering intravenous medicines on hospital wards, in line with standard UK practice. Three rounds of user testing were completed with 10 hospital nurses per round, using the Injectable Medicines Guide for voriconazole and aminophylline. Participants used the guidelines to answer 17 questions related to the administration of these medicines. Answers were scored for "finding" and "understanding" the required information. Semistructured interviews explored participants' opinions of guideline content, design, and wording, with responses analyzed thematically. The guidelines were revised between rounds.

Results

In round 1, 8 of 17 questions were answered correctly by all participants. Participants had difficulty with dose, dilution, administration rate, and adverse effects questions. Revisions included new subsections and increased calculation support. In round 2, 14 of 17 questions were answered correctly by all participants. Difficulty persisted with dose and administration rate questions and further revisions made. In round 3, 15 of 17 questions were answered correctly by all participants. Across all rounds, participants considered appropriate subheadings and information order as important for fast location of information. Specific, detailed, and practical instructions were perceived as important to improve understandability and usefulness.

Conclusions

Key information in medicines guidelines may not be found and/or understood by healthcare professionals. User testing increased information retrieval and comprehension and could have an important role in improving the safety of medicines use.

SUBMITTER: Jones MD

PROVIDER: S-EPMC7612138 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundUser-testing and subsequent modification of clinical guidelines increases health professionals' information retrieval and comprehension. No study has investigated whether this results in safer care.ObjectiveTo compare the frequency of medication errors when administering an intravenous medicine using the current National Health Service Injectable Medicines Guide (IMG) versus an IMG version revised with user-testing.MethodSingle-blind, randomised parallel group in situ simulation. Participants were on-duty nurses/midwives who regularly prepared intravenous medicines. Using a training manikin in their clinical area, participants administered a voriconazole infusion, a high-risk medicine requiring several steps to prepare. They were randomised to use current IMG guidelines or IMG guidelines revised with user-testing. Direct observation was used to time the simulation and identify errors. Participant confidence was measured using a validated instrument. The primary outcome was the percentage of simulations with at least one moderate-severe IMG-related error, with error severity classified by an expert panel.ResultsIn total, 133 participants were randomised to current guidelines and 140 to user-tested guidelines. Fewer moderate-severe IMG-related errors occurred with the user-tested guidelines (n=68, 49%) compared with current guidelines (n=79, 59%), but this difference was not statistically significant (risk ratio: 0.82; 95% CI 0.66 to 1.02). Significantly more simulations were completed without any IMG-related errors with the user-tested guidelines (n=67, 48%) compared with current guidelines (n=26, 20%) (risk ratio: 2.46; 95% CI 1.68 to 3.60). Median simulation completion time was 1.6 min (95% CI 0.2 to 3.0) less with the user-tested guidelines. Participants who used user-tested guidelines reported greater confidence.ConclusionUser-testing injectable medicines guidelines reduces the number of errors and the time taken to prepare and administer intravenous medicines, while increasing staff confidence.Trial registration numberresearchregistry5275.

Project description:BackgroundGuideline producers are increasingly producing versions of guidelines for the public, and knowledge of what the public want from them is also increasing. The aim of this study was to user test a patient version of a SIGN clinical guideline that was designed based on preliminary work for the DECIDE project. SIGN is the leading national guideline producer in Scotland.MethodsPeople with a diagnosis of glaucoma and non-professional carers were recruited from across Scotland. User testing was conducted using a think-aloud protocol method. Each session was conducted by one interviewer and one observer. All sessions were recorded and transcribed. The data was analysed, problems with the guideline were identified and resolved and key findings were themed using a user experience model.ResultsThirteen user testing sessions with people with glaucoma and one with a carer were conducted. Key facilitators of desirability and usability identified include clear branding as a patient version and a clearly described purpose, audience and contents page. Other facilitators include a "friendly" tone which is achieved by the use of colour, quotes, icons, simple language and charts, and brief chunked text. The value and usefulness of the patient guideline was influenced by its ability to: inform the public, link information to actions, and empower people in their interaction with healthcare professionals. Participants were disappointed by the lack of information on treatment in the patient version, which was outside its scope. Information on the evidence based guideline production process and the involvement of appropriately skilled professionals was key to the credibility of the guideline. Lack of awareness of guidelines and guideline producing bodies, is a potentially serious threat to findability/accessibility.ConclusionsIt is important for guideline producers to maximise the user experience of the public when they access patient versions of guidelines, particularly given the current low level of access and awareness. One size does not fit all and guideline producers need to strike a balance between keeping the patient version simple and providing sufficient information to facilitate shared decision making and empower the public. Guideline producers may find the results of this study useful in designing their own patient versions.

Project description:BackgroundVaginal mesh implants are medical devices used in a number of operations to treat stress urinary incontinence and pelvic organ prolapse. Although many of these operations have delivered good outcomes, some women have experienced serious complications that have profoundly affected their quality of life. To ensure that evolving patient information is up-to-date, accurate and appropriate, the Transvaginal Mesh Oversight Group 'user-tested' a newly developed Scottish patient resource, the first to focus exclusively on the issue of complications. The aim of this research was to gather feedback on usability, content, language and presentation to inform the development of the resource from a user perspective.MethodsThe experience of using the patient resource was captured through semi-structured interviews that followed a 'think-aloud' protocol. The interviewer observed each participant as they went through the resource, asking questions and making field notes. Participants' comments were then categorised using a validated model of user experience and subsequently analysed thematically.ResultsThirteen people participated in the user testing interviews, including women with lived experience of mesh implants (n = 7), a convenience sample of staff working for Healthcare Improvement Scotland (n = 5) and a patient's carer (n = 1). The majority of participants considered the resource as clear and helpful. Respondents reported that some presentational aspects promoted usability and understandability, including the use of a font that is easy to read, bullet lists, coloured headings and simple language. Barriers included the reliance on some technical language and an explicit anatomical diagram. Participants endorsed the valuable role of health professionals as co-mediators of patient information.ConclusionsThe findings illustrate the value of undertaking in-depth user-testing for patient information resources before their dissemination. The study highlighted how the direct guidance or navigation of a patient information resource by a health professional could increase its salience and accuracy of interpretation by patients, their families and carers. These insights may also be useful to other developers in improving patient information.

Project description:BackgroundTools for shared decision-making (e.g. decision aids) are intended to support health care professionals and patients engaged in clinical encounters involving shared decision-making. However, decision aids are hard to produce, and onerous to update. Consequently, they often do not reflect best current evidence, and show limited uptake in practice. In response, we initiated the Sharing Evidence to Inform Treatment decisions (SHARE-IT) project. Our goal was to develop and refine a new generation of decision aids that are generically produced along digitally structured guidelines and evidence summaries.MethodsApplying principles of human-centred design and following the International Patient Decision Aid Standards (IPDAS) and GRADE methods for trustworthy evidence summaries we developed a decision aid prototype in collaboration with the Developing and Evaluating Communication strategies to support Informed Decisions and practice based on Evidence project (DECIDE). We iteratively user-tested the prototype in clinical consultations between clinicians and patients. Semi-structured interviews of participating clinicians and patients were conducted. Qualitative content analysis of both user-testing sessions and interviews was performed and results categorized according to a revised Morville's framework of user-experience. We made it possible to produce, publish and use these decision aids in an electronic guideline authoring and publication platform (MAGICapp).ResultsDirect observations and analysis of user-testing of 28 clinical consultations between physicians and patients informed four major iterations that addressed readability, understandability, usability and ways to cope with information overload. Participants reported that the tool supported natural flow of the conversation and induced a positive shift in consultation habits towards shared decision-making. We integrated the functionality of SHARE-IT decision aids in MAGICapp, which has since generated numerous decision aids.ConclusionOur study provides a proof of concept that encounter decision aids can be generically produced from GRADE evidence summaries and clinical guidelines. Online authoring and publication platforms can help scale up production including continuous updating of electronic encounter decision aids, fully integrated with evidence summaries and clinical practice guidelines.