Dataset Information

Usability Issues in Evidence-Based Psychosocial Interventions and Implementation Strategies: Cross-project Analysis.

ABSTRACT:

Background

People often prefer evidence-based psychosocial interventions (EBPIs) for mental health care; however, these interventions frequently remain unavailable to people in nonspecialty or integrated settings, such as primary care and schools. Previous research has suggested that usability, a concept from human-centered design, could support an understanding of the barriers to and facilitators of the successful adoption of EBPIs and support the redesign of EBPIs and implementation strategies.

Objective

This study aimed to identify and categorize usability issues in EBPIs and their implementation strategies.

Methods

We adapted a usability issue analysis and reporting format from a human-centered design. A total of 13 projects supported by the National Institute of Mental Health-funded Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness Center at the University of Washington used this format to describe usability issues for EBPIs and implementation strategies with which they were working. Center researchers used iterative affinity diagramming and coding processes to identify usability issue categories. On the basis of these categories and the underlying issues, we propose heuristics for the design or redesign of EBPIs and implementation strategies.

Results

The 13 projects reported a total of 90 usability issues, which we categorized into 12 categories, including complex and/or cognitively overwhelming, required time exceeding available time, incompatibility with interventionist preference or practice, incompatibility with existing workflow, insufficient customization to clients/recipients, intervention buy-in (value), interventionist buy-in (trust), overreliance on technology, requires unavailable infrastructure, inadequate scaffolding for client/recipient, inadequate training and scaffolding for interventionists, and lack of support for necessary communication. These issues range from minor inconveniences that affect a few interventionists or recipients to severe issues that prevent all interventionists or recipients in a setting from completing part or all of the intervention. We propose 12 corresponding heuristics to guide EBPIs and implementation strategy designers in preventing and addressing these usability issues.

Conclusions

Usability issues were prevalent in the studied EBPIs and implementation strategies. We recommend using the lens of usability evaluation to understand and address barriers to the effective use and reach of EBPIs and implementation strategies.

International registered report identifier (irrid)

RR2-10.2196/14990.

SUBMITTER: Munson SA

PROVIDER: S-EPMC9240934 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundCaring for people with dementia is complex, and there are various evidence-based interventions. However, a gap exists between the available interventions and how to implement them. The objectives of our review are to identify implementation strategies, implementation outcomes, and influencing factors for the implementation of evidence-based interventions that focus on three preselected phenomena in people with dementia: (A) behavior that challenges supporting a person with dementia in long-term care, (B) delirium in acute care, and (C) postacute care needs.MethodsWe conducted a scoping review according to the description of the Joanna Briggs Institute. We searched MEDLINE, CINAHL, and PsycINFO. For the data analysis, we conducted deductive content analysis. For this analysis, we used the Expert Recommendations for Implementation Change (ERIC), implementation outcomes according to Proctor and colleagues, and the Consolidated Framework for Implementation Research (CFIR).ResultsWe identified 362 (A), 544 (B), and 714 records (C) on the three phenomena and included 7 (A), 3 (B), and 3 (C) studies. Among the studies, nine reported on the implementation strategies they used. Clusters with the most reported strategies were adapt and tailor to context and train and educate stakeholders. We identified one study that tested the effectiveness of the applied implementation strategy, while ten studies reported implementation outcomes (mostly fidelity). Regarding factors that influence implementation, all identified studies reported between 1 and 19 factors. The most reported factors were available resources and the adaptability of the intervention. To address dementia-specific influencing factors, we enhanced the CFIR construct of patient needs and resources to include family needs and resources.ConclusionsWe found a high degree of homogeneity across the different dementia phenomena, the evidence-based interventions, and the care settings in terms of the implementation strategies used, implementation outcomes measured, and influencing factors identified. However, it remains unclear to what extent implementation strategies themselves are evidence-based and which intervention strategy can be used by practitioners when either the implementation outcomes are not adjusted to the implementation strategy and/or the effects of implementation strategies are mostly unknown. Future research needs to focus on investigating the effectiveness of implementation strategies for evidence-based interventions for dementia care.Trial registrationThe review protocol was prospectively published (Manietta et al., BMJ Open 11:e051611, 2021).

Project description:Family members remain the main care providers for the increasing numbers of people with dementia, and often become depressed or anxious. In an implementation research project, we aimed to widen access to Strategies for RelaTives (START), a clinically and cost-effective intervention for the mental health of family carers, by laying the foundations for its implementation in the third sector. We used the integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework to guide implementation of START, a manual-based, individually-delivered, multicomponent eight-session coping strategy intervention. We interviewed a maximum variation sample of twenty-seven stakeholders from the English Alzheimer's Society (AS), about possible difficulties in management, training, and delivery of START. We trained and supervised three AS dementia support workers in different locations, to each deliver START to three family carers. Two researchers independently coded pre-intervention interviews for themes. We assessed intervention feasibility through monitoring delivery fidelity, rating audio-recordings from 1-5 (5 being high) and interviewing facilitators, family carers and AS managers about their experiences. We assessed effectiveness on family carers' mental health using the Hospital Anxiety and Depression Scale (HADS) before and after receiving START (scores 0-42). We changed START's format by reflecting carer diversity more and increasing carer stories prominence, but core content or delivery processes were unchanged. All carers received START and attended every session. The mean fidelity score was 4.2. Mean HADS-total score reduced from baseline 18.4 (standard deviation 7.4) to follow-up 15.8 (9.7). Six (67%) carers scored as clinically depressed on baseline HADS and 2 (22%) at follow-up. Facilitators and carers rated START positively. Appropriately experienced third sector workers can be trained and supervised to deliver START and it remains effective. This has the potential for widened access at scale.

Project description:BackgroundAlignment (i.e., the process of creating fit between elements of the inner and outer context of an organization or system) in conjunction with implementation of an evidence-based intervention (EBI) has been identified as important for implementation outcomes. However, research evidence has so far not been systematically summarized. The aim of this scoping review is therefore to create an overview of how the concept of alignment has been applied in the EBI implementation literature to provide a starting point for future implementation efforts in health care.MethodsWe searched for peer-reviewed English language articles in four databases (MEDLINE, Cinahl, Embase, and Web of Science) published between 2003 and 2019. Extracted data were analyzed to address the study aims. A qualitative content analysis was carried out for items with more extensive information. The review was reported according to the preferred reporting items for systematic reviews and meta-analyses extension for scoping review (PRISMA-ScR) guidelines.ResultsThe database searches yielded 3629 publications, of which 235 were considered potentially relevant based on the predetermined eligibility criteria, and retrieved in full text. In this review, the results of 53 studies are presented. Different definitions and conceptualizations of alignment were found, which in general could be categorized as structural, as well as social, types of alignments. Whereas the majority of studies viewed alignment as important to understand the implementation process, only a few studies actually assessed alignment. Outcomes of alignment were focused on either EBI implementation, EBI sustainment, or healthcare procedures. Different actors were identified as important for creating alignment and five overall strategies were found for achieving alignment.ConclusionsAlthough investigating alignment has not been the primary focus of studies focusing on EBI implementation, it has still been identified as an important factor for the implementation success. Based on the findings from this review, future research should incorporate alignment and put a stronger emphasize on testing the effectiveness of alignment related to implementation outcomes.

Project description:BackgroundImplementation strategies have flourished in an effort to increase integration of research evidence into clinical practice. Most strategies are complex, socially mediated processes. Many are complicated, expensive, and ultimately impractical to deliver in real-world settings. The field lacks methods to assess the extent to which strategies are usable and aligned with the needs and constraints of the individuals and contexts who will deliver or receive them. Drawn from the field of human-centered design, cognitive walkthroughs are an efficient assessment method with potential to identify aspects of strategies that may inhibit their usability and, ultimately, effectiveness. This article presents a novel walkthrough methodology for evaluating strategy usability as well as an example application to a post-training consultation strategy to support school mental health clinicians to adopt measurement-based care.MethodThe Cognitive Walkthrough for Implementation Strategies (CWIS) is a pragmatic, mixed-methods approach for evaluating complex, socially mediated implementation strategies. CWIS includes six steps: (1) determine preconditions; (2) hierarchical task analysis; (3) task prioritization; (4) convert tasks to scenarios; (5) pragmatic group testing; and (6) usability issue identification, classification, and prioritization. A facilitator conducted two group testing sessions with clinician users (N = 10), guiding participants through 6 scenarios and 11 associated subtasks. Clinicians reported their anticipated likelihood of completing each subtask and provided qualitative justifications during group discussion. Following the walkthrough sessions, users completed an adapted quantitative assessment of strategy usability.ResultsAverage anticipated success ratings indicated substantial variability across participants and subtasks. Usability ratings (scale 0-100) of the consultation protocol averaged 71.3 (SD = 10.6). Twenty-one usability problems were identified via qualitative content analysis with consensus coding, and classified by severity and problem type. High-severity problems included potential misalignment between consultation and clinical service timelines as well as digressions during consultation processes.ConclusionsCWIS quantitative usability ratings indicated that the consultation protocol was at the low end of the "acceptable" range (based on norms from the unadapted scale). Collectively, the 21 resulting usability issues explained the quantitative usability data and provided specific direction for usability enhancements. The current study provides preliminary evidence for the utility of CWIS to assess strategy usability and generate a blueprint for redesign.