Dataset Information

The Stanford Lightning Report Method: A comparison of rapid qualitative synthesis results across four implementation evaluations.

ABSTRACT: Background:Current evaluation methods are mismatched with the speed of health care innovation and needs of health care delivery partners. We introduce a qualitative approach called the lightning report method and its specific product-the "Lightning Report." We compare implementation evaluation results across four projects to explore report sensitivity and the potential depth and breadth of lightning report method findings. Methods:The lightning report method was refined over 2.5 years across four projects: team-based primary care, cancer center transformation, precision health in primary care, and a national life-sustaining decisions initiative. The novelty of the lightning report method is the application of Plus/Delta/Insight debriefing to dynamic implementation evaluation. This analytic structure captures Plus ("what works"), Delta ("what needs to be changed"), and Insights (participant or evaluator insights, ideas, and recommendations). We used structured coding based on implementation science barriers and facilitators outlined in the Consolidated Framework for Implementation Research (CFIR) applied to 17 Lightning Reports from four projects. Results:Health care partners reported that Lighting Reports were valuable, easy to understand, and they implied reports supported "corrective action" for implementations. Comparative analysis revealed cross-project emphasis on the domains of Inner Setting and Intervention Characteristics, with themes of communication, resources/staffing, feedback/reflection, alignment with simultaneous interventions and traditional care, and team cohesion. In three of the four assessed projects, the largest proportion of coding was to the clinic-level domain of Inner Setting-ranging from 39% for the cancer center project to a high of 56% for the life-sustaining decisions project. Conclusions:The lightning report method can fill a gap in rapid qualitative approaches and is generalizable with consistent but flexible core methods. Comparative analysis suggests it is a sensitive tool, capable of uncovering differences and insights in implementation across projects. The Lightning Report facilitates partnered evaluation and communication with stakeholders by providing real-time, actionable insights in dynamic health care implementations.

SUBMITTER: Brown-Johnson C

PROVIDER: S-EPMC7156867 | biostudies-literature | 2020 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The Stanford Lightning Report Method: A comparison of rapid qualitative synthesis results across four implementation evaluations.

Brown-Johnson Cati C Safaeinili Nadia N Zionts Dani D Holdsworth Laura M LM Shaw Jonathan G JG Asch Steven M SM Mahoney Megan M Winget Marcy M

Learning health systems 20191221 2

<h4>Background</h4>Current evaluation methods are mismatched with the speed of health care innovation and needs of health care delivery partners. We introduce a qualitative approach called the lightning report method and its specific product-the "Lightning Report." We compare implementation evaluation results across four projects to explore report sensitivity and the potential depth and breadth of lightning report method findings.<h4>Methods</h4>The lightning report method was refined over 2.5 y ...[more]

PMID: 32313836

Similar Datasets

Project description:This study assesses the impact of the lightning nitric oxide (LNO) production schemes in the Community Multiscale Air Quality (CMAQ) model on ground-level air quality as well as aloft atmospheric chemistry through detailed evaluation of model predictions of nitrogen oxides (NO x ) and ozone (O3) with corresponding observations for the US. For ground-level evaluations, hourly O3 and NO x values from the U.S. EPA Air Quality System (AQS) monitoring network are used to assess the impact of different LNO schemes on model prediction of these species in time and space. Vertical evaluations are performed using ozonesonde and P-3B aircraft measurements during the Deriving Information on Surface Conditions from Column and Vertically Resolved Observations Relevant to Air Quality (DISCOVER-AQ) campaign conducted in the Baltimore- Washington region during July 2011. The impact on wet deposition of nitrate is assessed using measurements from the National Atmospheric Deposition Program's National Trends Network (NADP NTN). Compared with the Base model (without LNO), the impact of LNO on surface O3 varies from region to region depending on the Base model conditions. Overall statistics suggest that for regions where surface O3 mixing ratios are already overestimated, the incorporation of additional NO from lightning generally increased model overestimation of mean daily maximum 8 h (DM8HR) O3 by 1-2 ppb. In regions where surface O3 is underestimated by the Base model, LNO can significantly reduce the underestimation and bring model predictions close to observations. Analysis of vertical profiles reveals that LNO can significantly improve the vertical structure of modeled O3 distributions by reducing underestimation aloft and to a lesser degree decreasing overestimation near the surface. Since the Base model underestimates the wet deposition of nitrate in most regions across the modeling domain with the exception of the Pacific Coast, the inclusion of LNO leads to reduction in biases and errors and an increase in correlation coefficients at almost all the NADP NTN sites. Among the three LNO schemes described in Kang et al. (2019), the hNLDN scheme, which is implemented using hourly observed lightning flash data from National Lightning Detection Network (NLDN), performs best for comparisons with ground-level values, vertical profiles, and wet deposition of nitrate; the mNLDN scheme (the monthly NLDN-based scheme) performed slightly better. However, when observed lightning flash data are not available, the linear regression-based parameterization scheme, pNLDN, provides an improved estimate for nitrate wet deposition compared to the base simulation that does not include LNO.

Project description:BackgroundQualitative approaches, alone or in mixed methods, are prominent within implementation science. However, traditional qualitative approaches are resource intensive, which has led to the development of rapid qualitative approaches. Published rapid approaches are often inductive in nature and rely on transcripts of interviews. We describe a deductive rapid analysis approach using the Consolidated Framework for Implementation Research (CFIR) that uses notes and audio recordings. This paper compares our rapid versus traditional deductive CFIR approach.MethodsSemi-structured interviews were conducted for two cohorts of the Veterans Health Administration (VHA) Diffusion of Excellence (DoE). The CFIR guided data collection and analysis. In cohort A, we used our traditional CFIR-based deductive analysis approach (directed content analysis), where two analysts completed independent in-depth manual coding of interview transcripts using qualitative software. In cohort B, we used our new rapid CFIR-based deductive analysis approach (directed content analysis), where the primary analyst wrote detailed notes during interviews and immediately "coded" notes into a MS Excel CFIR construct by facility matrix; a secondary analyst then listened to audio recordings and edited the matrix. We tracked time for our traditional and rapid deductive CFIR approaches using a spreadsheet and captured transcription costs from invoices. We retrospectively compared our approaches in terms of effectiveness and rigor.ResultsCohorts A and B were similar in terms of the amount of data collected. However, our rapid deductive CFIR approach required 409.5 analyst hours compared to 683 h during the traditional deductive CFIR approach. The rapid deductive approach eliminated $7250 in transcription costs. The facility-level analysis phase provided the greatest savings: 14 h/facility for the traditional analysis versus 3.92 h/facility for the rapid analysis. Data interpretation required the same number of hours for both approaches.ConclusionOur rapid deductive CFIR approach was less time intensive and eliminated transcription costs, yet effective in meeting evaluation objectives and establishing rigor. Researchers should consider the following when employing our approach: (1) team expertise in the CFIR and qualitative methods, (2) level of detail needed to meet project aims, (3) mode of data to analyze, and (4) advantages and disadvantages of using the CFIR.

Project description:BACKGROUND:An interprofessional medication adherence program (IMAP) for chronic patients was developed and successfully implemented in the community pharmacy of the Department of ambulatory care and community medicine (Lausanne, Switzerland). This study assesses the capacity of a physician and a nurse at the infectious diseases service of a public hospital and of community pharmacists in the Neuchâtel area (Switzerland) to implement the IMAP in their practice. METHODS:Mixed method, prospective, observational study. Quantitative and qualitative analyses of the implementation process were conducted following the RE-AIM model (reach, effectiveness, adoption, implementation and maintenance). RESULTS:Implementation started in November 2014. One physician, one nurse, and five pharmacists agreed to participate. Healthcare professionals perceived the benefits of the program and were motivated to implement it in their practice (adoption). Seventeen patients were included in the program; 13 refused to participate. The inclusion of naïve HIV patients was easier than the inclusion of experienced patients with difficult psychosocial issues (reach). Pharmacists were engaged in reinforcing patient medication adherence in 25% of interviews (effectiveness). Key facilitators expressed by healthcare professionals were patient inclusion by the physician and the nurse instead of the pharmacist and the organisation of regular meetings between all stakeholders. In contrast, the encountered barriers were the lack of time and resources, the lack of team uptake, and the lack of adoption by senior managers (implementation). Interviewed patients were all satisfied with this new program, encouraging healthcare professionals to scale it up. Structural changes allowed the hospital and one pharmacy to enter the maintenance stage (maintenance). CONCLUSION:The research team and collaboration between all professionals involved played an important role in this implementation. However, the dissemination of such a program to a larger scale and for the long term requires financial and structural resources as well as transitional external support.

Project description:BackgroundEthnographic approaches offer a method and a way of thinking about implementation. This manuscript applies a specific case study method to describe the impact of the longitudinal interplay between implementation stakeholders. Growing out of science and technology studies (STS) and drawing on the latent archaeological sensibilities implied by ethnographic methods, the STS case-study is a tool for implementors to use when a piece of material culture is an essential component of an innovation.MethodsWe conducted an ethnographic process evaluation of the clinical implementation of tele-critical care (Tele-CC) services in the Department of Veterans Affairs. We collected fieldnotes and conducted participant observation at virtual and in-person education and planning events (n = 101 h). At Go-Live and 6-months post-implementation, we conducted site visits to the Tele-CC hub and 3 partnered ICUs. We led semi-structured interviews with ICU staff at Go-Live (43 interviews with 65 participants) and with ICU and Tele-CC staff 6-months post-implementation (44 interviews with 67 participants). We used verification strategies, including methodological coherence, appropriate sampling, collecting and analyzing data concurrently, and thinking theoretically, to ensure the reliability and validity of our data collection and analysis process.ResultsThe STS case-study helped us realize that we must think differently about how a Tele-CC clinician could be noticed moving from communal to intimate space. To understand how perceptions of surveillance impacted staff acceptance, we mapped the materials through which surveillance came to matter in the stories staff told about cameras, buttons, chimes, motors, curtains, and doorbells.ConclusionsSTS case-studies contribute to the literature on longitudinal qualitive research (LQR) in implementation science, including pen portraits and periodic reflections. Anchored by the material, the heterogeneity of an STS case-study generates questions and encourages exploring differences. Begun early enough, the STS case-study method, like periodic reflections, can serve to iteratively inform data collection for researchers and implementors. The next step is to determine systematically how material culture can reveal implementation barriers and direct attention to potential solutions that address tacit, deeply rooted challenges to innovations in practice and technology.

Project description:BackgroundThis paper reports on a qualitative evaluation of a hybrid type II stepped-wedge, cluster randomized trial using implementation facilitation to implement team-based care in the form of the collaborative chronic care model (CCM) in interdisciplinary outpatient mental health teams. The objective of this analysis is to compare the alignment of sites' clinical processes with the CCM elements at baseline (time 1) and after 12 months of implementation facilitation (time 2) from the perspective of providers.MethodsWe conducted semi-structured interviews to assess the extent to which six CCM elements were in place: work role redesign, patient self-management support, provider decision support, clinical information systems, linkages to community resources, and organizational/leadership support. Interviews were transcribed and a priori CCM elements were coded using a directed content analysis approach at times 1 and 2. We sought consensus on, and compared, the extent to which each CCM element was in place at times 1 and 2.ResultsWe conducted 27 and 31 telephone interviews at times 1 and 2, respectively, with outpatient mental health providers at nine participating sites. At time 1 and time 2, three CCM elements were most frequently present across the sites: work role redesign, patient self-management support, and provider decision support. The CCM elements with increased implementation from time 1 to time 2 were work role redesign, patient self-management support, and clinical information systems. For two CCM elements, linkages to community resources and organizational/leadership support, some sites had increased implementation at time 2 compared to time 1, while others had reductions. For the provider decision support element, we saw little change in the extent of its implementation.ConclusionsSites increased the extent of implementation on several CCM elements. The most progress was made in the CCM elements where sites had CCM-aligned processes in place at time 1. Teams made progress on elements they could more easily control, such as work role redesign. Our results suggest that maximizing the benefits of CCM-based outpatient mental health care may require targeting resources and training toward specific CCM elements-especially in the use of clinical information systems and linking with community resources.Trial registrationClinical Trials NCT02543840 .

Dataset Information

The Stanford Lightning Report Method: A comparison of rapid qualitative synthesis results across four implementation evaluations.

Publications

The Stanford Lightning Report Method: A comparison of rapid qualitative synthesis results across four implementation evaluations.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets