Dataset Information

Creating effective interrupted time series graphs: Review and recommendations.

ABSTRACT:

Introduction

Interrupted Time Series (ITS) studies may be used to assess the impact of an interruption, such as an intervention or exposure. The data from such studies are particularly amenable to visual display and, when clearly depicted, can readily show the short- and long-term impact of an interruption. Further, well-constructed graphs allow data to be extracted using digitizing software, which can facilitate their inclusion in systematic reviews and meta-analyses.

Aim

We provide recommendations for graphing ITS data, examine the properties of plots presented in ITS studies, and provide examples employing our recommendations.

Methods and results

Graphing recommendations from seminal data visualization resources were adapted for use with ITS studies. The adapted recommendations cover plotting of data points, trend lines, interruptions, additional lines and general graph components. We assessed whether 217 graphs from recently published (2013-2017) ITS studies met our recommendations and found that 130 graphs (60%) had clearly distinct data points, 100 (46%) had trend lines, and 161 (74%) had a clearly defined interruption. Accurate data extraction (requiring distinct points that align with axis tick marks and labels that allow the points to be interpreted) was possible in only 72 (33%) graphs.

Conclusion

We found that many ITS graphs did not meet our recommendations and could be improved with simple changes. Our proposed recommendations aim to achieve greater standardization and improvement in the display of ITS data, and facilitate re-use of the data in systematic reviews and meta-analyses.

SUBMITTER: Turner SL

PROVIDER: S-EPMC7818488 | biostudies-literature | 2021 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Creating effective interrupted time series graphs: Review and recommendations.

Turner Simon L SL Karahalios Amalia A Forbes Andrew B AB Taljaard Monica M Grimshaw Jeremy M JM Korevaar Elizabeth E Cheng Allen C AC Bero Lisa L McKenzie Joanne E JE

Research synthesis methods 20200722 1

<h4>Introduction</h4>Interrupted Time Series (ITS) studies may be used to assess the impact of an interruption, such as an intervention or exposure. The data from such studies are particularly amenable to visual display and, when clearly depicted, can readily show the short- and long-term impact of an interruption. Further, well-constructed graphs allow data to be extracted using digitizing software, which can facilitate their inclusion in systematic reviews and meta-analyses.<h4>Aim</h4>We prov ...[more]

PMID: 32657532

Similar Datasets

Project description:Background:Non-pharmacological interventions including physical activity programmes, occupational therapy and caregiver education programmes have been shown to lead to better outcomes for people with dementia and their care partners. Yet, there are gaps between what is recommended in guidelines and what happens in practice. The aim of this study was to bring together clinicians working in dementia care and establish a quality improvement collaborative. The aim of the quality improvement collaborative was to increase self-reported guideline adherence to three guideline recommendations. Methods:Interrupted time series. We recruited health professionals from community, hospital and aged care settings across Australia to join the collaborative. Members of the collaborative participated in a start-up meeting, completed an online learning course with clinical and quality improvement content, formed a quality improvement plan which was reviewed by a team of experts, received feedback following an audit of their current practice and were able to share experiences with their peers. The primary outcome was self-reported adherence to their guideline recommendation of interest which was measured using checklists. Data were collected monthly over a period of 18?months, and the study used an interrupted time series design and multilevel Poisson regression analysis to evaluate changes in self-reported adherence. Results:A total of 45 health professionals (78% therapists) from different sites joined the collaborative and 28 completed all requirements. Data from 1717 checklists were included in the analyses. Over the duration of the project, there was a significant increase in clinician self-reported adherence to guideline recommendations with a 42.1% immediate increase in adherence (incidence rate ratio = 1.42; 95% confidence interval = 1.08-1.87; p = 0.012). Conclusion:Health professionals working with people with dementia are interested in and willing to join a quality improvement collaborative with the goal of improving non-pharmacological aspects of care. Participation in the collaborative improved the quality of care for people with dementia as measured through self-reported adherence to guideline recommendations. Although there are challenges in implementation of guideline recommendations within dementia care, the quality improvement collaborative method was considered successful. A strength was that it equipped and empowered clinicians to lead improvement activities and allowed for heterogeneity in terms of service and setting. Trial registration:ACTRN12618000268246.

Project description:BackgroundDetailed intervention descriptions and robust evaluations that test intervention impact--and explore reasons for impact--are an essential part of progressing implementation science. Time series designs enable the impact and sustainability of intervention effects to be tested. When combined with time series designs, qualitative methods can provide insight into intervention effectiveness and help identify areas for improvement for future interventions. This paper describes the development, delivery, and evaluation of a tailored intervention designed to increase primary health care professionals' adoption of a national recommendation that women with mild to moderate postnatal depression (PND) are referred for psychological therapy as a first stage treatment.MethodsThree factors influencing referral for psychological treatment were targeted using three related intervention components: a tailored educational meeting, a tailored educational leaflet, and changes to an electronic system data template used by health professionals during consultations for PND. Evaluation comprised time series analysis of monthly audit data on percentage referral rates and monthly first prescription rates for anti-depressants. Interviews were conducted with a sample of health professionals to explore their perceptions of the intervention components and to identify possible factors influencing intervention effectiveness.ResultsThe intervention was associated with a significant, immediate, positive effect upon percentage referral rates for psychological treatments. This effect was not sustained over the ten month follow-on period. Monthly rates of anti-depressant prescriptions remained consistently high after the intervention. Qualitative interview findings suggest key messages received from the intervention concerned what appropriate antidepressant prescribing is, suggesting this to underlie the lack of impact upon prescribing rates. However, an understanding that psychological treatment can have long-term benefits was also cited. Barriers to referral identified before intervention were cited again after the intervention, suggesting the intervention had not successfully tackled the barriers targeted.ConclusionA time series design allowed the initial and sustained impact of our intervention to be tested. Combined with qualitative interviews, this provided insight into intervention effectiveness. Future research should test factors influencing intervention sustainability, and promote adoption of the targeted behavior and dis-adoption of competing behaviors where appropriate.

Project description:INTRODUCTION:An interrupted time series (ITS) design is an important observational design used to examine the effects of an intervention or exposure. This design has particular utility in public health where it may be impracticable or infeasible to use a randomised trial to evaluate health system-wide policies, or examine the impact of exposures (such as earthquakes). There have been relatively few studies examining the design characteristics and statistical methods used to analyse ITS designs. Further, there is a lack of guidance to inform the design and analysis of ITS studies.This is the first study in a larger project that aims to provide tools and guidance for researchers in the design and analysis of ITS studies. The objectives of this study are to (1) examine and report the design characteristics and statistical methods used in a random sample of contemporary ITS studies examining public health interventions or exposures that impact on health-related outcomes, and (2) create a repository of time series data extracted from ITS studies. Results from this study will inform the remainder of the project which will investigate the performance of a range of commonly used statistical methods, and create a repository of input parameters required for sample size calculation. METHODS AND ANALYSIS:We will collate 200 ITS studies evaluating public health interventions or the impact of exposures. ITS studies will be identified from a search of the bibliometric database PubMed between the years 2013 and 2017, combined with stratified random sampling. From eligible studies, we will extract study characteristics, details of the statistical models and estimation methods, effect metrics and parameter estimates. Further, we will extract the time series data when available. We will use systematic review methods in the screening, application of inclusion and exclusion criteria, and extraction of data. Descriptive statistics will be used to summarise the data. ETHICS AND DISSEMINATION:Ethics approval is not required since information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. A repository of data extracted from the published ITS studies will be made publicly available.

Project description:BACKGROUND:When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potential risk of producing misleading results when this rather robust design is not used appropriately. We performed a methodological systematic review of the literature to investigate the extent to which the use of ITS has followed best practice standards and recommendations in the evaluation of QI interventions. METHODS:We searched multiple databases from inception to June 2018 to identify QI intervention studies that were evaluated using ITS. There was no restriction on date, language and participants. Data were synthesised narratively using appropriate descriptive statistics. The risk of bias for ITS studies was assessed using the Cochrane Effective Practice and Organisation of Care standard criteria. The systematic review protocol was registered in PROSPERO (registration number: CRD42018094427). RESULTS:Of 4061 potential studies and 2028 unique records screened for inclusion, 120 eligible studies assessed eight QI strategies and were from 25 countries. Most studies were published since 2010 (86.7%), reported data using monthly interval (71.4%), used ITS without a control (81%) and modelled data using segmented regression (62.5%). Autocorrelation was considered in 55% of studies, seasonality in 20.8% and non-stationarity in 8.3%. Only 49.2% of studies specified the ITS impact model. The risk of bias was high or very high in 72.5% of included studies and did not change significantly over time. CONCLUSIONS:The use of ITS in the evaluation of health system QI interventions has increased considerably over the past decade. However, variations in methodological considerations and reporting of ITS in QI remain a concern, warranting a need to develop and reinforce formal reporting guidelines to improve its application in the evaluation of health system QI interventions.

Dataset Information

Creating effective interrupted time series graphs: Review and recommendations.

Introduction

Aim

Methods and results

Conclusion

Publications

Creating effective interrupted time series graphs: Review and recommendations.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets