Dataset Information

Interrupted time series regression for the evaluation of public health interventions: a tutorial.

ABSTRACT: Interrupted time series (ITS) analysis is a valuable study design for evaluating the effectiveness of population-level health interventions that have been implemented at a clearly defined point in time. It is increasingly being used to evaluate the effectiveness of interventions ranging from clinical therapy to national public health legislation. Whereas the design shares many properties of regression-based approaches in other epidemiological studies, there are a range of unique features of time series data that require additional methodological considerations. In this tutorial we use a worked example to demonstrate a robust approach to ITS analysis using segmented regression. We begin by describing the design and considering when ITS is an appropriate design choice. We then discuss the essential, yet often omitted, step of proposing the impact model a priori. Subsequently, we demonstrate the approach to statistical analysis including the main segmented regression model. Finally we describe the main methodological issues associated with ITS analysis: over-dispersion of time series data, autocorrelation, adjusting for seasonal trends and controlling for time-varying confounders, and we also outline some of the more complex design adaptations that can be used to strengthen the basic ITS design.

SUBMITTER: Bernal JL

PROVIDER: S-EPMC5407170 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:INTRODUCTION:An interrupted time series (ITS) design is an important observational design used to examine the effects of an intervention or exposure. This design has particular utility in public health where it may be impracticable or infeasible to use a randomised trial to evaluate health system-wide policies, or examine the impact of exposures (such as earthquakes). There have been relatively few studies examining the design characteristics and statistical methods used to analyse ITS designs. Further, there is a lack of guidance to inform the design and analysis of ITS studies.This is the first study in a larger project that aims to provide tools and guidance for researchers in the design and analysis of ITS studies. The objectives of this study are to (1) examine and report the design characteristics and statistical methods used in a random sample of contemporary ITS studies examining public health interventions or exposures that impact on health-related outcomes, and (2) create a repository of time series data extracted from ITS studies. Results from this study will inform the remainder of the project which will investigate the performance of a range of commonly used statistical methods, and create a repository of input parameters required for sample size calculation. METHODS AND ANALYSIS:We will collate 200 ITS studies evaluating public health interventions or the impact of exposures. ITS studies will be identified from a search of the bibliometric database PubMed between the years 2013 and 2017, combined with stratified random sampling. From eligible studies, we will extract study characteristics, details of the statistical models and estimation methods, effect metrics and parameter estimates. Further, we will extract the time series data when available. We will use systematic review methods in the screening, application of inclusion and exclusion criteria, and extraction of data. Descriptive statistics will be used to summarise the data. ETHICS AND DISSEMINATION:Ethics approval is not required since information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. A repository of data extracted from the published ITS studies will be made publicly available.

Project description:BACKGROUND:When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potential risk of producing misleading results when this rather robust design is not used appropriately. We performed a methodological systematic review of the literature to investigate the extent to which the use of ITS has followed best practice standards and recommendations in the evaluation of QI interventions. METHODS:We searched multiple databases from inception to June 2018 to identify QI intervention studies that were evaluated using ITS. There was no restriction on date, language and participants. Data were synthesised narratively using appropriate descriptive statistics. The risk of bias for ITS studies was assessed using the Cochrane Effective Practice and Organisation of Care standard criteria. The systematic review protocol was registered in PROSPERO (registration number: CRD42018094427). RESULTS:Of 4061 potential studies and 2028 unique records screened for inclusion, 120 eligible studies assessed eight QI strategies and were from 25 countries. Most studies were published since 2010 (86.7%), reported data using monthly interval (71.4%), used ITS without a control (81%) and modelled data using segmented regression (62.5%). Autocorrelation was considered in 55% of studies, seasonality in 20.8% and non-stationarity in 8.3%. Only 49.2% of studies specified the ITS impact model. The risk of bias was high or very high in 72.5% of included studies and did not change significantly over time. CONCLUSIONS:The use of ITS in the evaluation of health system QI interventions has increased considerably over the past decade. However, variations in methodological considerations and reporting of ITS in QI remain a concern, warranting a need to develop and reinforce formal reporting guidelines to improve its application in the evaluation of health system QI interventions.

Project description:BackgroundReducing maternal mortality remains a global priority. In 2000, the United Nations Member States pledged to work towards a series of Millennium Development Goals (MDGs), in which the fifth target was to reduce maternal mortality ratio by 75% from 1990 to 2015. The Chinese government introduced Basic Public Health Service project in 2009 to the further improvement of maternal health services and reduction in maternal mortality. China had achieved the goal of MDG5 1 year ahead of the schedule in 2014, but the effects of the project on reducing maternal mortality were rarely evaluated with robust methods.MethodsWe conducted a longitudinal study on maternal mortality ratio by extracting mortality data from the National Maternal Mortality Surveillance System (1991-2016) and maternal health services measures from the China health statistic yearbook (2001-2016). We utilized the segmented linear regression model to assess changes and trends of maternal mortality ratio and maternal health services before and after the introduction of Basic Public Health Service project. Pearson correlation analysis was conducted to measure the strength of association between the maternal mortality ratio and maternal health services.ResultsThe yearly trend change of national maternal mortality ratio was - 1.76 (p < 0.01) after the introduction of Basic Public Health Service project in 2009, while the yearly trend change of maternal health record establish rate, prenatal examination rate, postpartum visit rate was 0.77 (p < 0.01), 0.61 (p < 0.01) and 0.83 (p < 0.01) separately. The negative correlations were also found between national maternal mortality ratio and prenatal examination rate (r = - 0.95, p < 0.01), maternal health record establish rate (r = - 0.93, p < 0.01) and postpartum visit rate (r = - 0.92, p < 0.01).ConclusionsThe Basic Public Health Service project was found to be associated with the improvements in the maternal health services and reduction in maternal mortality. The design and implementation of the project may serve as a positive example for other developing countries. Continued monitoring and assessment of project effects should be stressed.

Project description:BackgroundThe long-term impact of COVID-19-associated public health interventions on zoonotic and vector-borne infectious diseases (ZVBs) remains uncertain. This study sought to examine the changes in ZVBs in China during the COVID-19 pandemic and predict their future trends.MethodsMonthly incidents of seven ZVBs (Hemorrhagic fever with renal syndrome [HFRS], Rabies, Dengue fever [DF], Human brucellosis [HB], Leptospirosis, Malaria, and Schistosomiasis) were gathered from January 2004 to July 2023. An autoregressive fractionally integrated moving average (ARFIMA) by incorporating the COVID-19-associated public health intervention variables was developed to evaluate the long-term effectiveness of interventions and forecast ZVBs epidemics from August 2023 to December 2025.ResultsOver the study period, there were 1,599,647 ZVBs incidents. HFRS and rabies exhibited declining trends, HB showed an upward trajectory, while the others remained relatively stable. The ARFIMA, incorporating a pulse pattern, estimated the average monthly number of changes of - 83 (95% confidence interval [CI] - 353-189) cases, - 3 (95% CI - 33-29) cases, - 468 (95% CI - 1531-597) cases, 2191 (95% CI 1056-3326) cases, 7 (95% CI - 24-38) cases, - 84 (95% CI - 222-55) cases, and - 214 (95% CI - 1036-608) cases for HFRS, rabies, DF, HB, leptospirosis, malaria, and schistosomiasis, respectively, although these changes were not statistically significant besides HB. ARFIMA predicted a decrease in HB cases between August 2023 and December 2025, while indicating a relative plateau for the others.ConclusionsChina's dynamic zero COVID-19 strategy may have exerted a lasting influence on HFRS, rabies, DF, malaria, and schistosomiasis, beyond immediate consequences, but not affect HB and leptospirosis. ARFIMA emerges as a potent tool for intervention analysis, providing valuable insights into the sustained effectiveness of interventions. Consequently, the application of ARFIMA contributes to informed decision-making, the design of effective interventions, and advancements across various fields.

Project description:BACKGROUND:Rapid evaluation was at the heart of National Health Service England's evaluation strategy of the new models of care vanguard programme. This was to facilitate the scale and spread of successful models of care throughout the health & social care system. The aim of this paper is to compare the findings of the two evaluations of the Enhanced health in Care Homes (EHCH) vanguard in Gateshead, one using a smaller data set for rapidity and one using a larger longitudinal data set and to investigate the implications of the use of rapid evaluations using interrupted time series (ITS) methods. METHODS:A quasi-experimental design study in the form of an ITS was used to evaluate the impact of the vanguard on secondary care use. Two different models are presented differing by timeframes only. The short-term model consisted of data for 11 months data pre and 20 months post vanguard. The long-term model consisted of data for 23 months pre and 34 months post vanguard. RESULTS:The cost consequences, including the cost of running the EHCH vanguard, were estimated using both a single tariff non-elective admissions methodology and a tariff per bed day methodology. The short-term model estimated a monthly cost increase of £73,408 using a single tariff methodology. When using a tariff per bed day, there was an estimated monthly cost increase of £14,315. The long-term model had, using a single tariff for non-elective admissions, an overall cost increase of £7576 per month. However, when using a tariff per bed-days, there was an estimated monthly cost reduction of £57,168. CONCLUSIONS:Although it is acknowledged that there is often a need for rapid evaluations in order to identify "quick wins" and to expedite learning within health and social care systems, we conclude that this may not be appropriate for quasi-experimental designs estimating effect using ITS for complex interventions. Our analyses suggests that care must be taken when conducting and interpreting the results of short-term evaluations using ITS methods, as they may produce misleading results and may lead to a misallocation of resources.

Dataset Information

Interrupted time series regression for the evaluation of public health interventions: a tutorial.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets