Dataset Information

Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review.

ABSTRACT: BACKGROUND:When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potential risk of producing misleading results when this rather robust design is not used appropriately. We performed a methodological systematic review of the literature to investigate the extent to which the use of ITS has followed best practice standards and recommendations in the evaluation of QI interventions. METHODS:We searched multiple databases from inception to June 2018 to identify QI intervention studies that were evaluated using ITS. There was no restriction on date, language and participants. Data were synthesised narratively using appropriate descriptive statistics. The risk of bias for ITS studies was assessed using the Cochrane Effective Practice and Organisation of Care standard criteria. The systematic review protocol was registered in PROSPERO (registration number: CRD42018094427). RESULTS:Of 4061 potential studies and 2028 unique records screened for inclusion, 120 eligible studies assessed eight QI strategies and were from 25 countries. Most studies were published since 2010 (86.7%), reported data using monthly interval (71.4%), used ITS without a control (81%) and modelled data using segmented regression (62.5%). Autocorrelation was considered in 55% of studies, seasonality in 20.8% and non-stationarity in 8.3%. Only 49.2% of studies specified the ITS impact model. The risk of bias was high or very high in 72.5% of included studies and did not change significantly over time. CONCLUSIONS:The use of ITS in the evaluation of health system QI interventions has increased considerably over the past decade. However, variations in methodological considerations and reporting of ITS in QI remain a concern, warranting a need to develop and reinforce formal reporting guidelines to improve its application in the evaluation of health system QI interventions.

SUBMITTER: Hategeka C

PROVIDER: S-EPMC7559052 | biostudies-literature | 2020 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review.

Hategeka Celestin C Ruton Hinda H Karamouzian Mohammad M Lynd Larry D LD Law Michael R MR

BMJ global health 20201001 10

<h4>Background</h4>When randomisation is not possible, interrupted time series (ITS) design has increasingly been advocated as a more robust design to evaluating health system quality improvement (QI) interventions given its ability to control for common biases in healthcare QI. However, there is a potential risk of producing misleading results when this rather robust design is not used appropriately. We performed a methodological systematic review of the literature to investigate the extent to ...[more]

PMID: 33055094

Similar Datasets

Project description:BACKGROUND:As part of a partnership between the Institute for Healthcare Improvement and the Ethiopian Federal Ministry of Health, woreda-based quality improvement collaboratives took place between November 2016 and December 2017 aiming to accelerate reduction of maternal and neonatal mortality in Lemu Bilbilu, Tanqua Abergele and Duguna Fango woredas. Before starting the collaboratives, assessments found inaccuracies in core measures obtained from Health Management Information System reports. METHODS AND RESULTS:Building on the quality improvement collaborative design, data quality improvement activities were added and we used the World Health Organization review methodology to drive a verification factor for the core measures of number of pregnant women that received their first antenatal care visit, number of pregnant women that received antenatal care on at least four visits, number of pregnant women tested for syphilis and number of births attended by skilled health personnel. Impact of the data quality improvement was assessed using interrupted time series analysis. We found accurate data across all time periods for Tanqua Abergele. In Lemu Bilbilu and Duguna Fango, data quality improved for all core metrics over time. In Duguna Fango, the verification factor for number of pregnant women that received their first antenatal care visit improved from 0.794 (95%CI 0.753, 0.836; p<0.001) pre-intervention by 0.173 (95%CI 0.128, 0.219; p<0.001) during the collaborative; and the verification factor for number of pregnant women tested for syphilis improved from 0.472 (95%CI 0.390, 0.554; p<0.001) pre-intervention by 0.460 (95%CI 0.369, 0.552; p<0.001) during the collaborative. In Lemu Bilbilu, the verification factor for number of pregnant women receiving a fourth antenatal visit rose from 0.589 (95%CI 0.513, 0.664; p<0.001) at baseline by 0.358 (95%CI 0.258, 0.458; p<0.001) post-intervention; and skilled birth attendance rose from 0.917 (95%CI 0.869, 0.965) at baseline by 0.083 (95%CI 0.030, 0.136; p<0.001) during the collaborative. CONCLUSIONS:A Data quality improvement initiative embedded within woreda clinical improvement collaborative improved accuracy of data used to monitor maternal and newborn health services in Ethiopia.

Project description:Blood culture contamination is a common problem in the emergency department (ED) that leads to unnecessary patient morbidity and health care costs. The study objective was to develop and evaluate the effectiveness of a quality improvement (QI) intervention for reducing blood culture contamination in an ED.The authors developed a QI intervention to reduce blood culture contamination in the ED and then evaluated its effectiveness in a prospective interrupted times series study. The QI intervention involved changing the technique of blood culture specimen collection from the traditional clean procedure to a new sterile procedure, with standardized use of sterile gloves and a new materials kit containing a 2% chlorhexidine skin antisepsis device, a sterile fenestrated drape, a sterile needle, and a procedural checklist. The intervention was implemented in a university-affiliated ED and its effect on blood culture contamination evaluated by comparing the biweekly percentages of blood cultures contaminated during a 48-week baseline period (clean technique) and 48-week intervention period (sterile technique), using segmented regression analysis with adjustment for secular trends and first-order autocorrelation. The goal was to achieve and maintain a contamination rate below 3%.During the baseline period, 321 of 7,389 (4.3%) cultures were contaminated, compared to 111 of 6,590 (1.7%) during the intervention period (p < 0.001). In the segmented regression model, the intervention was associated with an immediate 2.9% (95% confidence interval [CI] = 2.2% to 3.2%) absolute reduction in contamination. The contamination rate was maintained below 3% during each biweekly interval throughout the intervention period.A QI assessment of ED blood culture contamination led to development of a targeted intervention to convert the process of blood culture collection from a clean to a fully sterile procedure. Implementation of this intervention led to an immediate and sustained reduction of contamination in an ED with a high baseline contamination rate.

Project description:INTRODUCTION:An interrupted time series (ITS) design is an important observational design used to examine the effects of an intervention or exposure. This design has particular utility in public health where it may be impracticable or infeasible to use a randomised trial to evaluate health system-wide policies, or examine the impact of exposures (such as earthquakes). There have been relatively few studies examining the design characteristics and statistical methods used to analyse ITS designs. Further, there is a lack of guidance to inform the design and analysis of ITS studies.This is the first study in a larger project that aims to provide tools and guidance for researchers in the design and analysis of ITS studies. The objectives of this study are to (1) examine and report the design characteristics and statistical methods used in a random sample of contemporary ITS studies examining public health interventions or exposures that impact on health-related outcomes, and (2) create a repository of time series data extracted from ITS studies. Results from this study will inform the remainder of the project which will investigate the performance of a range of commonly used statistical methods, and create a repository of input parameters required for sample size calculation. METHODS AND ANALYSIS:We will collate 200 ITS studies evaluating public health interventions or the impact of exposures. ITS studies will be identified from a search of the bibliometric database PubMed between the years 2013 and 2017, combined with stratified random sampling. From eligible studies, we will extract study characteristics, details of the statistical models and estimation methods, effect metrics and parameter estimates. Further, we will extract the time series data when available. We will use systematic review methods in the screening, application of inclusion and exclusion criteria, and extraction of data. Descriptive statistics will be used to summarise the data. ETHICS AND DISSEMINATION:Ethics approval is not required since information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. A repository of data extracted from the published ITS studies will be made publicly available.

Dataset Information

Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review.

Publications

Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets