Dataset Information

Matching and Regression to the Mean in Difference-in-Differences Analysis.

ABSTRACT:

Objective

To demonstrate regression to the mean bias introduced by matching on preperiod variables in difference-in-differences studies.

Data sources

Simulated data.

Study design

We performed a Monte Carlo simulation to estimate the effect of a placebo intervention on simulated longitudinal data for units in treatment and control groups using unmatched and matched difference-in-differences analyses. We varied the preperiod level and trend differences between the treatment and control groups, and the serial correlation of the matching variables. We assessed estimator bias as the mean absolute deviation of estimated program effects from the true value of zero.

Principal findings

When preperiod outcome level is correlated with treatment assignment, an unmatched analysis is unbiased, but matching units on preperiod outcome levels produces biased estimates. The bias increases with greater preperiod level differences and weaker serial correlation in the outcome. This problem extends to matching on preperiod level of a time-varying covariate. When treatment assignment is correlated with preperiod trend only, the unmatched analysis is biased, and matching units on preperiod level or trend does not introduce additional bias.

Conclusions

Researchers should be aware of the threat of regression to the mean when constructing matched samples for difference-in-differences. We provide guidance on when to incorporate matching in this study design.

SUBMITTER: Daw JR

PROVIDER: S-EPMC6232412 | biostudies-literature | 2018 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Matching and Regression to the Mean in Difference-in-Differences Analysis.

Daw Jamie R JR Hatfield Laura A LA

Health services research 20180629 6

<h4>Objective</h4>To demonstrate regression to the mean bias introduced by matching on preperiod variables in difference-in-differences studies.<h4>Data sources</h4>Simulated data.<h4>Study design</h4>We performed a Monte Carlo simulation to estimate the effect of a placebo intervention on simulated longitudinal data for units in treatment and control groups using unmatched and matched difference-in-differences analyses. We varied the preperiod level and trend differences between the treatment a ...[more]

PMID: 29957834

Similar Datasets

Project description:To evaluate a multidisciplinary team (MDT) case management intervention, at the individual (direct effects of intervention) and practice levels (potential spillover effects).Difference-in-differences design with multiple intervention start dates, analysing hospital admissions data. In secondary analyses, we stratified individual-level results by risk score.Single clinical commissioning group (CCG) in the UK's National Health Service (NHS).At the individual level, we matched 2049 intervention patients using propensity scoring one-to-one with control patients. At the practice level, 30 practices were compared using a natural experiment through staged implementation.Practice Integrated Care Teams (PICTs), using MDT case management of high-risk patients together with a summary record of care versus usual care.Primary measures of intervention effects were accident and emergency (A&E) visits; inpatient non-elective stays, 30-day re-admissions; inpatient elective stays; outpatient visits; and admissions for ambulatory care sensitive conditions. Secondary measures included inpatient length of stay; total cost of secondary care services; and patient satisfaction (at the practice level only).At the individual level, we found slight, clinically trivial increases in inpatient non-elective admissions (+0.01 admissions per patient per month; 95% CI 0.00 to 0.01. Effect size (ES): 0.02) and 30-day re-admissions (+0.00; 0.00 to 0.01. ES: 0.03). We found no indication that highest risk patients benefitted more from the intervention. At the practice level, we found a small decrease in inpatient non-elective admissions (-0.63 admissions per 1000 patients per month; -1.17 to -0.09. ES: -0.24). However, this result did not withstand a robustness check; the estimate may have absorbed some differences in underlying practice trends.The intervention does not meet its primary aim, and the clinical significance and cost-effectiveness of these small practice-level effects is debatable. There is an ongoing need to develop effective ways to reduce unnecessary attendances in secondary care for the high-risk population.

Project description:OBJECTIVE:To evaluate the association between US state policies that establish age 18 or 21 years as the minimum purchaser age for the sale of handguns and adolescent suicide rate. DESIGN:Regression discontinuity and difference-in-differences analyses. SETTING:46 US states without policy changes between 2001 and 2017; Missouri and South Carolina, which lowered the age for handgun sales in 2007 and 2008, respectively; and West Virginia and Wyoming, which increased the age for handgun sales in 2010. PARTICIPANTS:Adolescents aged 13 to 20 years(554?461?961 from 2001 to 2017) in the regression discontinuity analysis, and adolescents aged 18 to 20 years (168?934?041 from 2002 to 2014) in the main difference-in-differences analysis. MAIN OUTCOME MEASURE:Suicide rate per 100?000 adolescents. RESULTS:In the regression discontinuity analysis, state policies that limited the sale of handguns to those aged 18 or older (relative to 21 or older) were associated with an increase in suicide rate among adolescents aged 18 to 20 years equivalent to 344 additional suicides in each state where they were in place between 2001 and 2017. In the difference-in-differences analysis, state policies that limited the sale of handguns to those aged 21 or older were associated with 1.91 fewer suicides per 100?000 adolescents aged 18 to 20 years (95% confidence interval -3.13 to -0.70, permutation adjusted P=0.025). In the difference-in-differences analysis, there were 1.83 fewer firearm related suicides per 100?000 adolescents (-2.66 to -1.00, permutation adjusted P=0.002), with no association between age 21 handgun sales policies and non-firearm related suicides. Separate event study estimates indicated increases in suicide rates in states that lowered the age of handgun sales, with no association in states that increased the age of handgun sales. CONCLUSIONS:A clear discontinuity was shown in the suicide rate by age at age 18 in states that limited the sale of handguns to individuals aged 18 or older. State policies to limit the sale of handguns to individuals aged 21 or older were associated with a reduction in suicide rates among adolescents. Increases in suicide rates were observed after states lowered the age of handgun sales, but no effect was found in states that increased the age of handgun sales.

Dataset Information

Matching and Regression to the Mean in Difference-in-Differences Analysis.

Objective

Data sources

Study design

Principal findings

Conclusions

Publications

Matching and Regression to the Mean in Difference-in-Differences Analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets