Dataset Information

Measuring Model Misspecification: Application to Propensity Score Methods with Complex Survey Data.

ABSTRACT: Model misspecification is a potential problem for any parametric-model based analysis. However, the measurement and consequences of model misspecification have not been well formalized in the context of causal inference. A measure of model misspecification is proposed, and the consequences of model misspecification in non-experimental causal inference methods are investigated. The metric is then used to explore which estimators are more sensitive to misspecification of the outcome and/or treatment assignment model. Three frequently used estimators of the treatment effect are considered, all of which rely on the propensity score: (1) full matching, (2) 1:1 nearest neighbor matching, and (3) weighting. The performance of these estimators is evaluated under two different sampling designs: (1) simple random sampling (SRS) and (2) a two-stage stratified survey. As the degree of misspecification of either the propensity score or outcome model increases, so does the bias and the root mean square error, while the coverage decreases. Results are similar for the simple random sample and a complex survey design.

SUBMITTER: Lenis D

PROVIDER: S-EPMC6034692 | biostudies-literature | 2018 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Measuring Model Misspecification: Application to Propensity Score Methods with Complex Survey Data.

Lenis David D Ackerman Benjamin B Stuart Elizabeth A EA

Computational statistics & data analysis 20180522

Model misspecification is a potential problem for any parametric-model based analysis. However, the measurement and consequences of model misspecification have not been well formalized in the context of causal inference. A measure of model misspecification is proposed, and the consequences of model misspecification in non-experimental causal inference methods are investigated. The metric is then used to explore which estimators are more sensitive to misspecification of the outcome and/or treatme ...[more]

PMID: 29988991

Similar Datasets

Project description:BackgroundLow-income and middle-income countries (LMICs) seek to better utilize household and health facility survey data for monitoring and evaluation, as well as for health program planning. However, analysis of this complex survey data are complicated. In Tanzania, the National Evaluation Platform project sought to analyze Demographic and Health Survey (DHS) data and Service Provision Assessment (SPA) data as part of an evaluation of the national One Plan for Maternal and Child Health. To support this evaluation, we used this survey data to answer two key methodological questions: 1) what are the benefits and costs of using sampling weights in rate estimation; and 2) what is the best method for calculating standard errors in these two surveys?MethodsWe conducted a simulation study for each methodologic question. The first simulation study assessed the benefits and costs of using sampling weights in rate estimation. This simulation used weighted and unweighted estimates and examined bias, variance, and the mean squared error (MSE). The second simulation study assessed the best method for calculating standard errors comparing cluster bootstrapped variance estimation, design based asymptotic variance with one level (svy1), and design based asymptotic variance with three levels (svy3). We compared coverage probability and confidence interval length.ResultsOur results showed that although weighted estimates were less biased, unweighted estimates were less variable. The weighted estimates had a lower MSE, indicating that the effect of the bias trade-off was greater than the effect of the variance trade-off for most indicators assessed. The best performer for variance estimation was the cluster bootstrap method, followed by the svy3 method. The svy1 method was the worst performer for most indicators assessed.ConclusionsAs complex survey data become more widely used for policymaking in LMICs, there is a need for guidance on the best methods for analyzing this data. The standard of practice has been a design-based analysis using survey weights and the single-level svy method for calculating standard errors. This study puts forth an alternative approach to analysis. In addition, this study offers practical guidance on determining the best method for analysis of complex survey data.

Dataset Information

Measuring Model Misspecification: Application to Propensity Score Methods with Complex Survey Data.

Publications

Measuring Model Misspecification: Application to Propensity Score Methods with Complex Survey Data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets