Dataset Information

Bias due to sample selection in propensity score matching for a supportive housing program evaluation in New York City.

ABSTRACT:

Objectives

Little is known about influences of sample selection on estimation in propensity score matching. The purpose of the study was to assess potential selection bias using one-to-one greedy matching versus optimal full matching as part of an evaluation of supportive housing in New York City (NYC).

Study design and settings

Data came from administrative data for 2 groups of applicants who were eligible for an NYC supportive housing program in 2007-09, including chronically homeless adults with a substance use disorder and young adults aging out of foster care. We evaluated the 2 matching methods in their ability to balance covariates and represent the original population, and in how those methods affected outcomes related to Medicaid expenditures.

Results

In the population with a substance use disorder, only optimal full matching performed well in balancing covariates, whereas both methods created representative populations. In the young adult population, both methods balanced covariates effectively, but only optimal full matching created representative populations. In the young adult population, the impact of the program on Medicaid expenditures was attenuated when one-to-one greedy matching was used, compared with optimal full matching.

Conclusion

Given covariate balancing with both methods, attenuated program impacts in the young adult population indicated that one-to-one greedy matching introduced selection bias.

SUBMITTER: Lim S

PROVIDER: S-EPMC4195658 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Bias due to sample selection in propensity score matching for a supportive housing program evaluation in New York City.

Lim Sungwoo S Marcus Sue M SM Singh Tejinder P TP Harris Tiffany G TG Levanon Seligson Amber A

PloS one 20141013 10

<h4>Objectives</h4>Little is known about influences of sample selection on estimation in propensity score matching. The purpose of the study was to assess potential selection bias using one-to-one greedy matching versus optimal full matching as part of an evaluation of supportive housing in New York City (NYC).<h4>Study design and settings</h4>Data came from administrative data for 2 groups of applicants who were eligible for an NYC supportive housing program in 2007-09, including chronically ho ...[more]

PMID: 25310449

Similar Datasets

Project description:ObjectivesAfter the Great East Japan Earthquake (GEJE) of 2011, many survivors have been forced to live in prefabricated temporary housing, which is uncomfortable and insufficiently durable for permanent living. Public reconstruction housing has been built to improve their living conditions; however, those moving have to rebuild personal relationships and adapt to a new environment. This study examined whether survivors moving to public reconstruction housing became more socially isolated than those remaining in prefabricated temporary housing.Design, setting and participantsSelf-report questionnaire data collected in 2015 (4 years after the GEJE) were used as the baseline for follow-up surveys in 2016 and 2017, as many survivors moved from prefabricated temporary housing to public reconstruction housing from 2015. We analysed longitudinal data from 393 survivors, distinguishing those who moved to public reconstruction housing during the 5th year after the disaster from those who remained in prefabricated temporary housing. Participants were assessed using the Lubben Social Network Scale-6 (LSNS-6) in all three surveys, with social isolation defined by a score of <12/30. To reduce the effect of selection bias, propensity score analysis was performed (178 of 393 participants were retained). We used a generalised estimated equation to evaluate the association between moving from prefabricated temporary housing to public reconstruction housing and changes in social isolation over 2 years.ResultsLSNS-6 scores of the reconstruction housing group were worse than those of the prefabricated housing group between 4 and 6 years after the GEJE (P=0.006). Over the same period, social isolation worsened in the reconstruction housing group but improved in the prefabricated housing group (P=0.002).ConclusionsSocial isolation should be monitored while supporting survivors who moved to public reconstruction housing, and further longitudinal research is needed to clarify the risk of social isolation.

Project description:BackgroundPropensity score matching is a statistical method that is often used to make inferences on the treatment effects in observational studies. In recent years, there has been widespread use of the technique in the cardiothoracic surgery literature to evaluate to potential benefits of new surgical therapies or procedures. However, the small sample size and the strong dependence of the treatment assignment on the baseline covariates that often characterize these studies make such an evaluation challenging from a statistical point of view. In such settings, the use of propensity score matching in combination with oversampling and replacement may provide a solution to these issues by increasing the initial sample size of the study and thus improving the statistical power that is needed to detect the effect of interest. In this study, we review the use of propensity score matching in combination with oversampling and replacement in small sample size settings.MethodsWe performed a series of Monte Carlo simulations to evaluate how the sample size, the proportion of treated, and the assignment mechanism affect the performances of the proposed approaches. We assessed the performances with overall balance, relative bias, root mean squared error and nominal coverage. Moreover, we illustrate the methods using a real case study from the cardiac surgery literature.ResultsMatching without replacement produced estimates with lower bias and better nominal coverage than matching with replacement when 1:1 matching was considered. In contrast to that, matching with replacement showed better balance, relative bias, and root mean squared error than matching without replacement for increasing levels of oversampling. The best nominal coverage was obtained by using the estimator that accounts for uncertainty in the matching procedure on sets of units obtained after matching with replacement.ConclusionsThe use of replacement provides the most reliable treatment effect estimates and that no more than 1 or 2 units from the control group should be matched to each treated observation. Moreover, the variance estimator that accounts for the uncertainty in the matching procedure should be used to estimate the treatment effect.

Dataset Information

Bias due to sample selection in propensity score matching for a supportive housing program evaluation in New York City.

Objectives

Study design and settings

Results

Conclusion

Publications

Bias due to sample selection in propensity score matching for a supportive housing program evaluation in New York City.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets