Dataset Information

Estimation of unobservable selection effects in on-line surveys through propensity score matching: An application to public acceptance of healthy eating policies.

ABSTRACT: The use of model-based propensity scores as matching tools opens the way to the indirect estimation of mode-related measurement effects and selection effects in web surveys, including a component of selection that cannot be traced back to observable characteristics. By matching and comparing respondents from real independent surveys that use the same questionnaire, but different administration modes, it becomes possible to isolate the selection effect induced by unobservable (or unobserved) respondent characteristics. This study applies a stratification matching algorithm to compare a web survey from a proprietary panel with a computer-assisted telephone survey based on random digit-dialing. The experiment is run in two countries (UK and Italy) to check for consistencies across different cultures and different internet penetration rates. The application to the elicitation of support for healthy eating policies indicates large and significant measurement and selection effects. After controlling for differences in the observed characteristics of respondents and the intensity of internet use, findings suggest that web surveys record lower support and higher neutrality. Similarly, after controlling for administration mode and observed respondent characteristics, internet users are less likely to state support compared to non-users. This suggests that unobserved characteristics play a major role, and post-stratification weighting is not a sufficient countermeasure. As demonstrated by the cross-country comparison, rising internet penetration rates are not a guarantee against this type of error, as disparities in these unobserved characteristics are likely to increase at the same time.

SUBMITTER: Capacci S

PROVIDER: S-EPMC5903641 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Estimation of unobservable selection effects in on-line surveys through propensity score matching: An application to public acceptance of healthy eating policies.

Capacci Sara S Mazzocchi Mario M Brasini Sergio S

PloS one 20180417 4

The use of model-based propensity scores as matching tools opens the way to the indirect estimation of mode-related measurement effects and selection effects in web surveys, including a component of selection that cannot be traced back to observable characteristics. By matching and comparing respondents from real independent surveys that use the same questionnaire, but different administration modes, it becomes possible to isolate the selection effect induced by unobservable (or unobserved) resp ...[more]

PMID: 29664975

Similar Datasets

Project description:BackgroundThe diagnostic process is a key element of medicine but it is complex and prone to errors. Infectious diseases are one of the three categories of diseases in which diagnostic errors can be most harmful to patients. In this study we aimed to estimate the effect of initial misdiagnosis of the source of infection in patients with bacteraemia on 14 day mortality using propensity score methods to adjust for confounding.MethodsData from a previously described longitudinal cohort of patients diagnosed with monobacterial bloodstream infection (BSI) at the Leiden University Medical Centre (LUMC) between 2013 and 2015 were used. Propensity score matching and inversed probability of treatment weighting (IPTW) were applied to correct for confounding. The average treatment effect on the treated (ATT), which in this study was the average effect of initial misdiagnosis on the misdiagnosed (AEMM), was estimated. Methodological issues that were encountered when applying propensity score methods were addressed by performing additional sensitivity analyses. Sensitivity analyses consisted of varying caliper in propensity score matching and using different truncated weights in inversed probability of treatment weighting.ResultsData of 887 patients were included in the study. Propensity scores ranged between 0.015 and 0.999 and 80 patients (9.9%) had a propensity score > 0.95. In the matched analyses, 35 of the 171 misdiagnosed patients died within 14 days (20.5%), versus 10 of the 171 correctly diagnosed patients (5.8%), yielding a difference of 14.6% (7.6%; 21.6%). In the total group of patients, the observed percentage of patients with an incorrect initial diagnosis that died within 14 days was 19.8% while propensity score reweighting estimated that their probability of dying would have been 6.5%, if they had been correctly diagnosed (difference 13.3% (95% CI 6.9%;19.6%)). After adjustment for all variables that showed disbalance in the propensity score a difference of 13.7% (7.4%; 19.9%) was estimated. Sensitivity analyses yielded similar results. However, performing weighted analyses without truncation yielded unstable results.ConclusionThus, we observed a substantial increase of 14 day mortality in initially misdiagnosed patients. Furthermore, several patients received propensity scores extremely close to one and were almost sure to be initially misdiagnosed.

Project description:BackgroundLoneliness and unemployment are each detrimental to health and well-being. Recent evidence suggests a potential bidirectional relationship between loneliness and unemployment in working age individuals. As most existing research focuses on the outcomes of unemployment, this paper seeks to understand the impact of loneliness on unemployment, potential interaction with physical health, and assess bidirectionality in the working age population.MethodsThis study utilised data from waves 9 (2017-19) and 10 (2018-2020) of the Understanding Society UK Household Longitudinal Study. Nearest-neighbour probit propensity score matching with at least one match was used to infer causality by mimicking randomisation. Analysis was conducted in three steps: propensity score estimation; matching; and stratification. Propensity scores were estimated controlling for age, gender, ethnicity, education, marital status, household composition, number of own children in household and region. Findings were confirmed in panel data random effect models, and heterogeneous treatment effects assessed by the matching-smoothing method.ResultsExperience of loneliness in at least one wave increased the probability of being unemployed in wave 10 by 17.5 [95%CI: 14.8, 20.2] percentage points. Subgroup analysis revealed a greater effect from sustained than transitory loneliness. Further exploratory analysis identified a positive average treatment effect, of smaller magnitude, for unemployment on loneliness suggesting bidirectionality in the relationship. The impact of loneliness on unemployment was further exacerbated by interaction with physical health.ConclusionsThis is the first study to directly consider the potentially bidirectional relationship between loneliness and unemployment through analysis of longitudinal data from a representative sample of the working age population. Findings reinforce the need for greater recognition of wider societal impacts of loneliness. Given the persisting and potentially scarring effects of both loneliness and unemployment on health and the economy, prevention of both experiences is key. Decreased loneliness could mitigate unemployment, and employment abate loneliness, which may in turn relate positively to other factors including health and quality of life. Thus, particular attention should be paid to loneliness with additional support from employers and government to improve health and well-being.

Dataset Information

Estimation of unobservable selection effects in on-line surveys through propensity score matching: An application to public acceptance of healthy eating policies.

Publications

Estimation of unobservable selection effects in on-line surveys through propensity score matching: An application to public acceptance of healthy eating policies.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets