Dataset Information

Comparison of response patterns in different survey designs: a longitudinal panel with mixed-mode and online-only design.

ABSTRACT:

Background

Increasing availability of the Internet allows using only online data collection for more epidemiological studies. We compare response patterns in a population-based health survey using two survey designs: mixed-mode (choice between paper-and-pencil and online questionnaires) and online-only design (without choice).

Methods

We used data from a longitudinal panel, the Hygiene and Behaviour Infectious Diseases Study (HaBIDS), conducted in 2014/2015 in four regions in Lower Saxony, Germany. Individuals were recruited using address-based probability sampling. In two regions, individuals could choose between paper-and-pencil and online questionnaires. In the other two regions, individuals were offered online-only participation. We compared sociodemographic characteristics of respondents who filled in all panel questionnaires between the mixed-mode group (n = 1110) and the online-only group (n = 482). Using 134 items, we performed multinomial logistic regression to compare responses between survey designs in terms of type (missing, "do not know" or valid response) and ordinal regression to compare responses in terms of content. We applied the false discovery rates (FDR) to control for multiple testing and investigated effects of adjusting for sociodemographic characteristic. For validation of the differential response patterns between mixed-mode and online-only, we compared the response patterns between paper and online mode among the respondents in the mixed-mode group in one region (n = 786).

Results

Respondents in the online-only group were older than those in the mixed-mode group, but both groups did not differ regarding sex or education. Type of response did not differ between the online-only and the mixed-mode group. Survey design was associated with different content of response in 18 of the 134 investigated items; which decreased to 11 after adjusting for sociodemographic variables. In the validation within the mixed-mode, only two of those were among the 11 significantly different items. The probability of observing by chance the same two or more significant differences in this setting was 22%.

Conclusions

We found similar response patterns in both survey designs with only few items being answered differently, likely attributable to chance. Our study supports the equivalence of the compared survey designs and suggests that, in the studied setting, using online-only design does not cause strong distortion of the results.

SUBMITTER: Rubsamen N

PROVIDER: S-EPMC5361716 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparison of response patterns in different survey designs: a longitudinal panel with mixed-mode and online-only design.

Rübsamen Nicole N Akmatov Manas K MK Castell Stefanie S Karch André A Mikolajczyk Rafael T RT

Emerging themes in epidemiology 20170321

<h4>Background</h4>Increasing availability of the Internet allows using only online data collection for more epidemiological studies. We compare response patterns in a population-based health survey using two survey designs: mixed-mode (choice between paper-and-pencil and online questionnaires) and online-only design (without choice).<h4>Methods</h4>We used data from a longitudinal panel, the Hygiene and Behaviour Infectious Diseases Study (HaBIDS), conducted in 2014/2015 in four regions in Lowe ...[more]

PMID: 28344629

Similar Datasets

Project description:Background:Population-based surveys currently face the problem of decreasing response rates. Mixed-mode designs are now being implemented more often to account for this, to improve sample composition and to reduce overall costs. This study examines whether a concurrent or sequential mixed-mode design achieves better results on a number of indicators of survey quality. Methods:Data were obtained from a population-based health interview survey of adults in Germany that was conducted as a methodological pilot study as part of the German Health Update (GEDA). Participants were randomly allocated to one of two surveys; each of the surveys had a different design. In the concurrent mixed-mode design (n?=?617) two types of self-administered questionnaires (SAQ-Web and SAQ-Paper) and computer-assisted telephone interviewing were offered simultaneously to the respondents along with the invitation to participate. In the sequential mixed-mode design (n?=?561), SAQ-Web was initially provided, followed by SAQ-Paper, with an option for a telephone interview being sent out together with the reminders at a later date. Finally, this study compared the response rates, sample composition, health indicators, item non-response, the scope of fieldwork and the costs of both designs. Results:No systematic differences were identified between the two mixed-mode designs in terms of response rates, the socio-demographic characteristics of the achieved samples, or the prevalence rates of the health indicators under study. The sequential design gained a higher rate of online respondents. Very few telephone interviews were conducted for either design. With regard to data quality, the sequential design (which had more online respondents) showed less item non-response. There were minor differences between the designs in terms of their costs. Postage and printing costs were lower in the concurrent design, but labour costs were lower in the sequential design. No differences in health indicators were found between the two designs. Modelling these results for higher response rates and larger net sample sizes indicated that the sequential design was more cost and time-effective. Conclusions:This study contributes to the research available on implementing mixed-mode designs as part of public health surveys. Our findings show that SAQ-Paper and SAQ-Web questionnaires can be combined effectively. Sequential mixed-mode designs with higher rates of online respondents may be of greater benefit to studies with larger net sample sizes than concurrent mixed-mode designs.

Project description:BACKGROUND:The implementation of an Internet option in an existing public health interview survey using a mixed-mode design is attractive because of lower costs and faster data availability. Additionally, mixed-mode surveys can increase response rates and improve sample composition. However, mixed-mode designs can increase the risk of measurement error (mode effects). OBJECTIVE:This study aimed to determine whether the prevalence rates or mean values of self- and parent-reported health indicators for children and adolescents aged 0-17 years differ between self-administered paper-based questionnaires (SAQ-paper) and self-administered Web-based questionnaires (SAQ-Web), as well as between a single-mode control group and different mixed-mode groups. METHODS:Data were collected for a methodological pilot of the third wave of the "German Health Interview and Examination Survey for Children and Adolescents". Questionnaires were completed by parents or adolescents. A population-based sample of 11,140 children and adolescents aged 0-17 years was randomly allocated to 4 survey designs-a single-mode control group with paper-and-pencil questionnaires only (n=970 parents, n=343 adolescents)-and 3 mixed-mode designs, all of which offered Web-based questionnaire options. In the concurrent mixed-mode design, both questionnaires were offered at the same time (n=946 parents, n=290 adolescents); in the sequential mixed-mode design, the SAQ-Web was sent first, followed by the paper questionnaire along with a reminder (n=854 parents, n=269 adolescents); and in the preselect mixed-mode design, both options were offered and the respondents were asked to request the desired type of questionnaire (n=698 parents, n=292 adolescents). In total, 3468 questionnaires of parents of children aged 0-17 years (SAQ-Web: n=708; SAQ-paper: n=2760) and 1194 questionnaires of adolescents aged 11-17 years (SAQ-Web: n=299; SAQ-paper: n=895) were analyzed. Sociodemographic characteristics and a broad range of health indicators for children and adolescents were compared by survey design and data collection mode by calculating predictive margins from regression models. RESULTS:There were no statistically significant differences in sociodemographic characteristics or health indicators between the single-mode control group and any of the mixed-mode survey designs. Differences in sociodemographic characteristics between SAQ-Web and SAQ-paper were found. Web respondents were more likely to be male, have higher levels of education, and higher household income compared with paper respondents. After adjusting for sociodemographic characteristics, only one of the 38 analyzed health indicators showed different prevalence rates between the data collection modes, with a higher prevalence rate for lifetime alcohol consumption among the online-responding adolescents (P<.001). CONCLUSIONS:These results suggest that mode bias is limited in health interview surveys for children and adolescents using a mixed-mode design with Web-based and paper questionnaires.

Project description:BackgroundPatient and consumer access to eHealth information is of crucial importance because of its role in patient-centered medicine and to improve knowledge about general aspects of health and medical topics.ObjectivesThe objectives were to analyze and compare eHealth search patterns in a private (United States) and a public (United Kingdom) health care market.MethodsA new taxonomy of eHealth websites is proposed to organize the largest eHealth websites. An online measurement framework is developed that provides a precise and detailed measurement system. Online panel data are used to accurately track and analyze detailed search behavior across 100 of the largest eHealth websites in the US and UK health care markets.ResultsThe health, medical, and lifestyle categories account for approximately 90% of online activity, and e-pharmacies, social media, and professional categories account for the remaining 10% of online activity. Overall search penetration of eHealth websites is significantly higher in the private (United States) than the public market (United Kingdom). Almost twice the number of eHealth users in the private market have adopted online search in the health and lifestyle categories and also spend more time per website than those in the public market. The use of medical websites for specific conditions is almost identical in both markets. The allocation of search effort across categories is similar in both the markets. For all categories, the vast majority of eHealth users only access one website within each category. Those that conduct a search of two or more websites display very narrow search patterns. All users spend relatively little time on eHealth, that is, 3-7 minutes per website.ConclusionsThe proposed online measurement framework exploits online panel data to provide a powerful and objective method of analyzing and exploring eHealth behavior. The private health care system does appear to have an influence on eHealth search behavior in terms of search penetration and time spent per website in the health and lifestyle categories. Two explanations are offered: (1) the personal incentive of medical costs in the private market incentivizes users to conduct online search; and (2) health care information is more easily accessible through health care professionals in the United Kingdom compared with the United States. However, the use of medical websites is almost identical, suggesting that patients interested in a specific condition have a motivation to search and evaluate health information, irrespective of the health care market. The relatively low level of search in terms of the number of websites accessed and the average time per website raise important questions about the actual level of patient informedness in both the markets. Areas for future research are outlined.

Project description:BackgroundHealth interview surveys are important data sources for empirical research in public health. However, the diversity of methods applied, such as in the mode of data collection, make it difficult to compare results across surveys, time, or countries. The aim of this study was to explore whether the prevalence rates of health-related indicators amongst adults differ when self-administered paper mail questionnaires (SAQ-Paper), self-administered web surveys (SAQ-Web), and computer-assisted telephone interviews (CATI) are used for data collection in a health survey.MethodsData were obtained from a population-based mixed-mode health interview survey of adults in Germany carried out within the 'German Health Update' (GEDA) study. Data were collected either by SAQ-Paper (n = 746), SAQ-Web (n = 414), or CATI (n = 411). Predictive margins from logistic regression models were used to estimate the prevalence rates of chronic conditions, subjective health, mental health, psychosocial factors, and health behaviours, adjusted for the socio-demographic characteristics of each mode group.ResultsSocio-demographic characteristics were found to differ significantly between study participants who responded by SAQ-Paper, SAQ-Web, and CATI. Crude prevalence rates for health-related indicators also showed significant variation across all three survey modes. After adjusting for socio-demographic factors though, significant differences in prevalence rates between the two self-administered modes (SAQ-Paper and SAQ-Web) were found in only 2 out of the 19 health-related indicators studied. The differences between CATI and the two self-administered modes remained significant however, especially for indicators of mental and psychosocial health and self-reported sporting activity.ConclusionsThe findings of this study indicate that prevalence rates obtained from health interview surveys can vary with the mode of data collection, primarily between interviewer and self-administered modes. Hence, the type of survey mode used should be considered when comparing results from different health surveys. Mixing self-administered modes, such as paper-based questionnaires and web surveys, may be a combination to minimize mode differences in mixed-mode health interview surveys.

Project description:BackgroundKnowing about predictors of attrition in a panel is important to initiate early measures against loss of participants. We investigated attrition in both early and late phase of an online panel with special focus on preferences regarding mode of participation.MethodsWe used data from the HaBIDS panel that was designed to investigate knowledge, attitudes, and practice regarding infections in the German general population. HaBIDS was divided into two phases: an initial phase when some participants could choose their preferred mode of participation (paper-and-pencil or online) and an extended phase when participants were asked to become members of an online panel that was not limited regarding its duration (i.e. participants initially preferring paper questionnaires switched to online participation). Using competing risks regression, we investigated two types of attrition (formal withdrawal and discontinuation without withdrawal) among online participants, separately for both phases. As potential predictors of attrition, we considered sociodemographic characteristics, physical and mental health as well as auxiliary information describing the survey process, and, in the extended phase, initial mode preference.ResultsIn the initial phase, higher age and less frequent Internet usage predicted withdrawal, while younger age, higher stress levels, delay in returning the consent form, and need for receiving reminder emails predicted discontinuation. In the extended phase, only need for receiving reminder emails predicted discontinuation. Numbers of withdrawal in the extended phase were too small for analysis. Initial mode preference did not predict attrition in the extended phase. Besides age, there was no evidence of differential attrition by sociodemographic factors in any phase.ConclusionsPredictors of attrition were similar in both phases of the panel, but they differed by type of attrition (withdrawal vs. discontinuation). Sociodemographic characteristics only played a minor role for both types of attrition. Need for receiving a reminder was the strongest predictor of discontinuation in any phase, but no predictor of withdrawal. We found predictors of attrition, which can be identified already in the early phase of a panel so that countermeasures (e.g. special incentives) can be taken.

Dataset Information

Comparison of response patterns in different survey designs: a longitudinal panel with mixed-mode and online-only design.

Background

Methods

Results

Conclusions

Publications

Comparison of response patterns in different survey designs: a longitudinal panel with mixed-mode and online-only design.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets