Dataset Information

Measurement equivalence in mixed mode surveys.

ABSTRACT: Surveys increasingly use mixed mode data collection (e.g., combining face-to-face and web) because this controls costs and helps to maintain good response rates. However, a combination of different survey modes in one study, be it cross-sectional or longitudinal, can lead to different kinds of measurement errors. For example, respondents in a face-to-face survey or a web survey may interpret the same question differently, and might give a different answer, just because of the way the question is presented. This effect of survey mode on the question-answer process is called measurement mode effect. This study develops methodological and statistical tools to identify the existence and size of mode effects in a mixed mode survey. In addition, it assesses the size and importance of mode effects in measurement instruments using a specific mixed mode panel survey (Netherlands Kinship Panel Study). Most measurement instruments in the NKPS are multi-item scales, therefore confirmatory factor analysis (CFA) will be used as the main analysis tool, using propensity score methods to correct for selection effects. The results show that the NKPS scales by and large have measurement equivalence, but in most cases only partial measurement equivalence. Controlling for respondent differences on demographic variables, and on scale scores from the previous uni-mode measurement occasion, tends to improve measurement equivalence, but not for all scales. The discussion ends with a review of the implications of our results for analyses employing these scales.

SUBMITTER: Hox JJ

PROVIDER: S-EPMC4318282 | biostudies-literature | 2015

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Measurement equivalence in mixed mode surveys.

Hox Joop J JJ De Leeuw Edith D ED Zijlmans Eva A O EA

Frontiers in psychology 20150205

Surveys increasingly use mixed mode data collection (e.g., combining face-to-face and web) because this controls costs and helps to maintain good response rates. However, a combination of different survey modes in one study, be it cross-sectional or longitudinal, can lead to different kinds of measurement errors. For example, respondents in a face-to-face survey or a web survey may interpret the same question differently, and might give a different answer, just because of the way the question is ...[more]

PMID: 25699002

Similar Datasets

Project description:BACKGROUND:The implementation of an Internet option in an existing public health interview survey using a mixed-mode design is attractive because of lower costs and faster data availability. Additionally, mixed-mode surveys can increase response rates and improve sample composition. However, mixed-mode designs can increase the risk of measurement error (mode effects). OBJECTIVE:This study aimed to determine whether the prevalence rates or mean values of self- and parent-reported health indicators for children and adolescents aged 0-17 years differ between self-administered paper-based questionnaires (SAQ-paper) and self-administered Web-based questionnaires (SAQ-Web), as well as between a single-mode control group and different mixed-mode groups. METHODS:Data were collected for a methodological pilot of the third wave of the "German Health Interview and Examination Survey for Children and Adolescents". Questionnaires were completed by parents or adolescents. A population-based sample of 11,140 children and adolescents aged 0-17 years was randomly allocated to 4 survey designs-a single-mode control group with paper-and-pencil questionnaires only (n=970 parents, n=343 adolescents)-and 3 mixed-mode designs, all of which offered Web-based questionnaire options. In the concurrent mixed-mode design, both questionnaires were offered at the same time (n=946 parents, n=290 adolescents); in the sequential mixed-mode design, the SAQ-Web was sent first, followed by the paper questionnaire along with a reminder (n=854 parents, n=269 adolescents); and in the preselect mixed-mode design, both options were offered and the respondents were asked to request the desired type of questionnaire (n=698 parents, n=292 adolescents). In total, 3468 questionnaires of parents of children aged 0-17 years (SAQ-Web: n=708; SAQ-paper: n=2760) and 1194 questionnaires of adolescents aged 11-17 years (SAQ-Web: n=299; SAQ-paper: n=895) were analyzed. Sociodemographic characteristics and a broad range of health indicators for children and adolescents were compared by survey design and data collection mode by calculating predictive margins from regression models. RESULTS:There were no statistically significant differences in sociodemographic characteristics or health indicators between the single-mode control group and any of the mixed-mode survey designs. Differences in sociodemographic characteristics between SAQ-Web and SAQ-paper were found. Web respondents were more likely to be male, have higher levels of education, and higher household income compared with paper respondents. After adjusting for sociodemographic characteristics, only one of the 38 analyzed health indicators showed different prevalence rates between the data collection modes, with a higher prevalence rate for lifetime alcohol consumption among the online-responding adolescents (P<.001). CONCLUSIONS:These results suggest that mode bias is limited in health interview surveys for children and adolescents using a mixed-mode design with Web-based and paper questionnaires.

Project description:BACKGROUND:Germany is a popular destination for immigrants, and migration has increased in recent years. It is therefore important to collect reliable data on migrants' health. The Robert Koch Institute, Berlin, Germany, has launched the Improving Health Monitoring in Migrant Populations (IMIRA) project to sustainably integrate migrant populations into health monitoring in Germany. OBJECTIVE:One of IMIRA's objectives is to implement a feasibility study (the IMIRA survey) that focuses on testing various interventions to increase the reachability of migrants with health interview surveys. Possible causes of nonresponse should be identified so as to increase participation in future surveys. METHODS:The survey target populations were Turkish, Polish, Romanian, Syrian, and Croatian migrants, who represent the biggest migrant groups living in Germany. We used probability sampling, using data from the registration offices in 2 states (Berlin and Brandenburg); we randomly selected 9068 persons by nationality in 7 sample points. We applied age (3 categories: 18-44, 45-64, and ?65 years) and sex strata. Modes and methods used to test their usability were culturally sensitive materials, online questionnaires, telephone interviews, personal contact, and personal interviews, using multilingual materials and interviewers. To evaluate the effectiveness of the interventions, we used an intervention group (group A) and a control group (group B). There were also focus groups with the interviewers to get more information about the participants' motivation. We used the European Health Interview Survey, with additional instruments on religious affiliation, experience of discrimination, and subjective social status. We evaluated results according to their final contact result (disposition code). RESULTS:We collected data from January to May 2018 in Berlin and Brandenburg, Germany. The survey had an overall response rate of 15.88% (1190/7494). However, final disposition codes varied greatly with regard to citizenship. In addition to the quantitative results, interviewers reported in the focus groups a "feeling of connectedness" to the participants due to the multilingual interventions. The interviewers were particularly positive about the home visits, because "if you are standing at the front door, you will be let in for sure." CONCLUSIONS:The IMIRA survey appraised the usability of mixed-mode or mixed-method approaches among migrant groups with a probability sample in 2 German states. When conducting the survey, we were confronted with issues regarding the translation of the questionnaire, as well as the validity of some instruments in the survey languages. A major result was that personal face-to-face contact was the most effective intervention to recruit our participants. We will implement the findings in the upcoming health monitoring study at the Robert Koch Institute.

Project description:ObjectiveRoutine measurement of patient safety from the patients' perspective receives increasing attention as an important component of safety measurement systems. The aim of this study was to examine patients' experience with patient safety in ambulatory care and the results' implications for routine patient safety measurement in ambulatory care.DesignCross-sectional mixed-mode survey.SettingGeneral practitioner and specialist practices.ParticipantsPatients aged >18 years seeking care in ambulatory care practices between February and June 2020.MethodsA 22-item-questionnaire was completed in the practice or at home either on paper or online. Multivariate logistic regression was used to analyse the influence of survey mode and patient characteristics on patient experience with patient safety.ResultsThe overall response rate was 71.1%. Most patients completed the questionnaire on site (76.6%) and on paper (96.1%). Between 30.1% to 68.5% of the respondents report the most positive option for patient experience with the main domains of patient safety. A total of 2.9% of patients reported having experienced a patient-safety event (PSE) during the last 12 months. Patients who filled in the questionnaire off site were more likely to report negative experiences for the scales communication & information (OR 1.2, 95% CI 1.0-1.5), rapport & participation (OR 1.4, 95% CI 1.1-1.7) and access (OR 1.3, 95% CI 0.9-1.4) than those who completed it on site. Those who chose a paper questionnaire were more likely to report negative experiences for all five scales compared to web responders.ConclusionRoutine measurement of patient experience with factors contributing to the occurrence of PSEs can achieve high response rates by offering flexible participation options. Results gained from mixed-mode surveys need to take mode-effects into account when interpreting and using the results. Further research is needed in how to adequately assess number and type of experienced events in routine measurements.

Project description:BackgroundHealth interview surveys are important data sources for empirical research in public health. However, the diversity of methods applied, such as in the mode of data collection, make it difficult to compare results across surveys, time, or countries. The aim of this study was to explore whether the prevalence rates of health-related indicators amongst adults differ when self-administered paper mail questionnaires (SAQ-Paper), self-administered web surveys (SAQ-Web), and computer-assisted telephone interviews (CATI) are used for data collection in a health survey.MethodsData were obtained from a population-based mixed-mode health interview survey of adults in Germany carried out within the 'German Health Update' (GEDA) study. Data were collected either by SAQ-Paper (n = 746), SAQ-Web (n = 414), or CATI (n = 411). Predictive margins from logistic regression models were used to estimate the prevalence rates of chronic conditions, subjective health, mental health, psychosocial factors, and health behaviours, adjusted for the socio-demographic characteristics of each mode group.ResultsSocio-demographic characteristics were found to differ significantly between study participants who responded by SAQ-Paper, SAQ-Web, and CATI. Crude prevalence rates for health-related indicators also showed significant variation across all three survey modes. After adjusting for socio-demographic factors though, significant differences in prevalence rates between the two self-administered modes (SAQ-Paper and SAQ-Web) were found in only 2 out of the 19 health-related indicators studied. The differences between CATI and the two self-administered modes remained significant however, especially for indicators of mental and psychosocial health and self-reported sporting activity.ConclusionsThe findings of this study indicate that prevalence rates obtained from health interview surveys can vary with the mode of data collection, primarily between interviewer and self-administered modes. Hence, the type of survey mode used should be considered when comparing results from different health surveys. Mixing self-administered modes, such as paper-based questionnaires and web surveys, may be a combination to minimize mode differences in mixed-mode health interview surveys.

Dataset Information

Measurement equivalence in mixed mode surveys.

Publications

Measurement equivalence in mixed mode surveys.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets