Dataset Information

Survey Satisficing Inflates Reliability and Validity Measures: An Experimental Comparison of College and Amazon Mechanical Turk Samples.

ABSTRACT: This study examined the predictors and psychometric outcomes of survey satisficing, wherein respondents provide quick, "good enough" answers (satisficing) rather than carefully considered answers (optimizing). We administered surveys to university students and respondents-half of whom held college degrees-from a for-pay survey website, and we used an experimental method to randomly assign the participants to survey formats, which presumably differed in task difficulty. Based on satisficing theory, we predicted that ability, motivation, and task difficulty would predict satisficing behavior and that satisficing would artificially inflate internal consistency reliability and both convergent and discriminant validity correlations. Indeed, results indicated effects for task difficulty and motivation in predicting survey satisficing, and satisficing in the first part of the study was associated with improved internal consistency reliability and convergent validity but also worse discriminant validity in the second part of the study. Implications for research designs and improvements are discussed.

SUBMITTER: Hamby T

PROVIDER: S-EPMC5965608 | biostudies-other | 2016 Dec

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Survey Satisficing Inflates Reliability and Validity Measures: An Experimental Comparison of College and Amazon Mechanical Turk Samples.

Hamby Tyler T Taylor Wyn W

Educational and psychological measurement 20160123 6

This study examined the predictors and psychometric outcomes of survey satisficing, wherein respondents provide quick, "good enough" answers (satisficing) rather than carefully considered answers (optimizing). We administered surveys to university students and respondents-half of whom held college degrees-from a for-pay survey website, and we used an experimental method to randomly assign the participants to survey formats, which presumably differed in task difficulty. Based on satisficing theor ...[more]

PMID: 29795893

Similar Datasets

Project description:ContextSelf-report questionnaires are an important method of evaluating lifespan health, exercise, and health-related quality of life (HRQL) outcomes among elite, competitive athletes. Few instruments, however, have undergone formal characterization of their psychometric properties within this population.ObjectiveTo evaluate the validity and reliability of a novel health and exercise questionnaire, the Trojan Lifetime Champions (TLC) Health Survey.DesignDescriptive laboratory study.SettingA large National Collegiate Athletic Association Division I university.Patients or other participantsA total of 63 university alumni (age range, 24 to 84 years), including former varsity collegiate athletes and a control group of nonathletes.Intervention(s)Participants completed the TLC Health Survey twice at a mean interval of 23 days with randomization to the paper or electronic version of the instrument.Main outcome measure(s)Content validity, feasibility of administration, test-retest reliability, parallel-form reliability between paper and electronic forms, and estimates of systematic and typical error versus differences of clinical interest were assessed across a broad range of health, exercise, and HRQL measures.ResultsCorrelation coefficients, including intraclass correlation coefficients (ICCs) for continuous variables and κ agreement statistics for ordinal variables, for test-retest reliability averaged 0.86, 0.90, 0.80, and 0.74 for HRQL, lifetime health, recent health, and exercise variables, respectively. Correlation coefficients, again ICCs and κ, for parallel-form reliability (ie, equivalence) between paper and electronic versions averaged 0.90, 0.85, 0.85, and 0.81 for HRQL, lifetime health, recent health, and exercise variables, respectively. Typical measurement error was less than the a priori thresholds of clinical interest, and we found minimal evidence of systematic test-retest error. We found strong evidence of content validity, convergent construct validity with the Short-Form 12 Version 2 HRQL instrument, and feasibility of administration in an elite, competitive athletic population.ConclusionsThese data suggest that the TLC Health Survey is a valid and reliable instrument for assessing lifetime and recent health, exercise, and HRQL, among elite competitive athletes. Generalizability of the instrument may be enhanced by additional, larger-scale studies in diverse populations.

Project description:AimThe objective of this study was to evaluate the Chinese version of the Smartphone Distraction Scale (C-SDS), which is an easy-to-use tool for screening the risk of smartphone distraction in Chinese college students.MethodsThe C-SDS, Smartphone Addiction Scale - Short Version (SAS-SV), Fear of Missing Out scale (FoMO) and Metacognition about Smartphone Use Questionnaire (MSUQ) were used in a sample of 1,002 Chinese college students to test smartphone distraction and its influencing factors. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were performed to test measurement properties and factor structures of the C-SDS. Multi-variable linear regressions examined the relationships of sex, age, education level, the purpose of using a smartphone, usage of smartphone (hours per day), fear of missing out, smartphone addiction and positive and negative metacognitions about smartphone use with the C-SDS.ResultsThe EFA showed a 3-factor structure, which consisted of attention impulsiveness, multitasking and emotion regulation. The CFA showed that the 3-factor demonstrated an overall better model fit (RMSEA = 0.07, SRMR = 0.05, CFI = 0.94, TLI = 0.93). The C-SDS showed internal consistency (Cronbach's α = 0.88, McDonald's Omega ω = 0.88). Findings included that negative metacognition about smartphone use was most correlated with the C-SDS (b = 0.73; p < 0.001). Smartphone addiction, positive metacognition about smartphone use and fear of missing out also correlated with the C-SDS (b = 0.66, p < 0.001; b = 0.53, p < 0.001; b = 0.40, p < 0.001, respectively). The study shows that males compared to females (b = -1.65; p = 0.003), had a higher C-SDS score.ConclusionThe C-SDS was valid and reliable for assessing the distraction of using smartphones in the Chinese context. Being female, the purpose of using a smartphone, smartphone usage (hours per day), fear of missing out, smartphone addiction and positive and negative metacognitions about smartphone use were positively correlated to the C-SDS.

Dataset Information

Survey Satisficing Inflates Reliability and Validity Measures: An Experimental Comparison of College and Amazon Mechanical Turk Samples.

Publications

Survey Satisficing Inflates Reliability and Validity Measures: An Experimental Comparison of College and Amazon Mechanical Turk Samples.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure