Dataset Information

Online panels in social science research: Expanding sampling methods beyond Mechanical Turk.

ABSTRACT: Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student subject pools, but it also has important limitations. In particular, the MTurk population is small and potentially overused, and some groups of interest to behavioral scientists are underrepresented and difficult to recruit. Here we examined whether online research panels can avoid these limitations. Specifically, we compared sample composition, data quality (measured by effect sizes, internal reliability, and attention checks), and the non-naivete of participants recruited from MTurk and Prime Panels-an aggregate of online research panels. Prime Panels participants were more diverse in age, family composition, religiosity, education, and political attitudes. Prime Panels participants also reported less exposure to classic protocols and produced larger effect sizes, but only after screening out several participants who failed a screening task. We conclude that online research panels offer a unique opportunity for research, yet one with some important trade-offs.

SUBMITTER: Chandler J

PROVIDER: S-EPMC6797699 | biostudies-literature | 2019 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Online panels in social science research: Expanding sampling methods beyond Mechanical Turk.

Chandler Jesse J Rosenzweig Cheskie C Moss Aaron J AJ Robinson Jonathan J Litman Leib L

Behavior research methods 20191001 5

Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student subject pools, but it also has important limitations. In particular, the MTurk population is small and potentially overused, and some groups of interest to behavioral scientists are underrepresented and difficult to recruit. Here we examined whether online research panels can avoid these limitations. Specifically, we compared sample composition ...[more]

PMID: 31512174

Similar Datasets

Project description:Internet interventions face significant challenges in recruitment and attrition rates are typically high and problematic. Finding innovative yet scientifically valid avenues for attaining and retaining participants is therefore of considerable importance. The main goal of this study was to compare recruitment process and participants characteristics between two similar randomized control trials of mood management interventions. One of the trials (Bunge et al., 2016) was conducted with participants recruited from Amazon's Mechanical Turk (AMT), and the other trial recruited via Unpaid Internet Resources (UIR). Methods:The AMT sample (Bunge et al., 2016) consisted of 765 adults, and the UIR sample (recruited specifically for this study) consisted of 329 adult US residents. Participants' levels of depression, anxiety, confidence, motivation, and perceived usefulness of the intervention were assessed. The AMT sample was financially compensated whereas the UIR was not. Results:AMT yielded higher recruitment rates per month (p?<?.05). At baseline, the AMT sample reported significantly lower depression and anxiety scores (p?<?.001 and p?<?.005, respectively) and significantly higher mood, motivation, and confidence (all p?<?.001) compared to the UIR sample. AMT participants spent significantly less time on the site (p?<?.05) and were more likely to complete follow-ups than the UIR sample (p?<?.05). Both samples reported a significant increase in their level of confidence and motivation from pre- to post-intervention. AMT participants showed a significant increase in perceived usefulness of the intervention (p?<?.0001), whereas the UIR sample did not (p?=?.1642). Conclusions:By using AMT, researchers can recruit very rapidly and obtain higher retention rates; however, these participants may not be representative of the general online population interested in clinical interventions. Considering that AMT and UIR participants differed in most baseline variables, data from clinical studies resulting from AMT samples should be interpreted with caution.

Project description:BackgroundWearable devices that are used for observational research and clinical trials hold promise for collecting data from study participants in a convenient, scalable way that is more likely to reach a broad and diverse population than traditional research approaches. Amazon Mechanical Turk (MTurk) is a potential resource that researchers can use to recruit individuals into studies that use data from wearable devices.ObjectiveThis study aimed to explore the characteristics of wearable device users on MTurk that are associated with a willingness to share wearable device data for research. We also aimed to determine whether compensation was a factor that influenced the willingness to share such data.MethodsThis was a secondary analysis of a cross-sectional survey study of MTurk workers who use wearable devices for health monitoring. A 19-question web-based survey was administered from March 1 to April 5, 2018, to participants aged ≥18 years by using the MTurk platform. In order to identify characteristics that were associated with a willingness to share wearable device data, we performed logistic regression and decision tree analyses.ResultsA total of 935 MTurk workers who use wearable devices completed the survey. The majority of respondents indicated a willingness to share their wearable device data (615/935, 65.8%), and the majority of these respondents were willing to share their data if they received compensation (518/615, 84.2%). The findings from our logistic regression analyses indicated that Indian nationality (odds ratio [OR] 2.74, 95% CI 1.48-4.01, P=.007), higher annual income (OR 2.46, 95% CI 1.26-3.67, P=.02), over 6 months of using a wearable device (OR 1.75, 95% CI 1.21-2.29, P=.006), and the use of heartbeat and pulse tracking monitoring devices (OR 1.60, 95% CI 0.14-2.07, P=.01) are significant parameters that influence the willingness to share data. The only factor associated with a willingness to share data if compensation is provided was Indian nationality (OR 0.47, 95% CI 0.24-0.9, P=.02). The findings from our decision tree analyses indicated that the three leading parameters associated with a willingness to share data were the duration of wearable device use, nationality, and income.ConclusionsMost wearable device users indicated a willingness to share their data for research use (with or without compensation; 615/935, 65.8%). The probability of having a willingness to share these data was higher among individuals who had used a wearable for more than 6 months, were of Indian nationality, or were of American (United States of America) nationality and had an annual income of more than US $20,000. Individuals of Indian nationality who were willing to share their data expected compensation significantly less often than individuals of American nationality (P=.02).

Project description:Climate and ocean literacy are two of the most important challenges facing society today. However, many students lack exposure to these topics upon entering college. As a result, these students must rely on learning climate literacy and ocean conservation through experiences outside of those provided in the traditional undergraduate classroom. To fill this gap, we initiated a marine science professional development program to expose undergraduate students to ocean literacy principles and climate change concepts through marine ecology research and educational outreach. This study evaluates the effects of our undergraduate experiential learning for individuals involved in our research team, our educational outreach team, or both. Clemson University alumni that participated in our program were surveyed to determine educational and professional gains in three areas related to: (1) knowledge; (2) careers; and (3) attitudes. Multiple linear and logistic regressions were used to understand the relationships between gains and program type, mentor experience, and duration of program enrollment. In addition, we evaluated demographic covariates including age, ideology, and gender. Our study found that perceived knowledge of marine science and science communication skills increased with positive mentor experience. Alumni that rated their experience with their mentors highly also indicated that the program was important to their careers after graduation. Students who participated in any program for a prolonged period were more likely to indicate that marine science was important to their careers. These students were also more likely to continue their education. Additionally, we saw that a sense of belonging and identity in science, as well as the understanding of climate change threat on the marine environment, all increased with longer program involvement, more than the type of experience (research versus outreach). Overall, we found that both the research and outreach programs offered opportunities for advancements in knowledge, careers, and attitudes. These results provide evidence that experiential learning has the potential to increase student engagement and understanding of climate change and ocean literacy communication as well as a sense of belonging in science-oriented fields.

Dataset Information

Online panels in social science research: Expanding sampling methods beyond Mechanical Turk.

Publications

Online panels in social science research: Expanding sampling methods beyond Mechanical Turk.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets