Dataset Information

One Size Doesn't Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research.

ABSTRACT: Across all sciences, the quality of measurements is important. Survey measurements are only appropriate for use when researchers have validity evidence within their particular context. Yet, this step is frequently skipped or is not reported in educational research. This article briefly reviews the aspects of validity that researchers should consider when using surveys. It then focuses on factor analysis, a statistical method that can be used to collect an important type of validity evidence. Factor analysis helps researchers explore or confirm the relationships between survey items and identify the total number of dimensions represented on the survey. The essential steps to conduct and interpret a factor analysis are described. This use of factor analysis is illustrated throughout by a validation of Diekman and colleagues' goal endorsement instrument for use with first-year undergraduate science, technology, engineering, and mathematics students. We provide example data, annotated code, and output for analyses in R, an open-source programming language and software environment for statistical computing. For education researchers using surveys, understanding the theoretical and statistical underpinnings of survey validity is fundamental for implementing rigorous education research.

SUBMITTER: Knekta E

PROVIDER: S-EPMC6757227 | biostudies-literature | 2019 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

One Size Doesn't Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research.

Knekta Eva E Runyon Christopher C Eddy Sarah S

CBE life sciences education 20190301 1

Across all sciences, the quality of measurements is important. Survey measurements are only appropriate for use when researchers have validity evidence within their particular context. Yet, this step is frequently skipped or is not reported in educational research. This article briefly reviews the aspects of validity that researchers should consider when using surveys. It then focuses on factor analysis, a statistical method that can be used to collect an important type of validity evidence. Fac ...[more]

PMID: 30821600

Similar Datasets

Project description:The structure of contacts that mediate transmission has a pronounced effect on the outbreak dynamics of infectious disease and simulation models are powerful tools to inform policy decisions. Most simulation models of livestock disease spread rely to some degree on predictions of animal movement between holdings. Typically, movements are more common between nearby farms than between those located far away from each other. Here, we assessed spatiotemporal variation in such distance dependence of animal movement contacts from an epidemiological perspective. We evaluated and compared nine statistical models, applied to Swedish movement data from 2008. The models differed in at what level (if at all), they accounted for regional and/or seasonal heterogeneities in the distance dependence of the contacts. Using a kernel approach to describe how probability of contacts between farms changes with distance, we developed a hierarchical Bayesian framework and estimated parameters by using Markov Chain Monte Carlo techniques. We evaluated models by three different approaches of model selection. First, we used Deviance Information Criterion to evaluate their performance relative to each other. Secondly, we estimated the log predictive posterior distribution, this was also used to evaluate their relative performance. Thirdly, we performed posterior predictive checks by simulating movements with each of the parameterized models and evaluated their ability to recapture relevant summary statistics. Independent of selection criteria, we found that accounting for regional heterogeneity improved model accuracy. We also found that accounting for seasonal heterogeneity was beneficial, in terms of model accuracy, according to two of three methods used for model selection. Our results have important implications for livestock disease spread models where movement is an important risk factor for between farm transmission. We argue that modelers should refrain from using methods to simulate animal movements that assume the same pattern across all regions and seasons without explicitly testing for spatiotemporal variation.

Project description:BackgroundPromoting the uptake of vaccination for infectious diseases such as COVID-19 remains a global challenge, necessitating collaborative efforts between public health units (PHUs) and communities. Applied behavioural science can play a crucial role in supporting PHUs' response by providing insights into human behaviour and informing tailored strategies to enhance vaccination uptake. Community engagement can help broaden the reach of behavioural science research by involving a more diverse range of populations and ensuring that strategies better represent the needs of specific communities. We developed and applied an approach to conducting community-based behavioural science research with ethnically and socioeconomically diverse populations to guide PHUs in tailoring their strategies to promote COVID-19 vaccination. This paper presents the community engagement methodology and the lessons learned in applying the methodology.MethodsThe community engagement methodology was developed based on integrated knowledge translation (iKT) and community-based participatory research (CBPR) principles. The study involved collaboration with PHUs and local communities in Ontario, Canada to identify priority groups for COVID-19 vaccination, understand factors influencing vaccine uptake and co-design strategies tailored to each community to promote vaccination. Community engagement was conducted across three large urban regions with individuals from Eastern European communities, African, Black, and Caribbean communities and low socioeconomic neighbourhoods.ResultsWe developed and applied a seven-step methodology for conducting community-based behavioural science research: (1) aligning goals with system-level partners; (2) engaging with PHUs to understand priorities; (3) understanding community strengths and dynamics; (4) building relationships with each community; (5) establishing partnerships (community advisory groups); (6) involving community members in the research process; and (7) feeding back and interpreting research findings. Research partnerships were successfully established with members of prioritized communities, enabling recruitment of participants for theory-informed behavioural science interviews, interpretation of findings, and co-design of targeted recommendations for each PHU to improve COVID-19 vaccination uptake. Lessons learned include the importance of cultural sensitivity and awareness of sociopolitical context in tailoring community engagement, being agile to address the diverse and evolving priorities of PHUs, and building trust to achieve effective community engagement.ConclusionEffective community engagement in behavioural science research can lead to more inclusive and representative research. The community engagement approach developed and applied in this study acknowledges the diversity of communities, recognizes the central role of PHUs, and can help in addressing complex public health challenges.

Project description:Improved efficiency of Markov chain Monte Carlo facilitates all aspects of statistical analysis with Bayesian hierarchical models. Identifying strategies to improve MCMC performance is becoming increasingly crucial as the complexity of models, and the run times to fit them, increases. We evaluate different strategies for improving MCMC efficiency using the open-source software NIMBLE (R package nimble) using common ecological models of species occurrence and abundance as examples. We ask how MCMC efficiency depends on model formulation, model size, data, and sampling strategy. For multiseason and/or multispecies occupancy models and for N-mixture models, we compare the efficiency of sampling discrete latent states vs. integrating over them, including more vs. fewer hierarchical model components, and univariate vs. block-sampling methods. We include the common MCMC tool JAGS in comparisons. For simple models, there is little practical difference between computational approaches. As model complexity increases, there are strong interactions between model formulation and sampling strategy on MCMC efficiency. There is no one-size-fits-all best strategy, but rather problem-specific best strategies related to model structure and type. In all but the simplest cases, NIMBLE's default or customized performance achieves much higher efficiency than JAGS. In the two most complex examples, NIMBLE was 10-12 times more efficient than JAGS. We find NIMBLE is a valuable tool for many ecologists utilizing Bayesian inference, particularly for complex models where JAGS is prohibitively slow. Our results highlight the need for more guidelines and customizable approaches to fit hierarchical models to ensure practitioners can make the most of occupancy and other hierarchical models. By implementing model-generic MCMC procedures in open-source software, including the NIMBLE extensions for integrating over latent states (implemented in the R package nimbleEcology), we have made progress toward this aim.

Dataset Information

One Size Doesn't Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research.

Publications

One Size Doesn't Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets