An implementation of N-way repeated measures ANOVA: Effect coding, automated unpacking of interactions, and randomization testing.
Ontology highlight
ABSTRACT:
SUBMITTER: Gladwin TE
PROVIDER: S-EPMC7315100 | biostudies-literature | 2020
REPOSITORIES: biostudies-literature
Publications
An implementation of N-way repeated measures ANOVA: Effect coding, automated unpacking of interactions, and randomization testing.
MethodsX 20200602
The paper presents the details of an implementation of repeated measures ANOVA, consisting of a set of functions to organize data and represent contrasts to be tested and run statistical tests. The implementation is focused on uses common in experimental psychology. An arbitrary number of within-subject factors, each with an arbitrary number of levels, can be used. A non-parametric, randomization- and permutation-based formulation of repeated measures ANOVA was defined and implemented. Methods f ...[more]
Similar Datasets
Project description:We present an approach to analyzing physiologic timetrends recorded during a stimulus by comparing means at each time point using repeated measures analysis of variance (RMANOVA). The approach allows temporal patterns to be examined without an a priori model of expected timing or pattern of response. The approach was originally applied to signals recorded from functional magnetic resonance imaging (fMRI) volumes-of-interest (VOI) during a physiologic challenge, but we have used the same technique to analyze continuous recordings of other physiological signals such as heart rate, breathing rate, and pulse oximetry. For fMRI, the method serves as a complement to whole-brain voxel-based analyses, and is useful for detecting complex responses within pre-determined brain regions, or as a post-hoc analysis of regions of interest identified by whole-brain assessments. We illustrate an implementation of the technique in the statistical software packages R and SAS. VOI timetrends are extracted from conventionally preprocessed fMRI images. A timetrend of average signal intensity across the VOI during the scanning period is calculated for each subject. The values are scaled relative to baseline periods, and time points are binned. In SAS, the procedure PROC MIXED implements the RMANOVA in a single step. In R, we present one option for implementing RMANOVA with the mixed model function "lme". Model diagnostics, and predicted means and differences are best performed with additional libraries and commands in R; we present one example. The ensuing results allow determination of significant overall effects, and time-point specific within- and between-group responses relative to baseline. We illustrate the technique using fMRI data from two groups of subjects who underwent a respiratory challenge. RMANOVA allows insight into the timing of responses and response differences between groups, and so is suited to physiologic testing paradigms eliciting complex response patterns.
| S-EPMC4995690 | biostudies-literature
Project description:IntroductionOne-way repeated measures ANOVA requires sphericity. Research indicates that violation of this assumption has an important impact on Type I error. Although more advanced alternative procedures exist, most classical texts recommend the use of adjusted F-tests, which are frequently employed because they are intuitive, easy to apply, and available in most statistical software. Adjusted F-tests differ in the procedure used to estimate the corrective factor ε, the most common being the Greenhouse-Geisser (F-GG) and Huynh-Feldt (F-HF) adjustments. Although numerous studies have analyzed the robustness of these procedures, the results are inconsistent, thus highlighting the need for further research.MethodsThe aim of this simulation study was to analyze the performance of the F-statistic, F-GG, and F-HF in terms of Type I error and power in one-way designs with normal data under a variety of conditions that may be encountered in real research practice. Values of ε were fixed according to the Greenhouse-Geisser procedure (ε^). We manipulated the number of repeated measures (3, 4, and 6) and sample size (from 10 to 300), with ε^ values ranging from the lower to its upper limit.ResultsOverall, the results showed that the F-statistic becomes more liberal as sphericity violation increases, whereas both F-HF and F-GG control Type I error; of the two, F-GG is more conservative, especially with large values of ε^ and small samples.DiscussionIf different statistical conclusions follow from application of the two tests, we recommend using F-GG for ε^ values below 0.60, and F-HF for ε^ values equal to or above 0.60.
| S-EPMC10499170 | biostudies-literature
Project description:With ever increasing accessibility to high throughput technologies, more complex treatment structures can be assessed in a variety of 'omics applications. This adds an extra layer of complexity to the analysis and interpretation, in particular when inferential univariate methods are applied en masse. It is well-known that mass univariate testing suffers from multiplicity issues and although this has been well documented for simple comparative tests, few approaches have focussed on more complex explanatory structures.Two frameworks are introduced incorporating corrections for multiplicity whilst maintaining appropriate structure in the explanatory variables. Within this paradigm, a choice has to be made as to whether multiplicity corrections should be applied to the saturated model, putting emphasis on controlling the rate of false positives, or to the predictive model, where emphasis is on model selection. This choice has implications for both the ranking and selection of the response variables identified as differentially expressed. The theoretical difference is demonstrated between the two approaches along with an empirical study of lipid composition in Arabidopsis under differing levels of salt stress.Multiplicity corrections have an inherent weakness when the full explanatory structure is not properly incorporated. Although a unifying 'single best' recommendation is not provided, two reasonable alternatives are provided and the applicability of these approaches is discussed for different scenarios where the aims of analysis will differ. The key result is that the point at which multiplicity is incorporated into the analysis will fundamentally change the interpretation of the results, and the choice of approach should therefore be driven by the specific aims of the experiment.
| S-EPMC6069693 | biostudies-other
Project description:MotivationGene regulatory networks (GRN) can be determined via various experimental techniques, and also by computational methods, which infer networks from gene expression data. However, these techniques treat interactions separately such that interdependencies of interactions forming meaningful subnetworks are typically not considered.MethodsFor the investigation of network properties and for the classification of different (sub-)networks based on gene expression data, we consider biological network motifs consisting of three genes and up to three interactions, e.g. the cascade chain (CSC), feed-forward loop (FFL), and dense-overlapping regulon (DOR). We examine several conventional methods for the inference of network motifs, which typically consider each interaction individually. In addition, we propose a new method based on three-way ANOVA (ANalysis Of VAriance) (3WA) that analyzes entire subnetworks at once. To demonstrate the advantages of such a more holistic perspective, we compare the ability of 3WA and other methods to detect and categorize network motifs on large real and artificial datasets.ResultsWe find that conventional methods perform much better on artificial data (AUC up to 80%), than on real E. coli expression datasets (AUC 50% corresponding to random guessing). To explain this observation, we examine several important properties that differ between datasets and analyze predicted motifs in detail. We find that in case of real networks our new 3WA method outperforms (AUC 70% in E. coli) previous methods by exploiting the interdependencies in the full motif structure. Because of important differences between current artificial datasets and real measurements, the construction and testing of motif detection methods should focus on real data.
| S-EPMC6078297 | biostudies-literature
Project description:During an ischemic stroke, the brain releases various factors, including glutamate and γ-aminobutyric acid. Glutamate can cause neurotoxic effects through certain receptors and exacerbate neurological damage, while γ-aminobutyric acid as an inhibitory neurotransmitter can antagonize the excitotoxic effects of glutamate and enhance the tolerance of neurons to ischemia. Therefore, in this study, the content of amino acid neurotransmitters in brain tissue before ischemia, after 10 min of ischemia, hypothermic perfusion, and rewarming were analyzed by high-performance liquid chromatography-UV in an animal model of ischemic stroke generated by blocking the bilateral common carotid arteries of rhesus monkeys. The changes in amino acid neurotransmitters in the rhesus monkey brain during post-ischemia hypothermic perfusion and rewarming were investigated by statistical methods of repeated measures ANOVA, showing that the concentration change of glutamate had not only a temporal factor but also was influenced by temperature, and there was an interaction effect between the two. Time but not temperature affected the change in γ-aminobutyric acid concentration, and there was an interaction effect between the two. Accordingly, hypoperfusion exerts a protective effect during ischemia by inhibiting the release of excitatory amino acid neurotransmitters, while the antagonistic effect of GABA on Glu is not significant.
| S-EPMC9411555 | biostudies-literature
Project description:Objective:retest effects may be attributed to 'repeated content' in neuropsychological tests such as words in word list-learning tests, or the 'testing context' which involves procedural memory and reduced test anxiety following repeated administration. Alzheimer's Disease (AD) severely impairs episodic memory, so longitudinal cognitive testing among people with dementia may reveal the relative contributions of content versus context to retest effects in neuropsychological testing. Method:we used data from the Critical Path Institute's repository of placebo arm data from randomized controlled trials (RCTs) of dementia conducted by participating pharmaceutical companies (N = 990 people, 4,170 study visits, up to 2.4 years of follow-up). To estimate retest effects on the Mini-Mental State Examination (MMSE), we used linear regressions with random effects for people and time, adjusting for age, sex and race, and longitudinal quantile regressions. Results:average MMSE score (16.6 points, SD = 5.5, range 1, 27) declined by 2.0 points/year (95% confidence interval, CI: -2.3, -1.8). Mean retest effect was 0.6 points (95% CI: 0.4, 0.8) at second assessment (average 4 months after baseline). Retest effects were similar among participants with and without any recall on the short-delay word recall subscale score at baseline, and at the 30th, 50th and 70th percentiles of the MMSE distribution, suggesting similar retest effects across the spectrum from mild to severe cases of dementia. Conclusions:retest effects are apparent in people with dementia despite reduced episodic memory, suggesting a prominent role of the testing context in RCTs and cohort studies.
| S-EPMC6201832 | biostudies-other
Project description:We investigated the effects of violations of the sphericity assumption on Type I error rates for different methodical approaches of repeated measures analysis using a simulation approach. In contrast to previous simulation studies on this topic, up to nine measurement occasions were considered. Effects of the level of inter-correlations between measurement occasions on Type I error rates were considered for the first time. Two populations with non-violation of the sphericity assumption, one with uncorrelated measurement occasions and one with moderately correlated measurement occasions, were generated. One population with violation of the sphericity assumption combines uncorrelated with highly correlated measurement occasions. A second population with violation of the sphericity assumption combines moderately correlated and highly correlated measurement occasions. From these four populations without any between-group effect or within-subject effect 5,000 random samples were drawn. Finally, the mean Type I error rates for Multilevel linear models (MLM) with an unstructured covariance matrix (MLM-UN), MLM with compound-symmetry (MLM-CS) and for repeated measures analysis of variance (rANOVA) models (without correction, with Greenhouse-Geisser-correction, and Huynh-Feldt-correction) were computed. To examine the effect of both the sample size and the number of measurement occasions, sample sizes of n = 20, 40, 60, 80, and 100 were considered as well as measurement occasions of m = 3, 6, and 9. With respect to rANOVA, the results plead for a use of rANOVA with Huynh-Feldt-correction, especially when the sphericity assumption is violated, the sample size is rather small and the number of measurement occasions is large. For MLM-UN, the results illustrate a massive progressive bias for small sample sizes (n = 20) and m = 6 or more measurement occasions. This effect could not be found in previous simulation studies with a smaller number of measurement occasions. The proportionality of bias and number of measurement occasions should be considered when MLM-UN is used. The good news is that this proportionality can be compensated by means of large sample sizes. Accordingly, MLM-UN can be recommended even for small sample sizes for about three measurement occasions and for large sample sizes for about nine measurement occasions.
| S-EPMC5651023 | biostudies-literature
Project description:BACKGROUND:The prospect of assessing cognition longitudinally and remotely is attractive to researchers, health practitioners, and pharmaceutical companies alike. However, such repeated testing regimes place a considerable burden on participants, and with cognitive tasks typically being regarded as effortful and unengaging, these studies may experience high levels of participant attrition. One potential solution is to gamify these tasks to make them more engaging: increasing participant willingness to take part and reducing attrition. However, such an approach must balance task validity with the introduction of entertaining gamelike elements. OBJECTIVE:This study aims to investigate the effects of gamelike features on participant attrition using a between-subjects, longitudinal Web-based testing study. METHODS:We used three variants of a common cognitive task, the Stop Signal Task (SST), with a single gamelike feature in each: one variant where points were rewarded for performing optimally; another where the task was given a graphical theme; and a third variant, which was a standard SST and served as a control condition. Participants completed four compulsory test sessions over 4 consecutive days before entering a 6-day voluntary testing period where they faced a daily decision to either drop out or continue taking part. Participants were paid for each session they completed. RESULTS:A total of 482 participants signed up to take part in the study, with 265 completing the requisite four consecutive test sessions. No evidence of an effect of gamification on attrition was observed. A log-rank test showed no evidence of a difference in dropout rates between task variants (?22=3.0, P=.22), and a one-way analysis of variance of the mean number of sessions completed per participant in each variant also showed no evidence of a difference (F2,262=1.534, P=.21, partial ?2=0.012). CONCLUSIONS:Our findings raise doubts about the ability of gamification to reduce attrition from longitudinal cognitive testing studies.
| S-EPMC5719230 | biostudies-literature
Project description:BackgroundTraditional clinical assessments are used extensively in neurology; however, they can be coarse, which can also make them insensitive to change. Kinarm is a robotic assessment system that has been used for precise assessment of individuals with neurological impairments. However, this precision also leads to the challenge of identifying whether a given change in performance reflects a significant change in an individual's ability or is simply natural variation. Our objective here is to derive confidence intervals and thresholds of significant change for Kinarm Standard Tests™ (KST).MethodsWe assessed participants twice within 15 days on all tasks presently available in KST. We determined the 5-95% confidence intervals for each task parameter, and derived thresholds for significant change. We tested for learning effects and corrected for the false discovery rate (FDR) to identify task parameters with significant learning effects. Finally, we calculated intraclass correlation of type ICC [1, 2] (ICC-C) to quantify consistency across assessments.ResultsWe recruited an average of 56 participants per task. Confidence intervals for Z-Task Scores ranged between 0.61 and 1.55, and the threshold for significant change ranged between 0.87 and 2.19. We determined that 4/11 tasks displayed learning effects that were significant after FDR correction; these 4 tasks primarily tested cognition or cognitive-motor integration. ICC-C values for Z-Task Scores ranged from 0.26 to 0.76.ConclusionsThe present results provide statistical bounds on individual performance for KST as well as significant changes across repeated testing. Most measures of performance had good inter-rater reliability. Tasks with a higher cognitive burden seemed to be more susceptible to learning effects, which should be taken into account when interpreting longitudinal assessments of these tasks.
| S-EPMC7331240 | biostudies-literature
Project description:Mendelian randomization (MR) is often used to estimate effects of time-varying exposures on health outcomes using observational data. However, MR studies typically use a single measurement of exposure and apply conventional instrumental variable (IV) methods designed to handle time-fixed exposures. As such, MR effect estimates for time-varying exposures are often biased, and interpretations are unclear. We describe the instrumental conditions required for IV estimation with a time-varying exposure, and the additional conditions required to causally interpret MR estimates as a point effect, a period effect or a lifetime effect depending on whether researchers have measurements at a single or multiple time points. We propose methods to incorporate time-varying exposures in MR analyses based on g-estimation of structural mean models, and demonstrate its application by estimating the period effect of alcohol intake, high-density lipoprotein cholesterol and low-density lipoprotein cholesterol on intermediate coronary heart disease outcomes using data from the Framingham Heart Study. We use this data example to highlight the challenges of interpreting MR estimates as causal effects, and describe other extensions of structural mean models for more complex data scenarios.
| S-EPMC9067358 | biostudies-literature