Dataset Information

Fixed or random? On the reliability of mixed-effects models for a small number of levels in grouping variables.

ABSTRACT: Biological data are often intrinsically hierarchical (e.g., species from different genera, plants within different mountain regions), which made mixed-effects models a common analysis tool in ecology and evolution because they can account for the non-independence. Many questions around their practical applications are solved but one is still debated: Should we treat a grouping variable with a low number of levels as a random or fixed effect? In such situations, the variance estimate of the random effect can be imprecise, but it is unknown if this affects statistical power and type I error rates of the fixed effects of interest. Here, we analyzed the consequences of treating a grouping variable with 2-8 levels as fixed or random effect in correctly specified and alternative models (under- or overparametrized models). We calculated type I error rates and statistical power for all-model specifications and quantified the influences of study design on these quantities. We found no influence of model choice on type I error rate and power on the population-level effect (slope) for random intercept-only models. However, with varying intercepts and slopes in the data-generating process, using a random slope and intercept model, and switching to a fixed-effects model, in case of a singular fit, avoids overconfidence in the results. Additionally, the number and difference between levels strongly influences power and type I error. We conclude that inferring the correct random-effect structure is of great importance to obtain correct type I error rates. We encourage to start with a mixed-effects model independent of the number of levels in the grouping variable and switch to a fixed-effects model only in case of a singular fit. With these recommendations, we allow for more informative choices about study design and data analysis and make ecological inference with mixed-effects models more robust for small number of levels.

SUBMITTER: Oberpriller J

PROVIDER: S-EPMC9309037 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Studies of the primate visual system have begun to test a wide range of complex computational object-vision models. Realistic models have many parameters, which in practice cannot be fitted using the limited amounts of brain-activity data typically available. Task performance optimization (e.g. using backpropagation to train neural networks) provides major constraints for fitting parameters and discovering nonlinear representational features appropriate for the task (e.g. object classification). Model representations can be compared to brain representations in terms of the representational dissimilarities they predict for an image set. This method, called representational similarity analysis (RSA), enables us to test the representational feature space as is (fixed RSA) or to fit a linear transformation that mixes the nonlinear model features so as to best explain a cortical area's representational space (mixed RSA). Like voxel/population-receptive-field modelling, mixed RSA uses a training set (different stimuli) to fit one weight per model feature and response channel (voxels here), so as to best predict the response profile across images for each response channel. We analysed response patterns elicited by natural images, which were measured with functional magnetic resonance imaging (fMRI). We found that early visual areas were best accounted for by shallow models, such as a Gabor wavelet pyramid (GWP). The GWP model performed similarly with and without mixing, suggesting that the original features already approximated the representational space, obviating the need for mixing. However, a higher ventral-stream visual representation (lateral occipital region) was best explained by the higher layers of a deep convolutional network and mixing of its feature set was essential for this model to explain the representation. We suspect that mixing was essential because the convolutional network had been trained to discriminate a set of 1000 categories, whose frequencies in the training set did not match their frequencies in natural experience or their behavioural importance. The latter factors might determine the representational prominence of semantic dimensions in higher-level ventral-stream areas. Our results demonstrate the benefits of testing both the specific representational hypothesis expressed by a model's original feature space and the hypothesis space generated by linear transformations of that feature space.

Project description:Non-linear mixed effects models typically deal with stochasticity in observed processes but models accounting for only observed processes may not be the most appropriate for all data. Hidden Markov models (HMMs) characterize the relationship between observed and hidden variables where the hidden variables can represent an underlying and unmeasurable disease status for example. Adding stochasticity to HMMs results in mixed HMMs (MHMMs) which potentially allow for the characterization of variability in unobservable processes. Further, HMMs can be extended to include more than one observation source and are then multivariate HMMs. In this work MHMMs were developed and applied in a chronic obstructive pulmonary disease example. The two hidden states included in the model were remission and exacerbation and two observation sources were considered, patient reported outcomes (PROs) and forced expiratory volume (FEV1). Estimation properties in the software NONMEM of model parameters were investigated with and without random and covariate effect parameters. The influence of including random and covariate effects of varying magnitudes on the parameters in the model was quantified and a power analysis was performed to compare the power of a single bivariate MHMM with two separate univariate MHMMs. A bivariate MHMM was developed for simulating and analysing hypothetical COPD data consisting of PRO and FEV1 measurements collected every week for 60 weeks. Parameter precision was high for all parameters with the exception of the variance of the transition rate dictating the transition from remission to exacerbation (relative root mean squared error [RRMSE]?>?150%). Parameter precision was better with higher magnitudes of the transition probability parameters. A drug effect was included on the transition rate probability and the precision of the drug effect parameter improved with increasing magnitude of the parameter. The power to detect the drug effect was improved by utilizing a bivariate MHMM model over the univariate MHMM models where the number of subject required for 80% power was 25 with the bivariate MHMM model versus 63 in the univariate MHMM FEV1 model and?>?100 in the univariate MHMM PRO model. The results advocates for the use of bivariate MHMM models when implementation is possible.

Dataset Information

Fixed or random? On the reliability of mixed-effects models for a small number of levels in grouping variables.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets