The Influence of Sample Size on Parameter Estimates in Three-Level Random-Effects Models.
Ontology highlight
ABSTRACT:
SUBMITTER: Kerkhoff D
PROVIDER: S-EPMC6536630 | biostudies-literature | 2019
REPOSITORIES: biostudies-literature
Publications
The Influence of Sample Size on Parameter Estimates in Three-Level Random-Effects Models.
Kerkhoff Denise D Nussbeck Fridtjof W FW
Frontiers in psychology 20190521
In educational psychology, observational units are oftentimes nested within superordinate groups. Researchers need to account for hierarchy in the data by means of multilevel modeling, but especially in three-level longitudinal models, it is often unclear which sample size is necessary for reliable parameter estimation. To address this question, we generated a population dataset based on a study in the field of educational psychology, consisting of 3000 classrooms (level-3) with 55000 students ( ...[more]
Similar Datasets
Project description:BACKGROUND: Matrix models are widely used to study the dynamics and demography of populations. An important but overlooked issue is how the number of individuals sampled influences estimates of the population growth rate (lambda) calculated with matrix models. Even unbiased estimates of vital rates do not ensure unbiased estimates of lambda-Jensen's Inequality implies that even when the estimates of the vital rates are accurate, small sample sizes lead to biased estimates of lambda due to increased sampling variance. We investigated if sampling variability and the distribution of sampling effort among size classes lead to biases in estimates of lambda. METHODOLOGY/PRINCIPAL FINDINGS: Using data from a long-term field study of plant demography, we simulated the effects of sampling variance by drawing vital rates and calculating lambda for increasingly larger populations drawn from a total population of 3842 plants. We then compared these estimates of lambda with those based on the entire population and calculated the resulting bias. Finally, we conducted a review of the literature to determine the sample sizes typically used when parameterizing matrix models used to study plant demography. CONCLUSIONS/SIGNIFICANCE: We found significant bias at small sample sizes when survival was low (survival = 0.5), and that sampling with a more-realistic inverse J-shaped population structure exacerbated this bias. However our simulations also demonstrate that these biases rapidly become negligible with increasing sample sizes or as survival increases. For many of the sample sizes used in demographic studies, matrix models are probably robust to the biases resulting from sampling variance of vital rates. However, this conclusion may depend on the structure of populations or the distribution of sampling effort in ways that are unexplored. We suggest more intensive sampling of populations when individual survival is low and greater sampling of stages with high elasticities.
| S-EPMC2518208 | biostudies-literature
Project description:Three-level cluster randomized trials (CRTs) are increasingly used in implementation science, where 2fold-nested-correlated data arise. For example, interventions are randomly assigned to practices, and providers within the same practice who provide care to participants are trained with the assigned intervention. Teerenstra et al proposed a nested exchangeable correlation structure that accounts for two levels of clustering within the generalized estimating equations (GEE) approach. In this article, we utilize GEE models to test the treatment effect in a two-group comparison for continuous, binary, or count data in three-level CRTs. Given the nested exchangeable correlation structure, we derive the asymptotic variances of the estimator of the treatment effect for different types of outcomes. When the number of clusters is small, researchers have proposed bias-corrected sandwich estimators to improve performance in two-level CRTs. We extend the variances of two bias-corrected sandwich estimators to three-level CRTs. The equal provider and practice sizes were assumed to calculate number of practices for simplicity. However, they are not guaranteed in practice. Relative efficiency (RE) is defined as the ratio of variance of the estimator of the treatment effect for equal to unequal provider and practice sizes. The expressions of REs are obtained from both asymptotic variance estimation and bias-corrected sandwich estimators. Their performances are evaluated for different scenarios of provider and practice size distributions through simulation studies. Finally, a percentage increase in the number of practices is proposed due to efficiency loss from unequal provider and/or practice sizes.
| S-EPMC8351402 | biostudies-literature
Project description:ObjectiveTo assess the influence of trial sample size on treatment effect estimates within meta-analyses.DesignMeta-epidemiological study.Data sources93 meta-analyses (735 randomised controlled trials) assessing therapeutic interventions with binary outcomes, published in the 10 leading journals of each medical subject category of the Journal Citation Reports or in the Cochrane Database of Systematic Reviews.Data extractionSample size, outcome data, and risk of bias extracted from each trial.Data synthesisTrials within each meta-analysis were sorted by their sample size: using quarters within each meta-analysis (from quarter 1 with 25% of the smallest trials, to quarter 4 with 25% of the largest trials), and using size groups across meta-analyses (ranging from <50 to ≥ 1000 patients). Treatment effects were compared within each meta-analysis between quarters or between size groups by average ratios of odds ratios (where a ratio of odds ratios less than 1 indicates larger effects in smaller trials).ResultsTreatment effect estimates were significantly larger in smaller trials, regardless of sample size. Compared with quarter 4 (which included the largest trials), treatment effects were, on average, 32% larger in trials in quarter 1 (which included the smallest trials; ratio of odds ratios 0.68, 95% confidence interval 0.57 to 0.82), 17% larger in trials in quarter 2 (0.83, 0.75 to 0.91), and 12% larger in trials in quarter 3 (0.88, 0.82 to 0.95). Similar results were obtained when comparing treatment effect estimates between different size groups. Compared with trials of 1000 patients or more, treatment effects were, on average, 48% larger in trials with fewer than 50 patients (0.52, 0.41 to 0.66) and 10% larger in trials with 500-999 patients (0.90, 0.82 to 1.00).ConclusionsTreatment effect estimates differed within meta-analyses solely based on trial sample size, with stronger effect estimates seen in small to moderately sized trials than in the largest trials.
| S-EPMC3634626 | biostudies-literature
Project description:Disease risk estimation plays an important role in disease prevention. Many studies have found that the ability to predict risk improves as the number of risk single-nucleotide polymorphisms (SNPs) in the risk model increases. However, the width of the confidence interval of the risk estimate is often not considered in the evaluation of the risk model. Here, we explore how the risk and the confidence interval width change as more SNPs are added to the model in the order of decreasing effect size, using both simulated data and real data from studies of abdominal aortic aneurysms and age-related macular degeneration. Our results show that confidence interval width is positively correlated with model size and the majority of the bigger models have wider confidence interval widths than smaller models. Once the model size is bigger than a certain level, the risk does not shift markedly, as 100% of the risk estimates of the one-SNP-bigger models lie inside the confidence interval of the one-SNP-smaller models. We also created a confidence interval-augmented reclassification table. It shows that both more effective SNPs with larger odds ratios and less effective SNPs with smaller odds ratios contribute to the correct decision of whom to screen. The best screening strategy is selected and evaluated by the net benefit quantity and the reclassification rate. We suggest that individuals whose upper bound of their risk confidence interval is above the screening threshold, which corresponds to the population prevalence of the disease, should be screened.
| S-EPMC5628612 | biostudies-literature
Project description:Given the high technical reproducibility and orders of magnitude greater resolution than microarrays, next-generation sequencing of mRNA (RNA-Seq) is quickly becoming the de facto standard for measuring levels of gene expression in biological experiments. Two important questions must be taken into consideration when designing a particular experiment, namely, 1) how deep does one need to sequence? and, 2) how many biological replicates are necessary to observe a significant change in expression?Based on the gene expression distributions from 127 RNA-Seq experiments, we find evidence that 91% ± 4% of all annotated genes are sequenced at a frequency of 0.1 times per million bases mapped, regardless of sample source. Based on this observation, and combining this information with other parameters such as biological variation and technical variation that we empirically estimate from our large datasets, we developed a model to estimate the statistical power needed to identify differentially expressed genes from RNA-Seq experiments.Our results provide a needed reference for ensuring RNA-Seq gene expression studies are conducted with the optimally sample size, power, and sequencing depth. We also make available both R code and an Excel worksheet for investigators to calculate for their own experiments.
| S-EPMC3842884 | biostudies-other
Project description:Aberrant changes to interactions among cellular components have been conjectured to be potential causes of abnormalities in cellular functions. By systematic analysis of high-throughput-omics data, researchers hope to detect potential associations among measured variables for better biomarker identification and phenotype prediction. In this paper, we focus on the methods to measure pairwise interactive effects among continuous random variables, representing molecular expressions, with respect to a given categorical outcome. Together with a comprehensive review on the existing measures, we further propose new measures that better estimate interactive effects, especially in small sample size scenarios. We first evaluate the performance of the existing and new methods for both small and large sample sizes based on simulated datasets that shows our proposed methods outperform previous methods in general. The best performing method for small sample size scenarios suggested by simulation experiments is then implemented to estimate interactive effects among genes with respect to the metastasis outcome in two breast cancer studies based on micro-array gene expression datasets. Our results further demonstrate that integrating detected interactive effects together with individual effects can help in finding more accurate biomarkers for breast cancer metastasis, which are indeed involved in important pathways related to cancer metastasis based on gene set enrichment analysis.
| S-EPMC5775817 | biostudies-literature
Project description:Stepped-wedge (SW) designs have been steadily implemented in a variety of trials. A SW design typically assumes a three-level hierarchical data structure where participants are nested within times or periods which are in turn nested within clusters. Therefore, statistical models for analysis of SW trial data need to consider two correlations, the first and second level correlations. Existing power functions and sample size determination formulas had been derived based on statistical models for two-level data structures. Consequently, the second-level correlation has not been incorporated in conventional power analyses. In this paper, we derived a closed-form explicit power function based on a statistical model for three-level continuous outcome data. The power function is based on a pooled overall estimate of stratified cluster-specific estimates of an intervention effect. The sampling distribution of the pooled estimate is derived by applying a fixed-effect meta-analytic approach. Simulation studies verified that the derived power function is unbiased and can be applicable to varying number of participants per period per cluster. In addition, when data structures are assumed to have two levels, we compare three types of power functions by conducting additional simulation studies under a two-level statistical model. In this case, the power function based on a sampling distribution of a marginal, as opposed to pooled, estimate of the intervention effect performed the best. Extensions of power functions to binary outcomes are also suggested.
| S-EPMC5026912 | biostudies-literature
Project description:BackgroundPhysics-based cardiovascular models are only recently being considered for disease diagnosis or prognosis in clinical settings. These models depend on parameters representing the physical and physiological properties of the modeled system. Personalizing these parameters may give insight into the specific state of the individual and etiology of disease. We applied a relatively fast model optimization scheme based on common local optimization methods to two model formulations of the left ventricle and systemic circulation. One closed-loop model and one open-loop model were applied. Intermittently collected hemodynamic data from an exercise motivation study were used to personalize these models for data from 25 participants. The hemodynamic data were collected for each participant at the start, middle and end of the trial. We constructed two data sets for the participants, both consisting of systolic and diastolic brachial pressure, stroke volume, and left-ventricular outflow tract velocity traces paired with either the finger arterial pressure waveform or the carotid pressure waveform.ResultsWe examined the feasibility of separating parameter estimates for the individual from population estimates by assessing the variability of estimates using the interquartile range. We found that the estimated parameter values were similar for the two model formulations, but that the systemic arterial compliance was significantly different ([Formula: see text]) depending on choice of pressure waveform. The estimates of systemic arterial compliance were on average higher when using the finger artery pressure waveform as compared to the carotid waveform.ConclusionsWe found that for the majority of participants, the variability of parameter estimates for a given participant on any measurement day was lower than the variability both across all measurement days combined for one participant, and for the population. This indicates that it is possible to identify individuals from the population, and that we can distinguish different measurement days for the individual participant by parameter values using the presented optimization method.
| S-EPMC10099701 | biostudies-literature
Project description:Despite the popularity of the general linear mixed model for data analysis, power and sample size methods and software are not generally available for commonly used test statistics and reference distributions. Statisticians resort to simulations with homegrown and uncertified programs or rough approximations which are misaligned with the data analysis. For a wide range of designs with longitudinal and clustering features, we provide accurate power and sample size approximations for inference about fixed effects in linear models we call reversible. We show that under widely applicable conditions, the general linear mixed-model Wald test has non-central distributions equivalent to well-studied multivariate tests. In turn, exact and approximate power and sample size results for the multivariate Hotelling-Lawley test provide exact and approximate power and sample size results for the mixed-model Wald test. The calculations are easily computed with a free, open-source product that requires only a web browser to use. Commercial software can be used for a smaller range of reversible models. Simple approximations allow accounting for modest amounts of missing data. A real-world example illustrates the methods. Sample size results are presented for a multicenter study on pregnancy. The proposed study, an extension of a funded project, has clustering within clinic. Exchangeability among participants allows averaging across them to remove the clustering structure. The resulting simplified design is a single level longitudinal study. Multivariate methods for power provide an approximate sample size. All proofs and inputs for the example are in the Supplementary Materials (available online).
| S-EPMC7009022 | biostudies-literature
Project description:Robust maximum likelihood (RML) and asymptotically generalized least squares (AGLS) methods have been recommended for fitting ordinal structural equation models. Studies show that some of these methods underestimate standard errors. However, these studies have not investigated the coverage and bias of interval estimates. An estimate with a reasonable standard error could still be severely biased. This can only be known by systematically investigating the interval estimates. The present study compares Bayesian, RML, and AGLS interval estimates of factor correlations in ordinal confirmatory factor analysis models (CFA) for small sample data. Six sample sizes, 3 factor correlations, and 2 factor score distributions (multivariate normal and multivariate mildly skewed) were studied. Two Bayesian prior specifications, informative and relatively less informative were studied. Undercoverage of confidence intervals and underestimation of standard errors was common in non-Bayesian methods. Underestimated standard errors may lead to inflated Type-I error rates. Non-Bayesian intervals were more positive biased than negatively biased, that is, most intervals that did not contain the true value were greater than the true value. Some non-Bayesian methods had non-converging and inadmissible solutions for small samples and non-normal data. Bayesian empirical standard error estimates for informative and relatively less informative priors were closer to the average standard errors of the estimates. The coverage of Bayesian credibility intervals was closer to what was expected with overcoverage in a few cases. Although some Bayesian credibility intervals were wider, they reflected the nature of statistical uncertainty that comes with the data (e.g., small sample). Bayesian point estimates were also more accurate than non-Bayesian estimates. The results illustrate the importance of analyzing coverage and bias of interval estimates, and how ignoring interval estimates can be misleading. Therefore, editors and policymakers should continue to emphasize the inclusion of interval estimates in research.
| S-EPMC4626630 | biostudies-literature