Practical identifiability in the frame of nonlinear mixed effects models: the example of the in vitro erythropoiesis.
Ontology highlight
ABSTRACT: Background
Nonlinear mixed effects models provide a way to mathematically describe experimental data involving a lot of inter-individual heterogeneity. In order to assess their practical identifiability and estimate confidence intervals for their parameters, most mixed effects modelling programs use the Fisher Information Matrix. However, in complex nonlinear models, this approach can mask practical unidentifiabilities.Results
Herein we rather propose a multistart approach, and use it to simplify our model by reducing the number of its parameters, in order to make it identifiable. Our model describes several cell populations involved in the in vitro differentiation of chicken erythroid progenitors grown in the same environment. Inter-individual variability observed in cell population counts is explained by variations of the differentiation and proliferation rates between replicates of the experiment. Alternatively, we test a model with varying initial condition.Conclusions
We conclude by relating experimental variability to precise and identifiable variations between the replicates of the experiment of some model parameters.
SUBMITTER: Duchesne R
PROVIDER: S-EPMC8489053 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectiveThe current state-of-the-art for compartment modeling of dynamic PET data can be described as a two-stage approach. In Stage 1, individual estimates of kinetic parameters are obtained by fitting models using standard techniques, such as nonlinear least squares, to each individual's data one subject at a time. Population-level effects, such as the difference between diagnostic groups, are analyzed in Stage 2 using standard statistical methods by treating the individual estimates as if they were observed data. While this approach is generally valid, it is possible to increase efficiency and precision of the analysis, allow more complex models to be fitted, and also to permit parameter-specific investigation by fitting data across subjects simultaneously. We explore the application of nonlinear mixed-effects (NLME) models for estimation and inference in this setting.MethodsIn the NLME framework, subjects are modeled simultaneously through the inclusion of random effects of subjects for each kinetic parameter; meanwhile, population parameters are estimated directly in a joint model.ResultsSimulation results indicate that NLME outperforms the two-stage approach in estimating group-level effects and also has improved power to detect differences across groups. We applied our NLME approach to clinical PET data and found effects not detected by the two-stage approach.ConclusionThe proposed NLME approach is more accurate and correspondingly more powerful than the two-stage approach in compartment modeling of PET data.SignificanceThe NLME method can broaden the methodological scope of PET modeling because of its efficiency and stability.
| S-EPMC6436945 | biostudies-literature
Project description:Cross-validation is frequently used for model selection in a variety of applications. However, it is difficult to apply cross-validation to mixed effects models (including nonlinear mixed effects models or NLME models) due to the fact that cross-validation requires "out-of-sample" predictions of the outcome variable, which cannot be easily calculated when random effects are present. We describe two novel variants of cross-validation that can be applied to NLME models. One variant, where out-of-sample predictions are based on post hoc estimates of the random effects, can be used to select the overall structural model. Another variant, where cross-validation seeks to minimize the estimated random effects rather than the estimated residuals, can be used to select covariates to include in the model. We show that these methods produce accurate results in a variety of simulated data sets and apply them to two publicly available population pharmacokinetic data sets.
| S-EPMC3668859 | biostudies-literature
Project description:As systems approaches to the development of biological models become more mature, attention is increasingly focusing on the problem of inferring parameter values within those models from experimental data. However, particularly for nonlinear models, it is not obvious, either from inspection of the model or from the experimental data, that the inverse problem of parameter fitting will have a unique solution, or even a non-unique solution that constrains the parameters to lie within a plausible physiological range. Where parameters cannot be constrained they are termed 'unidentifiable'. We focus on gaining insight into the causes of unidentifiability using inference-based methods, and compare a recently developed measure-theoretic approach to inverse sensitivity analysis to the popular Markov chain Monte Carlo and approximate Bayesian computation techniques for Bayesian inference. All three approaches map the uncertainty in quantities of interest in the output space to the probability of sets of parameters in the input space. The geometry of these sets demonstrates how unidentifiability can be caused by parameter compensation and provides an intuitive approach to inference-based experimental design.
| S-EPMC6073654 | biostudies-literature
Project description:We examine the practical identifiability of parameters in a spatio-temporal reaction-diffusion model of a scratch assay. Experimental data involve fluorescent cell cycle labels, providing spatial information about cell position and temporal information about the cell cycle phase. Cell cycle labelling is incorporated into the reaction-diffusion model by treating the total population as two interacting subpopulations. Practical identifiability is examined using a Bayesian Markov chain Monte Carlo (MCMC) framework, confirming that the parameters are identifiable when we assume the diffusivities of the subpopulations are identical, but that the parameters are practically non-identifiable when we allow the diffusivities to be distinct. We also assess practical identifiability using a profile likelihood approach, providing similar results to MCMC with the advantage of being an order of magnitude faster to compute. Therefore, we suggest that the profile likelihood ought to be adopted as a screening tool to assess practical identifiability before MCMC computations are performed.
| S-EPMC7115235 | biostudies-literature
Project description:Cancer cell lines (CCLs) have been widely used to study of cancer. Recent studies have called into question the reliability of data collected on CCLs. Hence, we set out to determine CCLs that tend to be overly sensitive or resistant to a majority of drugs utilizing a nonlinear mixed-effects (NLME) modeling framework. Using drug response data collected in the Cancer Cell Line Encyclopedia (CCLE) and the Genomics of Drug Sensitivity in Cancer (GDSC), we determined the optimal functional form for each drug. Then, a NLME model was fit to the drug response data, with the estimated random effects used to determine sensitive or resistant CCLs. Out of the roughly 500 CCLs studies from the CCLE, we found 17 cell lines to be overly sensitive or resistant to the studied drugs. In the GDSC, we found 15 out of the 990 CCLs to be excessively sensitive or resistant. These results can inform researchers in the selection of CCLs to include in drug studies. Additionally, this study illustrates the need for assessing the dose-response functional form and the use of NLME models to achieve more stable estimates of drug response parameters.
| S-EPMC6783462 | biostudies-literature
Project description:Informative diagnostic tools are vital to the development of useful mixed-effects models. The Visual Predictive Check (VPC) is a popular tool for evaluating the performance of population PK and PKPD models. Ideally, a VPC will diagnose both the fixed and random effects in a mixed-effects model. In many cases, this can be done by comparing different percentiles of the observed data to percentiles of simulated data, generally grouped together within bins of an independent variable. However, the diagnostic value of a VPC can be hampered by binning across a large variability in dose and/or influential covariates. VPCs can also be misleading if applied to data following adaptive designs such as dose adjustments. The prediction-corrected VPC (pcVPC) offers a solution to these problems while retaining the visual interpretation of the traditional VPC. In a pcVPC, the variability coming from binning across independent variables is removed by normalizing the observed and simulated dependent variable based on the typical population prediction for the median independent variable in the bin. The principal benefit with the pcVPC has been explored by application to both simulated and real examples of PK and PKPD models. The investigated examples demonstrate that pcVPCs have an enhanced ability to diagnose model misspecification especially with respect to random effects models in a range of situations. The pcVPC was in contrast to traditional VPCs shown to be readily applicable to data from studies with a priori and/or a posteriori dose adaptations.
| S-EPMC3085712 | biostudies-literature
Project description:Shrinkage of empirical Bayes estimates (EBEs) of posterior individual parameters in mixed-effects models has been shown to obscure the apparent correlations among random effects and relationships between random effects and covariates. Empirical quantification equations have been widely used for population pharmacokinetic/pharmacodynamic models. The objectives of this manuscript were (1) to compare the empirical equations with theoretically derived equations, (2) to investigate and confirm the influencing factor on shrinkage, and (3) to evaluate the impact of shrinkage on estimation errors of EBEs using Monte Carlo simulations. A mathematical derivation was first provided for the shrinkage in nonlinear mixed effects model. Using a linear mixed model, the simulation results demonstrated that the shrinkage estimated from the empirical equations matched those based on the theoretically derived equations. Simulations with a two-compartment pharmacokinetic model verified that shrinkage has a reversed relationship with the relative ratio of interindividual variability to residual variability. Fewer numbers of observations per subject were associated with higher amount of shrinkage, consistent with findings from previous research. The influence of sampling times appeared to be larger when fewer PK samples were collected for each individual. As expected, sample size has very limited impact on shrinkage of the PK parameters of the two-compartment model. Assessment of estimation error suggested an average 1:1 relationship between shrinkage and median estimation error of EBEs.
| S-EPMC3475840 | biostudies-other
Project description:Genetic data is now collected in many clinical trials, especially in population pharmacokinetic studies. There is no consensus on methods to test the association between pharmacokinetics and genetic covariates. We performed a simulation study inspired by real clinical trials, using the pharmacokinetics (PK) of a compound under development having a nonlinear bioavailability along with genotypes for 176 single nucleotide polymorphisms (SNPs). Scenarios included 78 subjects extensively sampled (16 observations per subject) to simulate a phase I study, or 384 subjects with the same rich design. Under the alternative hypothesis (H1), six SNPs were drawn randomly to affect the log-clearance under an additive linear model. For each scenario, 200 PK data sets were simulated under the null hypothesis (no gene effect) and H1. We compared 16 combinations of four association tests, a stepwise procedure and three penalised regressions (ridge regression, Lasso, HyperLasso), applied to four pharmacokinetic phenotypes, two observed concentrations, area under the curve estimated by noncompartmental analysis and model-based clearance. The different combinations were compared in terms of true and false positives and probability to detect the genetic effects. In presence of nonlinearity and/or variability in bioavailability, model-based phenotype allowed a higher probability to detect the SNPs than other phenotypes. In a realistic setting with a limited number of subjects, all methods showed a low ability to detect genetic effects. Ridge regression had the best probability to detect SNPs, but also a higher number of false positives. No association test showed a much higher power than the others.
| S-EPMC4406951 | biostudies-literature
Project description:Increasingly, often ecologist collects data with nonlinear trends, heterogeneous variances, temporal correlation, and hierarchical structure. Nonlinear mixed-effects models offer a flexible approach to such data, but the estimation and interpretation of these models present challenges, partly associated with the lack of worked examples in the ecological literature.We illustrate the nonlinear mixed-effects modeling approach using temporal dynamics of vegetation moisture with field data from northwestern Patagonia. This is a Mediterranean-type climate region where modeling temporal changes in live fuel moisture content are conceptually relevant (ecological theory) and have practical implications (fire management). We used this approach to answer whether moisture dynamics varies among functional groups and aridity conditions, and compared it with other simpler statistical models. The modeling process is set out "step-by-step": We start translating the ideas about the system dynamics to a statistical model, which is made increasingly complex in order to include different sources of variability and correlation structures. We provide guidelines and R scripts (including a new self-starting function) that make data analyses reproducible. We also explain how to extract the parameter estimates from the R output.Our modeling approach suggests moisture dynamic to vary between grasses and shrubs, and between grasses facing different aridity conditions. Compared to more classical models, the nonlinear mixed-effects model showed greater goodness of fit and met statistical assumptions. While the mixed-effects approach accounts for spatial nesting, temporal dependence, and variance heterogeneity; the nonlinear function allowed to model the seasonal pattern.Parameters of the nonlinear mixed-effects model reflected relevant ecological processes. From an applied perspective, the model could forecast the time when fuel moisture becomes critical to fire occurrence. Due to the lack of worked examples for nonlinear mixed-effects models in the literature, our modeling approach could be useful to diverse ecologists dealing with complex data.
| S-EPMC6787861 | biostudies-literature
Project description:Many cancers are understood to be the product of multiple somatic mutations or other rate-limiting events. Multistage clonal expansion (MSCE) models are a class of continuous-time Markov chain models that capture the multi-hit initiation-promotion-malignant-conversion hypothesis of carcinogenesis. These models have been used broadly to investigate the epidemiology of many cancers, assess the impact of carcinogen exposures on cancer risk, and evaluate the potential impact of cancer prevention and control strategies on cancer rates. Structural identifiability (the analysis of the maximum parametric information available for a model given perfectly measured data) of certain MSCE models has been previously investigated. However, structural identifiability is a theoretical property and does not address the limitations of real data. In this study, we use pancreatic cancer as a case study to examine the practical identifiability of the two-, three-, and four-stage clonal expansion models given age-specific cancer incidence data using a numerical profile-likelihood approach. We demonstrate that, in the case of the three- and four-stage models, several parameters that are theoretically structurally identifiable, are, in practice, unidentifiable. This result means that key parameters such as the intermediate cell mutation rates are not individually identifiable from the data and that estimation of those parameters, even if structurally identifiable, will not be stable. We also show that products of these practically unidentifiable parameters are practically identifiable, and, based on this, we propose new reparameterizations of the model hazards that resolve the parameter estimation problems. Our results highlight the importance of identifiability to the interpretation of model parameter estimates.
| S-EPMC5367820 | biostudies-literature