Dataset Information

Assessing parameter uncertainty in small-n pharmacometric analyses: value of the log-likelihood profiling-based sampling importance resampling (LLP-SIR) technique.

ABSTRACT: Assessing parameter uncertainty is a crucial step in pharmacometric workflows. Small datasets with ten or fewer subjects appear regularly in drug development and therapeutic use, but it is unclear which method to assess parameter uncertainty is preferable in such situations. The aim of this study was to (i) systematically evaluate the performance of standard error (SE), bootstrap (BS), log-likelihood profiling (LLP), Bayesian approaches (BAY) and sampling importance resampling (SIR) to assess parameter uncertainty in small datasets and (ii) to evaluate methods to provide proposal distributions for the SIR. A simulation study was conducted and the 0-95% confidence interval (CI) and coverage for each parameter was evaluated and compared to reference CIs derived by stochastic simulation and estimation (SSE). A newly proposed LLP-SIR, combining the proposal distribution provided by LLP with SIR, was included in addition to conventional SE-SIR and BS-SIR. Additionally, the methods were applied to a clinical dataset. The determined CIs differed substantially across the methods. The CIs of SE, BS, LLP and BAY were not in line with the reference in datasets with???10 subjects. The best alignment was found for the LLP-SIR, which also provided the best coverage results among the SIR methods. The best overall results regarding the coverage were provided by LLP and BAY across all parameters and dataset sizes. To conclude, the popular SE and BS methods are not suitable to derive parameter uncertainty in small datasets containing???10 subjects, while best performances were observed with LLP, BAY and LLP-SIR.

SUBMITTER: Broeker A

PROVIDER: S-EPMC7289778 | biostudies-literature | 2020 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing parameter uncertainty in small-n pharmacometric analyses: value of the log-likelihood profiling-based sampling importance resampling (LLP-SIR) technique.

Broeker Astrid A Wicha Sebastian G SG

Journal of pharmacokinetics and pharmacodynamics 20200404 3

Assessing parameter uncertainty is a crucial step in pharmacometric workflows. Small datasets with ten or fewer subjects appear regularly in drug development and therapeutic use, but it is unclear which method to assess parameter uncertainty is preferable in such situations. The aim of this study was to (i) systematically evaluate the performance of standard error (SE), bootstrap (BS), log-likelihood profiling (LLP), Bayesian approaches (BAY) and sampling importance resampling (SIR) to assess pa ...[more]

PMID: 32248328

Similar Datasets

Project description:Taking parameter uncertainty into account is key to make drug development decisions such as testing whether trial endpoints meet defined criteria. Currently used methods for assessing parameter uncertainty in NLMEM have limitations, and there is a lack of diagnostics for when these limitations occur. In this work, a method based on sampling importance resampling (SIR) is proposed, which has the advantage of being free of distributional assumptions and does not require repeated parameter estimation. To perform SIR, a high number of parameter vectors are simulated from a given proposal uncertainty distribution. Their likelihood given the true uncertainty is then approximated by the ratio between the likelihood of the data given each vector and the likelihood of each vector given the proposal distribution, called the importance ratio. Non-parametric uncertainty distributions are obtained by resampling parameter vectors according to probabilities proportional to their importance ratios. Two simulation examples and three real data examples were used to define how SIR should be performed with NLMEM and to investigate the performance of the method. The simulation examples showed that SIR was able to recover the true parameter uncertainty. The real data examples showed that parameter 95 % confidence intervals (CI) obtained with SIR, the covariance matrix, bootstrap and log-likelihood profiling were generally in agreement when 95 % CI were symmetric. For parameters showing asymmetric 95 % CI, SIR 95 % CI provided a close agreement with log-likelihood profiling but often differed from bootstrap 95 % CI which had been shown to be suboptimal for the chosen examples. This work also provides guidance towards the SIR workflow, i.e.,which proposal distribution to choose and how many parameter vectors to sample when performing SIR, using diagnostics developed for this purpose. SIR is a promising approach for assessing parameter uncertainty as it is applicable in many situations where other methods for assessing parameter uncertainty fail, such as in the presence of small datasets, highly nonlinear models or meta-analysis.

Project description:Introduction: Computational models of the cardiovascular system are widely used to simulate cardiac (dys)function. Personalization of such models for patient-specific simulation of cardiac function remains challenging. Measurement uncertainty affects accuracy of parameter estimations. In this study, we present a methodology for patient-specific estimation and uncertainty quantification of parameters in the closed-loop CircAdapt model of the human heart and circulation using echocardiographic deformation imaging. Based on patient-specific estimated parameters we aim to reveal the mechanical substrate underlying deformation abnormalities in patients with arrhythmogenic cardiomyopathy (AC). Methods: We used adaptive multiple importance sampling to estimate the posterior distribution of regional myocardial tissue properties. This methodology is implemented in the CircAdapt cardiovascular modeling platform and applied to estimate active and passive tissue properties underlying regional deformation patterns, left ventricular volumes, and right ventricular diameter. First, we tested the accuracy of this method and its inter- and intraobserver variability using nine datasets obtained in AC patients. Second, we tested the trueness of the estimation using nine in silico generated virtual patient datasets representative for various stages of AC. Finally, we applied this method to two longitudinal series of echocardiograms of two pathogenic mutation carriers without established myocardial disease at baseline. Results: Tissue characteristics of virtual patients were accurately estimated with a highest density interval containing the true parameter value of 9% (95% CI [0-79]). Variances of estimated posterior distributions in patient data and virtual data were comparable, supporting the reliability of the patient estimations. Estimations were highly reproducible with an overlap in posterior distributions of 89.9% (95% CI [60.1-95.9]). Clinically measured deformation, ejection fraction, and end-diastolic volume were accurately simulated. In presence of worsening of deformation over time, estimated tissue properties also revealed functional deterioration. Conclusion: This method facilitates patient-specific simulation-based estimation of regional ventricular tissue properties from non-invasive imaging data, taking into account both measurement and model uncertainties. Two proof-of-principle case studies suggested that this cardiac digital twin technology enables quantitative monitoring of AC disease progression in early stages of disease.

Dataset Information

Assessing parameter uncertainty in small-n pharmacometric analyses: value of the log-likelihood profiling-based sampling importance resampling (LLP-SIR) technique.

Publications

Assessing parameter uncertainty in small-n pharmacometric analyses: value of the log-likelihood profiling-based sampling importance resampling (LLP-SIR) technique.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets