Dataset Information

Assessing the performance of different outcomes for tumor growth studies with animal models.

ABSTRACT: The consistency of reporting results for patient-derived xenograft (PDX) studies is an area of concern. The PDX method commonly starts by implanting a derivative of a human tumor into a mouse, then comparing the tumor growth under different treatment conditions. Currently, a wide array of statistical methods (e.g., t-test, regression, chi-squared test) are used to analyze these data, which ultimately depend on the outcome chosen (e.g., tumor volume, relative growth, categorical growth). In this simulation study, we provide empirical evidence for the outcome selection process by comparing the performance of both commonly used outcomes and novel variations of common outcomes used in PDX studies. Data were simulated to mimic tumor growth under multiple scenarios, then each outcome of interest was evaluated for 10 000 iterations. Comparisons between different outcomes were made with respect to average bias, variance, type-1 error, and power. A total of 18 continuous, categorical, and time-to-event outcomes were evaluated, with ultimately 2 outcomes outperforming the others: final tumor volume and change in tumor volume from baseline. Notably, the novel variations of the tumor growth inhibition index (TGII)-a commonly used outcome in PDX studies-was found to perform poorly in several scenarios with inflated type-1 error rates and a relatively large bias. Finally, all outcomes of interest were applied to a real-world dataset.

SUBMITTER: Patten LW

PROVIDER: S-EPMC9240739 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:IntroductionExternal validation studies are essential to study the generalizability of prediction models. Recently a permutation test, focusing on discrimination as quantified by the c-statistic, was proposed to judge whether a prediction model is transportable to a new setting. We aimed to evaluate this test and compare it to previously proposed procedures to judge any changes in c-statistic from development to external validation setting.MethodsWe compared the use of the permutation test to the use of benchmark values of the c-statistic following from a previously proposed framework to judge transportability of a prediction model. In a simulation study we developed a prediction model with logistic regression on a development set and validated them in the validation set. We concentrated on two scenarios: 1) the case-mix was more heterogeneous and predictor effects were weaker in the validation set compared to the development set, and 2) the case-mix was less heterogeneous in the validation set and predictor effects were identical in the validation and development set. Furthermore we illustrated the methods in a case study using 15 datasets of patients suffering from traumatic brain injury.ResultsThe permutation test indicated that the validation and development set were homogenous in scenario 1 (in almost all simulated samples) and heterogeneous in scenario 2 (in 17%-39% of simulated samples). Previously proposed benchmark values of the c-statistic and the standard deviation of the linear predictors correctly pointed at the more heterogeneous case-mix in scenario 1 and the less heterogeneous case-mix in scenario 2.ConclusionThe recently proposed permutation test may provide misleading results when externally validating prediction models in the presence of case-mix differences between the development and validation population. To correctly interpret the c-statistic found at external validation it is crucial to disentangle case-mix differences from incorrect regression coefficients.

Dataset Information

Assessing the performance of different outcomes for tumor growth studies with animal models.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets