Dataset Information

An alternative trial-level measure for evaluating failure-time surrogate endpoints based on prediction error.

ABSTRACT: To validate a failure-time surrogate for an established failure-time clinical endpoint such as overall survival, the meta-analytic approach is commonly used. The standard correlation approach considers two levels: the individual level, with Kendall's ? measuring the rank correlation between the endpoints, and the trial level, with the coefficient of determination R 2 measuring the correlation between the treatment effects on the surrogate and on the final endpoint. However, the estimation of R 2 is not robust with respect to the estimation error of the trial-specific treatment effects. The alternative proposed in this article uses a prediction error based on a measure of the weighted difference between the observed treatment effect on the final endpoint and a model-based predicted effect. The measures can be estimated by cross-validation within the meta-analytic setting or external validation on a set of trials. Several distances are presented, with varying weights, based on the standard error of the observed treatment effect and of its predicted value. A simulation study was conducted under different scenarios, varying the number and the size of the trials, Kendall's ? and R 2 . These measures have been applied to individual patient data from a meta-analysis of trials in advanced/recurrent gastric cancer (20 randomized trials of chemotherapy, 4069 patients). The distance-based measures appeared to be robust with respect to different values of simulation parameters in several scenarios (such as Kendall's ?, size and number of clinical trials). The absolute prediction error can be an alternative to the trial-level R 2 for evaluation of candidate time-to-event surrogates.

SUBMITTER: Belhechmi S

PROVIDER: S-EPMC6627581 | biostudies-literature | 2019 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An alternative trial-level measure for evaluating failure-time surrogate endpoints based on prediction error.

Belhechmi Shaima S Michiels Stefan S Paoletti Xavier X Rotolo Federico F

Contemporary clinical trials communications 20190705

To validate a failure-time surrogate for an established failure-time clinical endpoint such as overall survival, the meta-analytic approach is commonly used. The standard correlation approach considers two levels: the individual level, with Kendall's <i>τ</i> measuring the rank correlation between the endpoints, and the trial level, with the coefficient of determination R 2 measuring the correlation between the treatment effects on the surrogate and on the final endpoint. However, the estim ...[more]

PMID: 31338479

Similar Datasets

Project description:To evaluate Hotelling's T(2) statistic and the input variable squared prediction error (Q((X))) for detecting large respiratory surrogate-based tumor displacement prediction errors without directly measuring the tumor's position.Tumor and external marker positions from a database of 188 Cyberknife Synchrony™ lung, liver, and pancreas treatment fractions were analyzed. The first ten measurements of tumor position in each fraction were used to create fraction-specific models of tumor displacement using external surrogates as input; the models were used to predict tumor position from subsequent external marker measurements. A partial least squares (PLS) model with four scores was developed for each fraction to determine T(2) and Q((X)) confidence limits based on the first ten measurements in a fraction. The T(2) and Q((X)) statistics were then calculated for every set of external marker measurements. Correlations between model error and both T(2) and Q((X)) were determined. Receiver operating characteristic analysis was applied to evaluate sensitivities and specificities of T(2), Q((X)), and T(2)∪Q((X)) for predicting real-time tumor localization errors >3 mm over a range of T(2) and Q((X)) confidence limits.Sensitivity and specificity of detecting errors >3 mm varied with confidence limit selection. At 95% sensitivity, T(2)∪Q((X)) specificity was 15%, 2% higher than either T(2) or Q((X)) alone. The mean time to alarm for T(2)∪Q((X)) at 95% sensitivity was 5.3 min but varied with a standard deviation of 8.2 min. Results did not differ significantly by tumor site.The results of this study establish the feasibility of respiratory surrogate-based online monitoring of real-time respiration-induced tumor motion model accuracy for lung, liver, and pancreas tumors. The T(2) and Q((X)) statistics were able to indicate whether inferential model errors exceeded 3 mm with high sensitivity. Modest improvements in specificity were achieved by combining T(2) and Q((X)) results.

Project description:Background:In July 2018, the FDA first published a table listing all surrogate measures that it has used, and may accept for future use, in regulatory approval. However, the strength of surrogacy for those measures was not formally assessed. Using the case example of breast cancer, we aimed to evaluate the strength of correlation of surrogate measures listed in the FDA's Table with overall survival. Methods:This cross-sectional study of the FDA's Table of Surrogate Endpoints was conducted in May 2019. All surrogate measures listed in the FDA table as appropriate for accelerated or regular approval for breast cancer were extracted. We identified studies evaluating the correlation of treatment benefit in the surrogate with treatment benefit in overall survival and extracted results from the correlation analysis. Findings:Five surrogate endpoints were listed for breast cancer in the FDA website: pathological complete response rates (pCR), event-free survival (EFS), disease-free survival (DFS), objective response rates (ORR), and progression-free survival (PFS), of which pCR was listed as appropriate only for accelerated approval, while the rest were considered appropriate for accelerated or regular approval. No correlation study evaluated the correlation of treatment effects on EFS with that on OS. The results from correlation studies evaluating pCR, DFS, ORR, and PFS suggest that the treatment effects on none of these surrogate measures were strongly correlated with treatment effects on OS (r<0.85 or R2 < 0.7, except for DFS in HER2 positive early breast cancer (R2 = 0.75). Interpretation:Using breast cancer as an example, we evaluated the underlying evidence for the surrogate endpoints for solid tumors listed in the FDA's Table of Surrogate Endpoints and found weak or missing correlations of treatment effects on these surrogates with treatment effects on OS . Surrogate measures should be predictive of clinical benefit to be useful in supporting regular FDA approval. Funding:Work on this project was funded by the Arnold Ventures. Dr. Kesselheim is also supported by the Harvard-MIT Center for Regulatory Science. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Dataset Information

An alternative trial-level measure for evaluating failure-time surrogate endpoints based on prediction error.

Publications

An alternative trial-level measure for evaluating failure-time surrogate endpoints based on prediction error.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets