Dataset Information

A multiple-model generalisation of updating clinical prediction models.

ABSTRACT: There is growing interest in developing clinical prediction models (CPMs) to aid local healthcare decision-making. Frequently, these CPMs are developed in isolation across different populations, with repetitive de novo derivation a common modelling strategy. However, this fails to utilise all available information and does not respond to changes in health processes through time and space. Alternatively, model updating techniques have previously been proposed that adjust an existing CPM to suit the new population, but these techniques are restricted to a single model. Therefore, we aimed to develop a generalised method for updating and aggregating multiple CPMs. The proposed "hybrid method" re-calibrates multiple CPMs using stacked regression while concurrently revising specific covariates using individual participant data (IPD) under a penalised likelihood. The performance of the hybrid method was compared with existing methods in a clinical example of mortality risk prediction after transcatheter aortic valve implantation, and in 2 simulation studies. The simulation studies explored the effect of sample size and between-population-heterogeneity on the method, with each representing a situation of having multiple distinct CPMs and 1 set of IPD. When the sample size of the IPD was small, stacked regression and the hybrid method had comparable but highest performance across modelling methods. Conversely, in large IPD samples, development of a new model and the hybrid method gave the highest performance. Hence, the proposed strategy can inform the choice between utilising existing CPMs or developing a model de novo, thereby incorporating IPD, existing research, and prior (clinical) knowledge into the modelling strategy.

SUBMITTER: Martin GP

PROVIDER: S-EPMC5873448 | biostudies-literature | 2018 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A multiple-model generalisation of updating clinical prediction models.

Martin Glen P GP Mamas Mamas A MA Peek Niels N Buchan Iain I Sperrin Matthew M

Statistics in medicine 20171218 8

There is growing interest in developing clinical prediction models (CPMs) to aid local healthcare decision-making. Frequently, these CPMs are developed in isolation across different populations, with repetitive de novo derivation a common modelling strategy. However, this fails to utilise all available information and does not respond to changes in health processes through time and space. Alternatively, model updating techniques have previously been proposed that adjust an existing CPM to suit t ...[more]

PMID: 29250812

Similar Datasets

Project description:Background While many clinical prediction models (CPMs) exist to guide valvular heart disease treatment decisions, the relative performance of these CPMs is largely unknown. We systematically describe the CPMs available for patients with valvular heart disease with specific attention to performance in external validations. Methods and Results A systematic review identified 49 CPMs for patients with valvular heart disease treated with surgery (n=34), percutaneous interventions (n=12), or no intervention (n=3). There were 204 external validations of these CPMs. Only 35 (71%) CPMs have been externally validated. Sixty-five percent (n=133) of the external validations were performed on distantly related populations. There was substantial heterogeneity in model performance and a median percentage change in discrimination of -27.1% (interquartile range, -49.4%--5.7%). Nearly two-thirds of validations (n=129) demonstrate at least a 10% relative decline in discrimination. Discriminatory performance of EuroSCORE II and Society of Thoracic Surgeons (2009) models (accounting for 73% of external validations) varied widely: EuroSCORE II validation c-statistic range 0.50 to 0.95; Society of Thoracic Surgeons (2009) Models validation c-statistic range 0.50 to 0.86. These models performed well when tested on related populations (median related validation c-statistics: EuroSCORE II, 0.82 [0.76, 0.85]; Society of Thoracic Surgeons [2009], 0.72 [0.67, 0.79]). There remain few (n=9) external validations of transcatheter aortic valve replacement CPMs. Conclusions Many CPMs for patients with valvular heart disease have never been externally validated and isolated external validations appear insufficient to assess the trustworthiness of predictions. For surgical valve interventions, there are existing predictive models that perform reasonably well on related populations. For transcatheter aortic valve replacement (CPMs additional external validations are needed to broadly understand the trustworthiness of predictions.

Dataset Information

A multiple-model generalisation of updating clinical prediction models.

Publications

A multiple-model generalisation of updating clinical prediction models.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure