External validation of models for KIR2DS1/KIR3DL1-informed selection of hematopoietic cell donors fails.
Ontology highlight
ABSTRACT:
SUBMITTER: Schetelig J
PROVIDER: S-EPMC7162689 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Results from registry studies suggest that harnessing Natural Killer (NK) cell reactivity mediated through Killer cell Immunoglobulin-like Receptors (KIR) could reduce the risk of relapse after allogeneic Hematopoietic Cell Transplantation (HCT). Several competing models have been developed to classify donors as KIR-advantageous or disadvantageous. Basically, these models differ by grouping donors based on distinct KIR-KIR-ligand combinations or by haplotype motif assignment. This study aimed to validate different models for unrelated donor selection for patients with Myelodysplatic Syndromes (MDS) or secondary Acute Myeloid Leukemia (sAML). In a joint retrospective study of the European Society for Blood and Marrow Transplantation (EBMT) and the Center for International Blood and Marrow Transplant Research (CIBMTR) registry data from 1704 patients with secondary AML or MDS were analysed. The cohort consisted mainly of older patients (median age 61 years) with high risk disease who had received chemotherapy-based reduced intensity conditioning and anti-thymocyte globulin prior to allogeneic HCT from well-matched unrelated stem cell donors. The impact of the predictors on Overall Survival (OS) and relapse incidence was tested in Cox regression models adjusted for patient age, a modified disease risk index, performance status, donor age, HLA-match, sex-match, CMV-match, conditioning intensity, type of T-cell depletion and graft type. KIR genes were typed using high-resolution amplicon-based next generation sequencing. In univariable and multivariable analyses none of the models predicted OS and the risk of relapse consistently. Our results do not support the hypothesis that optimizing NK-mediated alloreactivity is possible by KIR-genotype informed selection of HLA-matched unrelated donors. However, in the context of allogeneic transplantation, NK-cell biology is complex and only partly understood. KIR-genes are highly diverse and current assignment of haplotype motifs based on the presence or absence of selected KIR genes is over-simplistic. As a consequence, further research is highly warranted and should integrate cutting edge knowledge on KIR genetics, and NK-cell biology into future studies focused on homogeneous groups of patients and treatment modalities.
| S-EPMC7851088 | biostudies-literature
Project description:KIR3DL1 is a natural killer (NK) cell receptor that recognizes the Bw4 epitope of human leukocyte antigen (HLA) class I molecules. Following hematopoietic stem cell transplantation for patients lacking Bw4, KIR3DL1-expressing NK cells from Bw4-positive donors can be alloreactive and eliminate tumor cells. However, KIR3DL1 alleles having T instead of C at nucleotide 320 (encoding leucine 86 instead of serine 86) are not expressed on the cell surface. Thus, not all individuals testing positive for KIR3DL1 are optimal donors for Bw4-negative recipients. Therefore, we developed a method for genotyping codon 86, which was validated by its perfect correlation with NK cell phenotype for 100 donors of diverse KIR3DL1/S1 genotype. We typed 600 donors and found that ∼12.2% had the KIR3DL1 gene, but did not express cell-surface KIR3DL1. By contrast, high-expressing allotypes were identified when haplotypes from four families with duplicated KIR3DL1/S1 genes were characterized at high resolution. Identifying donors who have KIR3DL1 but lack cell-surface KIR3DL1 would refine donor selection. With this technique, the number of individuals identified who may not be optimal donors for Bw4-negative patients increases by threefold, when compared with standard methods. Taken together, we propose that allele typing of killer cell Ig-like receptor (KIR) polymorphisms should become a standard practice when selecting donors.
| S-EPMC5065926 | biostudies-literature
Project description:BackgroundSex-specific prediction models for low hemoglobin (Hb) deferral have been developed in Dutch whole blood donors. In this study, we validated and updated the models in a cohort of Swiss whole blood donors.MethodsProspectively collected data from 53,772 Swiss whole blood donors were used. The predictive performance of the Dutch models was assessed in terms of calibration (agreement between predicted probabilities and observed frequencies) and discrimination (ability to discriminate between deferred and approved donors). The models were updated by revising the strength of the individual predictors in the models.ResultsA total of 1,065 men (3.3%) and 2,063 women (9.7%) were deferred from donation because of a low Hb level. Validation in Swiss donors demonstrated underestimation of predicted risks and significantly lower discriminative ability. The predictive effects of most predictors were weaker in Swiss donors. Updating the models increased the calibration for both men and women, and slightly increased the discriminative ability in men.ConclusionValidation of the Dutch prediction models in Swiss whole blood donors showed lower, though adequate performance. In general, the Dutch prediction models can reliably predict the risk of Hb deferral, although for application in other countries small adaptations are necessary.
| S-EPMC5159723 | biostudies-literature
Project description:Background and objectiveThe immunosuppressant everolimus is increasingly applied in renal transplantation. Its extensive pharmacokinetic variability necessitates therapeutic drug monitoring, typically based on whole-blood trough concentrations (C0). Unfortunately, therapeutic drug monitoring target attainment rates are often unsatisfactory and patients with on-target exposure may still develop organ rejection. As everolimus displays erythrocyte partitioning, haematocrit-normalised whole-blood exposure has been suggested as a more informative therapeutic drug monitoring marker. Furthermore, model-informed precision dosing has introduced options for more sophisticated dose adaptation. We have previously developed a mechanistic population pharmacokinetic model, which described everolimus plasma pharmacokinetics and enabled estimation of haematocrit-normalised whole-blood exposure. Here, we externally evaluated this model for its utility for model-informed precision dosing.MethodsThe retrospective dataset included 4123 pharmacokinetic observations from routine clinical therapeutic drug monitoring in 173 renal transplant recipients. Model appropriateness was confirmed with a visual predictive check. A fit-for-purpose analysis was conducted to evaluate whether the model accurately and precisely predicted a future C0 or area under the concentration-time curve (AUC) from prior pharmacokinetic observations. Bias and imprecision were expressed as the mean percentage prediction error (MPPE) and mean absolute percentage prediction error (MAPE), stratified on 6 months post-transplant. Additionally, we compared dose adaptation recommendations of conventional C0-based therapeutic drug monitoring and C0- or AUC-based model-informed precision dosing, and assessed the percentage of differences between observed and haematocrit-normalised C0 (∆C0) and AUC (∆AUC) exceeding ± 20%.ResultsThe model showed adequate accuracy and precision for C0 and AUC prediction at ≤ 6 months (MPPEC0: 8.1 ± 2.5%, MAPEC0: 26.8 ± 2.1%; MPPEAUC: - 9.7 ± 5.1%, MAPEAUC: 13.3 ± 3.9%) and > 6 months post-transplant (MPPEC0: 4.7 ± 2.0%, MAPEC0: 25.4 ± 1.4%; MPPEAUC: - 0.13 ± 4.8%, MAPEAUC: 13.3 ± 2.8%). On average, dose adaptation recommendations derived from C0-based and AUC-based model-informed precision dosing were 2.91 ± 0.01% and 13.7 ± 0.18% lower than for conventional C0-based therapeutic drug monitoring at ≤ 6 months, and 0.93 ± 0.01% and 3.14 ± 0.04% lower at > 6 months post-transplant. The ∆C0 and ∆AUC exceeded ± 20% on 13.6% and 14.3% of occasions, respectively.ConclusionsWe demonstrated that our population pharmacokinetic model was able to accurately and precisely predict future everolimus exposure from prior pharmacokinetic measurements. In addition, we illustrated the potential added value of performing everolimus therapeutic drug monitoring with haematocrit-normalised whole-blood concentrations. Our results provide reassurance to implement this methodology in clinical practice for further evaluation.
| S-EPMC7862213 | biostudies-literature
Project description:Natural killer (NK) cells are potent mediators of the graft versus leukemia phenomenon critical to the success of allogeneic hematopoietic cell transplantation. Central to calibrating NK effector function via their interaction with class I human leukocyte antigens are the numerous inhibitory killer Ig-like receptors (KIR). The KIR receptors are encoded by a family of polymorphic genes, whose expression is largely stochastic and uninfluenced by human leukocyte antigens genotype. These features provide the opportunity to select hematopoietic cell donors with favorable KIR genotypes that confer enhanced protection from relapse via NK-mediated graft versus leukemia. Over the last 2 decades, a large body of work has emerged examining the use of KIR genotyping to stratify potential donors based on anticipated NK alloreactivity. Overall, these results support KIR-based donor selection for patients undergoing allogeneic hematopoietic cell transplantation for a diagnosis of acute myelogenous leukemia. Despite this, the underlying factors that control NK cell responsiveness are not completely understood, and opportunities remain to refine donor selection using NK cell receptor genotyping. In this review, we will summarize the relevant findings with respect to KIR genotyping as a selection parameter for allogeneic hematopoietic cell donors and address practical considerations with respect to KIR-based selection of donors for patients with myeloid neoplasia.
| S-EPMC7708684 | biostudies-literature
Project description:To externally validate the prognostic models for predicting the time-dependent outcome in patients with locally advanced cervical cancer (LACC) who were treated with concurrent chemoradiotherapy in an independent cohort.A historical cohort of 297 women with LACC who were treated with radical concurrent chemoradiotherapy from 1999 to 2014 at the 12 de Octubre University Hospital (H12O), Madrid, Spain. The external validity of prognostic models was quantified regarding discrimination, calibration, measures of overall performance, and decision curve analyses.The review identified 8 studies containing 13 prognostic models. Different (International Federation of Gynecology and Obstetrics [FIGO] stages, parametrium involvement, hydronephrosis, location of positive nodes, and race) but related cohorts with validation cohort (5-year overall survival [OS]=70%; 5-year disease-free survival [DFS]=64%; average age of 50; and over 79% squamous cell) were evaluated. The following models exhibited good external validity in terms of discrimination and calibration but limited clinical utility: the OS model at 3 year from Kidd et al.'s study (area under the receiver operating characteristic curve [AUROC]=0.69; threshold of clinical utility [TCU] between 36% and 50%), the models of DFS at 1 year from Kidd et al.'s study (AUROC=0.64; TCU between 24% and 32%) and 2 years from Rose et al.'s study (AUROC=0.70; TCU between 19% and 58%) and the distant recurrence model at 5 years from Kang et al.'s study (AUROC=0.67; TCU between 12% and 36%).The external validation revealed the statistical and clinical usefulness of 4 prognostic models published in the literature.
| S-EPMC5540718 | biostudies-literature
Project description:IntroductionExternal validation studies are essential to study the generalizability of prediction models. Recently a permutation test, focusing on discrimination as quantified by the c-statistic, was proposed to judge whether a prediction model is transportable to a new setting. We aimed to evaluate this test and compare it to previously proposed procedures to judge any changes in c-statistic from development to external validation setting.MethodsWe compared the use of the permutation test to the use of benchmark values of the c-statistic following from a previously proposed framework to judge transportability of a prediction model. In a simulation study we developed a prediction model with logistic regression on a development set and validated them in the validation set. We concentrated on two scenarios: 1) the case-mix was more heterogeneous and predictor effects were weaker in the validation set compared to the development set, and 2) the case-mix was less heterogeneous in the validation set and predictor effects were identical in the validation and development set. Furthermore we illustrated the methods in a case study using 15 datasets of patients suffering from traumatic brain injury.ResultsThe permutation test indicated that the validation and development set were homogenous in scenario 1 (in almost all simulated samples) and heterogeneous in scenario 2 (in 17%-39% of simulated samples). Previously proposed benchmark values of the c-statistic and the standard deviation of the linear predictors correctly pointed at the more heterogeneous case-mix in scenario 1 and the less heterogeneous case-mix in scenario 2.ConclusionThe recently proposed permutation test may provide misleading results when externally validating prediction models in the presence of case-mix differences between the development and validation population. To correctly interpret the c-statistic found at external validation it is crucial to disentangle case-mix differences from incorrect regression coefficients.
| S-EPMC4755533 | biostudies-literature
Project description:We propose a cost-effective sampling design and estimating procedure for validating personal risk models using right-censored cohort data. Validation involves using each subject's covariates, as ascertained at cohort entry, in a risk model (specified independently of the data) to assign him/her a probability of an adverse outcome within a future time period. Subjects are then grouped according to the magnitudes of their assigned risks, and within each group, the mean assigned risk is compared with the probability of outcome occurrence as estimated using the follow-up data. Such validation presents two complications. First, in the presence of right-censoring, estimating the probability of developing the outcomes before death requires competing risk analysis. Second, for rare outcomes, validation using the full cohort requires assembling covariates and assigning risks to thousands of subjects. This can be costly when some covariates involve analyzing biological specimens. A two-stage sampling design addresses this problem by assembling covariates and assigning risks only to those subjects most informative for estimating key parameters. We use this design to estimate the outcome probabilities needed to evaluate model performance and we provide theoretical and bootstrap estimates of their variances. We also describe how to choose two-stage designs with minimal efficiency loss for a parameter of interest when the quantities determining optimality are unknown at the time of design. We illustrate these methods by using subjects in the California Teachers Study to validate ovarian cancer risk models. We find that a design with optimal efficiency for one performance parameter need not be so for others, and trade-offs will be required. A two-stage design that samples all outcome-positive subjects and more outcome-negative than censored subjects will perform well in most circumstances. The methods are implemented in Risk Model Assessment Program, an R program freely available at http://med.stanford.edu/epidemiology/two-stage.html.
| S-EPMC3971015 | biostudies-literature
Project description:BackgroundSeveral prognostic models for outcomes after chronic subdural hematoma (CSDH) treatment have been published in recent years. However, these models are not sufficiently validated for use in daily clinical practice. We aimed to assess the performance of existing prediction models for outcomes in patients diagnosed with CSDH.MethodsWe systematically searched relevant literature databases up to February 2021 to identify prognostic models for outcome prediction in patients diagnosed with CSDH. For the external validation of prognostic models, we used a retrospective database, containing data of 2384 patients from three Dutch regions. Prognostic models were included if they predicted either mortality, hematoma recurrence, functional outcome, or quality of life. Models were excluded when predictors were absent in our database or available for < 150 patients in our database. We assessed calibration, and discrimination (quantified by the concordance index C) of the included prognostic models in our retrospective database.ResultsWe identified 1680 original publications of which 1656 were excluded based on title or abstract, mostly because they did not concern CSDH or did not define a prognostic model. Out of 18 identified models, three could be externally validated in our retrospective database: a model for 30-day mortality in 1656 patients, a model for 2 months, and another for 3-month hematoma recurrence both in 1733 patients. The models overestimated the proportion of patients with these outcomes by 11% (15% predicted vs. 4% observed), 1% (10% vs. 9%), and 2% (11% vs. 9%), respectively. Their discriminative ability was poor to modest (C of 0.70 [0.63-0.77]; 0.46 [0.35-0.56]; 0.59 [0.51-0.66], respectively).ConclusionsNone of the examined models showed good predictive performance for outcomes after CSDH treatment in our dataset. This study confirms the difficulty in predicting outcomes after CSDH and emphasizes the heterogeneity of CSDH patients. The importance of developing high-quality models by using unified predictors and relevant outcome measures and appropriate modeling strategies is warranted.
| S-EPMC9519711 | biostudies-literature
Project description:BackgroundMany dementia prediction models have been developed, but only few have been externally validated, which hinders clinical uptake and may pose a risk if models are applied to actual patients regardless. Externally validating an existing prediction model is a difficult task, where we mostly rely on the completeness of model reporting in a published article. In this study, we aim to externally validate existing dementia prediction models. To that end, we define model reporting criteria, review published studies, and externally validate three well reported models using routinely collected health data from administrative claims and electronic health records.MethodsWe identified dementia prediction models that were developed between 2011 and 2020 and assessed if they could be externally validated given a set of model criteria. In addition, we externally validated three of these models (Walters' Dementia Risk Score, Mehta's RxDx-Dementia Risk Index, and Nori's ADRD dementia prediction model) on a network of six observational health databases from the United States, United Kingdom, Germany and the Netherlands, including the original development databases of the models.ResultsWe reviewed 59 dementia prediction models. All models reported the prediction method, development database, and target and outcome definitions. Less frequently reported by these 59 prediction models were predictor definitions (52 models) including the time window in which a predictor is assessed (21 models), predictor coefficients (20 models), and the time-at-risk (42 models). The validation of the model by Walters (development c-statistic: 0.84) showed moderate transportability (0.67-0.76 c-statistic). The Mehta model (development c-statistic: 0.81) transported well to some of the external databases (0.69-0.79 c-statistic). The Nori model (development AUROC: 0.69) transported well (0.62-0.68 AUROC) but performed modestly overall. Recalibration showed improvements for the Walters and Nori models, while recalibration could not be assessed for the Mehta model due to unreported baseline hazard.ConclusionWe observed that reporting is mostly insufficient to fully externally validate published dementia prediction models, and therefore, it is uncertain how well these models would work in other clinical settings. We emphasize the importance of following established guidelines for reporting clinical prediction models. We recommend that reporting should be more explicit and have external validation in mind if the model is meant to be applied in different settings.
| S-EPMC9720950 | biostudies-literature