Dataset Information

External validation of LCR1-LCR2, a multivariable HCC risk calculator, in patients with chronic HCV.

ABSTRACT:

Background & aims

The Liver Cancer Risk test algorithm (LCR1-LCR2) is a multianalyte blood test combining proteins involved in liver cell repair (apolipoprotein-A1 and haptoglobin), known hepatocellular carcinoma (HCC) risk factors (sex, age, and gamma-glutamyl transferase), a marker of fibrosis (alpha2-macroglobulin) and alpha-fetoprotein (AFP), a specific marker of HCC. The aim was to externally validate the LCR1-LCR2 in patients with chronic HCV (CHC) treated or not with antivirals.

Methods

Pre-included patients were from the Hepather cohort, a multicentre prospective study in adult patients with CHC in France. LCR1-LCR2 was assessed retrospectively in patients with the test components and AFP, available at baseline. The co-primary study outcome was the negative predictive value (NPV) of LCR1-LCR2 for the occurrence of HCC at 5 years and for survival without HCC according to the predetermined LCR1-LCR2 cut-offs. The cut-offs were adjusted for risk covariables and for the response to HCV treatment, and were quantified using time-dependent proportional hazards models.

Results

In total, 4,903 patients, 1,026 (21.9%) with baseline cirrhosis, were included in the study. Patients were followed for a median of 5.7 (IQR 4.2-11.3) years. A total of 3,788/4,903 (77.3%) patients had a sustained virological response. There were 137 cases of HCC at 5 years and 214 at the end of follow-up. HCC occurred at 5 years in 24/3,755 patients with low-risk LCR1-LCR2 compared with 113/1,148 patients with high-risk LCR1-LCR2. The NPV was 99.4% (95% CI 99.1-99.6). Similar findings (hazard ratio, 10.8; 95% CI, 8.1-14.3; p <0.001) were obtained after adjustment for exposure to antivirals, age, sex, geographical origin, HCV genotype 3, alcohol consumption, and type 2 diabetes mellitus.

Conclusions

The results showed that LCR1-LCR2 can be used to successfully identify patients with HCV at very low risk of HCC at 5 years.

Lay summary

Hepatocellular carcinoma (HCC) is the fourth leading cause of cancer-related death worldwide and the fastest growing cause of cancer death in many countries. We constructed and internally validated a new multianalyte blood test to assess this Liver Cancer Risk (LCR1-LCR2). This study confirmed the performance of LCR1-LCR2 in patients with chronic HCV in the national French cohort Hepather, and its ability to identify patients at a very low risk of HCC at 5 years.

Clinical trials registration

The study is registered at ClinicalTrials.gov (NCT01953458).

SUBMITTER: Poynard T

PROVIDER: S-EPMC8187244 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundExternal validation of risk calculators (RCs) is necessary to determine their clinical applicability beyond the setting in which these were developed.ObjectiveTo assess the performance of the Rotterdam Prostate Cancer RC (RPCRC) and the Prostate Biopsy Collaborative Group RC (PBCG-RC).Design setting and participantsWe used data from the prospective, population-based STHLM3 screening study, performed in 2012-2015. Participants with prostate-specific antigen ≥3 ng/ml who underwent systematic prostate biopsies were included.Outcome measurements and statistical analysisProbabilities for clinically significant prostate cancer (csPCa), defined as International Society of Urological Pathology grade ≥2, were calculated for each participant. External validity was assessed by calibration, discrimination, and clinical usefulness for both original and recalibrated models.Results and limitationsOut of 5841 men, 1054 (18%) had csPCa. Distribution of risk predictions differed between RCs; median risks for csPCa using the RPCRC and PBCG-RC were 3.3% (interquartile range [IQR] 2.1-7.1%) and 20% (IQR 15-28%), respectively. The correlation between RC risk estimates on individual level was moderate (Spearman's r = 0.55). Using the RPCRC's recommended risk threshold of ≥4% for finding csPCa, 36% of participants would get concordant biopsy recommendations. At 10% risk cut-off, RCs agreed in 23% of cases. Both RCs showed good discrimination, with areas under the curves for the RPCRC of 0.74 (95% confidence interval [CI] 0.72-0.76) and the PBCG-RC of 0.70 (95% CI 0.68-0.72). Calibration was adequate using the PBCG-RC (calibration slope: 1.13 [95% CI 1.03-1.23]), but the RPCRC underestimated the risk of csPCa (calibration slope: 0.73 [0.68-0.79]). The PBCG-RC showed a net benefit in a decision curve analysis, whereas the RPCRC showed no net benefit at clinically relevant risk threshold levels. Recalibration improved clinical benefit, and differences between RCs decreased.ConclusionsAssessment of calibration is essential to ensure the clinical value of risk prediction tools. The PBCG-RC provided clinical benefit in its current version online. On the contrary, the RPCRC cannot be recommended in this setting.Patient summaryPredicting the probability of finding prostate cancer on biopsy differed between two assessed risk calculators. After recalibration, the agreement of the models improved, and both were shown to be clinically useful.

Project description:BackgroundYoung people with psychosis are at high risk of developing cardiometabolic disorders; however, there is no suitable cardiometabolic risk prediction algorithm for this group. We aimed to develop and externally validate a cardiometabolic risk prediction algorithm for young people with psychosis.MethodsWe developed the Psychosis Metabolic Risk Calculator (PsyMetRiC) to predict up to 6-year risk of incident metabolic syndrome in young people (aged 16-35 years) with psychosis from commonly recorded information at baseline. We developed two PsyMetRiC versions using the forced entry method: a full model (including age, sex, ethnicity, body-mass index, smoking status, prescription of a metabolically active antipsychotic medication, HDL concentration, and triglyceride concentration) and a partial model excluding biochemical results. PsyMetRiC was developed using data from two UK psychosis early intervention services (Jan 1, 2013, to Nov 4, 2020) and externally validated in another UK early intervention service (Jan 1, 2012, to June 3, 2020). A sensitivity analysis was done in UK birth cohort participants (aged 18 years) who were at risk of developing psychosis. Algorithm performance was assessed primarily via discrimination (C statistic) and calibration (calibration plots). We did a decision curve analysis and produced an online data-visualisation app.Findings651 patients were included in the development samples, 510 in the validation sample, and 505 in the sensitivity analysis sample. PsyMetRiC performed well at internal (full model: C 0·80, 95% CI 0·74-0·86; partial model: 0·79, 0·73-0·84) and external validation (full model: 0·75, 0·69-0·80; and partial model: 0·74, 0·67-0·79). Calibration of the full model was good, but there was evidence of slight miscalibration of the partial model. At a cutoff score of 0·18, in the full model PsyMetRiC improved net benefit by 7·95% (sensitivity 75%, 95% CI 66-82; specificity 74%, 71-78), equivalent to detecting an additional 47% of metabolic syndrome cases.InterpretationWe have developed an age-appropriate algorithm to predict the risk of incident metabolic syndrome, a precursor of cardiometabolic morbidity and mortality, in young people with psychosis. PsyMetRiC has the potential to become a valuable resource for early intervention service clinicians and could enable personalised, informed health-care decisions regarding choice of antipsychotic medication and lifestyle interventions.FundingNational Institute for Health Research and Wellcome Trust.

Project description:BackgroundBefore considering whether to use a multivariable (diagnostic or prognostic) prediction model, it is essential that its performance be evaluated in data that were not used to develop the model (referred to as external validation). We critically appraised the methodological conduct and reporting of external validation studies of multivariable prediction models.MethodsWe conducted a systematic review of articles describing some form of external validation of one or more multivariable prediction models indexed in PubMed core clinical journals published in 2010. Study data were extracted in duplicate on design, sample size, handling of missing data, reference to the original study developing the prediction models and predictive performance measures.Results11,826 articles were identified and 78 were included for full review, which described the evaluation of 120 prediction models. in participant data that were not used to develop the model. Thirty-three articles described both the development of a prediction model and an evaluation of its performance on a separate dataset, and 45 articles described only the evaluation of an existing published prediction model on another dataset. Fifty-seven percent of the prediction models were presented and evaluated as simplified scoring systems. Sixteen percent of articles failed to report the number of outcome events in the validation datasets. Fifty-four percent of studies made no explicit mention of missing data. Sixty-seven percent did not report evaluating model calibration whilst most studies evaluated model discrimination. It was often unclear whether the reported performance measures were for the full regression model or for the simplified models.ConclusionsThe vast majority of studies describing some form of external validation of a multivariable prediction model were poorly reported with key details frequently not presented. The validation studies were characterised by poor design, inappropriate handling and acknowledgement of missing data and one of the most key performance measures of prediction models i.e. calibration often omitted from the publication. It may therefore not be surprising that an overwhelming majority of developed prediction models are not used in practice, when there is a dearth of well-conducted and clearly reported (external validation) studies describing their performance on independent participant data.