Developing risk prediction models for kidney injury and assessing incremental value for novel biomarkers.
Ontology highlight
ABSTRACT:
SUBMITTER: Kerr KF
PROVIDER: S-EPMC4123400 | biostudies-literature | 2014 Aug
REPOSITORIES: biostudies-literature
Publications
Developing risk prediction models for kidney injury and assessing incremental value for novel biomarkers.
Kerr Kathleen F KF Meisner Allison A Thiessen-Philbrook Heather H Coca Steven G SG Parikh Chirag R CR
Clinical journal of the American Society of Nephrology : CJASN 20140522 8
The field of nephrology is actively involved in developing biomarkers and improving models for predicting patients' risks of AKI and CKD and their outcomes. However, some important aspects of evaluating biomarkers and risk models are not widely appreciated, and statistical methods are still evolving. This review describes some of the most important statistical concepts for this area of research and identifies common pitfalls. Particular attention is paid to metrics proposed within the last 5 yea ...[more]
Similar Datasets
Project description:The discrimination of a risk prediction model measures that model's ability to distinguish between subjects with and without events. The area under the receiver operating characteristic curve (AUC) is a popular measure of discrimination. However, the AUC has recently been criticized for its insensitivity in model comparisons in which the baseline model has performed well. Thus, 2 other measures have been proposed to capture improvement in discrimination for nested models: the integrated discrimination improvement and the continuous net reclassification improvement. In the present study, the authors use mathematical relations and numerical simulations to quantify the improvement in discrimination offered by candidate markers of different strengths as measured by their effect sizes. They demonstrate that the increase in the AUC depends on the strength of the baseline model, which is true to a lesser degree for the integrated discrimination improvement. On the other hand, the continuous net reclassification improvement depends only on the effect size of the candidate variable and its correlation with other predictors. These measures are illustrated using the Framingham model for incident atrial fibrillation. The authors conclude that the increase in the AUC, integrated discrimination improvement, and net reclassification improvement offer complementary information and thus recommend reporting all 3 alongside measures characterizing the performance of the final model.
| S-EPMC3530349 | biostudies-literature
Project description:In early detection of disease, a single biomarker often has inadequate classification performance, making it important to identify new biomarkers to combine with the existing marker for improved performance. A biologically natural method for combining biomarkers is to use logic rules, e.g., the OR/AND rules. In our motivating example of early detection of pancreatic cancer, the established biomarker CA19-9 is only present in a subclass of cancers; it is of interest to identify new biomarkers present in the other subclasses and declare disease when either marker is positive. While there has been research on developing biomarker combinations using the OR/AND rules, inference regarding the incremental value of the new marker within this framework is lacking and challenging due to statistical non-regularity. In this article, we aim to answer the inferential question of whether combining the new biomarker achieves better classification performance than using the existing biomarker alone, based on a nonparametrically estimated OR rule that maximizes the weighted average of sensitivity and specificity. We propose and compare various procedures for testing the incremental value of the new biomarker and constructing its confidence interval, using bootstrap, cross-validation, and a novel fuzzy p-value-based technique. We compare the performance of different methods via extensive simulation studies and apply them to the pancreatic cancer example.
| S-EPMC7307975 | biostudies-literature
Project description:Accurate risk prediction models are needed to identify different risk groups for individualized prevention and treatment strategies. In the Nurses' Health Study, to examine the effects of several biomarkers and genetic markers on the risk of rheumatoid arthritis (RA), a three-phase nested case-control (NCC) design was conducted, in which two sequential NCC subcohorts were formed with one nested within the other, and one set of new markers measured on each of the subcohorts. One objective of the study is to evaluate clinical values of novel biomarkers in improving upon existing risk models because of potential cost associated with assaying biomarkers. In this paper, we develop robust statistical procedures for constructing risk prediction models for RA and estimating the incremental value (IncV) of new markers based on three-phase NCC studies. Our method also takes into account possible time-varying effects of biomarkers in risk modeling, which allows us to more robustly assess the biomarker utility and address the question of whether a marker is better suited for short-term or long-term risk prediction. The proposed procedures are shown to perform well in finite samples via simulation studies.
| S-EPMC4819437 | biostudies-literature
Project description:In many clinical applications, understanding when measurement of new markers is necessary to provide added accuracy to existing prediction tools could lead to more cost effective disease management. Many statistical tools for evaluating the incremental value (IncV) of the novel markers over the routine clinical risk factors have been developed in recent years. However, most existing literature focuses primarily on global assessment. Since the IncVs of new markers often vary across subgroups, it would be of great interest to identify subgroups for which the new markers are most/least useful in improving risk prediction. In this paper we provide novel statistical procedures for systematically identifying potential traditional-marker based subgroups in whom it might be beneficial to apply a new model with measurements of both the novel and traditional markers. We consider various conditional time-dependent accuracy parameters for censored failure time outcome to assess the subgroup-specific IncVs. We provide non-parametric kernel-based estimation procedures to calculate the proposed parameters. Simultaneous interval estimation procedures are provided to account for sampling variation and adjust for multiple testing. Simulation studies suggest that our proposed procedures work well in finite samples. The proposed procedures are applied to the Framingham Offspring Study to examine the added value of an inflammation marker, C-reactive protein, on top of the traditional Framingham risk score for predicting 10-year risk of cardiovascular disease.
| S-EPMC3633735 | biostudies-literature
Project description:Risk prediction is a critical step in patient selection for aortic valve replacement (AVR), yet existing risk scores incorporate very few echocardiographic parameters. We sought to evaluate the incremental predictive value of a complete echocardiogram to identify high-risk surgical candidates before AVR.A cohort of patients with severe aortic stenosis undergoing surgical AVR with or without coronary bypass was assembled at 2 tertiary centers. Preoperative echocardiograms were reviewed by independent observers to quantify chamber size/function and valve function. Patient databases were queried to extract clinical data. The cohort consisted of 432 patients with a mean age of 73.5 years and 38.7% females. Multivariable logistic regression revealed 3 echocardiographic predictors of in-hospital mortality or major morbidity: E/e' ratio reflective of elevated left ventricular (LV) filling pressure; myocardial performance index reflective of right ventricular (RV) dysfunction; and small LV end-diastolic cavity size. Addition of these echocardiographic parameters to the STS risk score led to an integrated discrimination improvement of 4.1% (P<0.0001). After a median follow-up of 2 years, Cox regression revealed 5 echocardiographic predictors of all-cause mortality: small LV end-diastolic cavity size; LV mass index; mitral regurgitation grade; right atrial area index; and mean aortic gradient <40 mm Hg.Echocardiographic measures of LV diastolic dysfunction and RV performance add incremental value to the STS risk score and should be integrated in prediction when evaluating the risk of AVR. In addition, findings of small hypertrophied LV cavities and/or low mean aortic gradients confer a higher risk of 2-year mortality.
| S-EPMC4845123 | biostudies-other
Project description:Rationale & objectiveRisk factors for acute kidney injury (AKI) in the hospital have been well studied. Yet, risk factors for identifying high-risk patients for AKI occurring and managed in the outpatient setting are unknown and may differ.Study designPredictive model development and external validation using observational electronic health record data.Setting & participantsPatients aged 18-90 years with recurrent primary care encounters, known baseline serum creatinine, and creatinine measured during an 18-month outcome period without established advanced kidney disease.New predictors & established predictorsEstablished predictors for inpatient AKI were considered. Potential new predictors were hospitalization history, smoking, serum potassium levels, and prior outpatient AKI.OutcomesA ≥50% increase in the creatinine level above a moving baseline of the recent measurement(s) without a hospital admission within 7 days defined outpatient AKI.Analytical approachLogistic regression with bootstrap sampling for backward stepwise covariate elimination was used. The model was then transformed into 2 binary tests: one identifying high-risk patients for research and another identifying patients for additional clinical monitoring or intervention.ResultsOutpatient AKI was observed in 4,611 (3.0%) and 115,744 (2.4%) patients in the development and validation cohorts, respectively. The model, with 18 variables and 3 interaction terms, produced C statistics of 0.717 (95% CI, 0.710-0.725) and 0.722 (95% CI, 0.720-0.723) in the development and validation cohorts, respectively. The research test, identifying the 5.2% most at-risk patients in the validation cohort, had a sensitivity of 0.210 (95% CI, 0.208-0.213) and specificity of 0.952 (95% CI, 0.951-0.952). The clinical test, identifying the 20% most at-risk patients, had a sensitivity of 0.494 (95% CI, 0.491-0.497) and specificity of 0.806 (95% CI, 0.806-0.807).LimitationsOnly surviving patients with measured creatinine levels during a baseline period and outcome period were included.ConclusionsThe outpatient AKI risk prediction model performed well in both the development and validation cohorts in both continuous and binary forms.
| S-EPMC8767133 | biostudies-literature
Project description:Rationale & objectiveThere is limited evidence to guide follow-up after acute kidney injury (AKI). Knowledge gaps include which patients to prioritize, at what time point, and for mitigation of which outcomes. In this study, we sought to compare the net benefit of risk model-based clinical decisions following AKI.Study designExternal validation of 2 risk models of AKI outcomes: the Grampian -Aberdeen (United Kingdom) AKI readmissions model and the Alberta (Canada) kidney disease risk model of chronic kidney disease (CKD) glomerular (G) filtration rate categories 4 and 5 (CKD G4 and G5). Process mining to delineate existing care pathways.Setting & participantsValidation was based on data from adult hospital survivors of AKI from Grampian, 2011-2013.PredictorsKDIGO-based measures of AKI severity and comorbidities specified in the original models.OutcomesDeath or readmission within 90 days for all hospital survivors. Progression to new CKD G4-G5 for patients surviving at least 90 days after AKI.Analytical approachDecision curve analysis to assess the "net benefit" of use of risk models to guide clinical care compared to alternative approaches (eg, prioritizing all AKI, severe AKI, or only those without kidney recovery).Results26,575 of 105,461 hospital survivors in Grampian (mean age, 60.9 ± 19.8 [SD] years) were included for validation of the death or readmission model, and 9,382 patients (mean age, 60.9 ± 19.8 years) for the CKD G4-G5 model. Both models discriminated well (area under the curve [AUC], 0.77 and 0.86, respectively). Decision curve analysis showed greater net benefit for follow up of all AKI than only severe AKI in most cases. Both original and refitted models provided net benefit superior to any other decision strategy. In process mining of all hospital discharges, 41% of readmissions and deaths occurred among people recovering after AKI. 1,464 of 3,776 people (39%) readmitted after AKI had received no intervening monitoring.LimitationsBoth original models overstated risks, indicating a need for regular updating.ConclusionsFollow up after AKI has potential net benefit for preempting readmissions, death, and subsequent CKD progression. Decisions could be improved by using risk models and by focusing on AKI across a full spectrum of severity. The current lack of monitoring among many with poor outcomes indicates possible opportunities for implementation of decision support.
| S-EPMC8234511 | biostudies-literature
Project description:BackgroundThe incidence of Acute Kidney Injury (AKI) and its human and economic cost is increasing steadily. One way to reduce the burden associated with AKI is to prevent the event altogether. An important step in prevention lies in AKI risk prediction. Due to the increasing number of available risk prediction models (RPMs) clinicians need to be able to rely on systematic reviews (SRs) to provide an objective assessment on which RPM can be used in a specific setting. Our aim was to assess the quality of SRs of RPMs in AKI.MethodsThe protocol for this overview was registered in PROSPERO. MEDLINE and Embase were searched for SRs of RPMs of AKI in any setting from 2003 till August 2020. We used the ROBIS tool to assess the methodological quality of the retrieved SRs.ResultsEight SRs were retrieved. All studies were assessed as being at high risk for bias using the ROBIS tool. Eight reviews had a high risk of bias in study eligibility criteria (domain 1), five for study identification and selection (domain 2), seven for data collection and appraisal (domain 3) and seven for synthesis and findings (domain 4). Five reviews were scored at high risk of bias across all four domains. Risk of bias assessment with a formal risk of bias tool was only performed in five reviews. Primary studies were heterogeneous and used a wide range of AKI definitions. Only 19 unique RPM were externally validated, of which 11 had only 1 external validation report.ConclusionThe methodological quality of SRs of RPMs of AKI is inconsistent. Most SRs lack a formal risk of bias assessment. SRs ought to adhere to certain standard quality criteria so that clinicians can rely on them to select a RPM for use in an individual patient.Trial registrationPROSPERO registration number is CRD 42020204236, available at https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=204236.
| S-EPMC8016311 | biostudies-literature
Project description:Acute kidney injury (AKI) is a major contributor to in-hospital morbidity and mortality. Vancomycin, one of the most commonly used antibiotics in a clinical setting, is associated with AKI, with its incidence ranging up to 43%. Despite the high demand, few studies have investigated serum biomarkers to detect vancomycin-induced kidney injury (VIKI). Here, we evaluated the diagnostic value of nine candidate serum biomarkers for VIKI. A total of 23,182 cases referred for vancomycin concentration measurement from January 2018 to December 2019 were screened and 28 subjects with confirmed VIKI were enrolled (VIKI group). Age- and sex- matched control group consisted of 21 subjects who underwent vancomycin therapy without developing VIKI (non-VIKI group), and 23 healthy controls (HC group). The serum concentrations of clusterin, retinol binding protein 4 (RBP4), interleukin-18 (IL-18), tumor necrosis factor receptor 1 (TNF-R1), C-X-C motif chemokine ligand 10 (CXCL10), neutrophil gelatinase-associated lipocalin (NGAL), osteopontin, trefoil factor-3 (TFF3), and cystatin C were compared among the three groups, and their correlations with estimated glomerular filtration rate (eGFR) and diagnostic values for VIKI were assessed. All of the biomarkers except clusterin and RBP4 exhibited significant elevation in the VIKI group. Serum TFF3, cystatin C, TNF-R1, and osteopontin demonstrated an excellent diagnostic value for VIKI (TFF3, area under the curve (AUC) 0.932; cystatin C, AUC 0.917; TNF-R1, AUC 0.866; osteopontin, AUC 0.787); and except osteopontin, a strong negative correlation with eGFR (TFF3, r = -0.71; cystatin C, r = -0.70; TNF-R1, r = -0.60). IL-18, CXCL10, and NGAL showed weak correlation with eGFR and moderate diagnostic value for VIKI. This study tested multiple serum biomarkers for VIKI and showed that serum TFF3, cystatin C, TNF-R1, and osteopontin could efficiently discriminate VIKI patients. Further studies are warranted to clarify the diagnostic value of these biomarkers in VIKI.
| S-EPMC8584616 | biostudies-literature
Project description:ObjectivesTo externally validate the extraprostatic extension (EPE) grade criteria on MRI and analyze the incremental value of EPE grade to clinical models of prostate cancer.MethodsA consecutive 130 patients who underwent preoperative prostate MRI followed by radical prostatectomy between January 2015 to January 2020 in our institution were retrospectively enrolled. The EPE grade, Cancer of the Prostate Risk Assessment (CAPRA), and Memorial Sloan Kettering Cancer Center nomogram (MSKCCn) score for each patient were assigned. Significant clinicopathological factors in univariate and multivariate analyses were combined with EPE grade to build the Clinical + EPE grade model, and the CAPRA and MSKCCn score were also combined with EPE grade to build the CAPRA + EPE grade and MSKCCn + EPE grade model, respectively. The area under the curve (AUC), sensitivity and specificity of these models were calculated to evaluate their diagnostic performance. Calibration and decision curve analyses were used to analyze their calibration performance and clinical utility.ResultsThe AUC for predicting EPE was 0.767-0.778 for EPE grade, 0.704 for CAPRA, and 0.723 for MSKCCn. After combination with EPE grade, the AUCs of these clinical models increased significantly than using clinical models along (P < 0.05), but was comparable with using EPE grade alone (P > 0.05). The calibration curves of EPE grade, clinical models and combined models showed that these models are well-calibrated for EPE. In the decision curve analysis, EPE grade showed slightly higher net benefit than MSKCCn and CAPRA.ConclusionThe EPE grade showed good performance for evaluating EPE in our cohort and possessed well clinical utility. Further combinations with the EPE grade could improve the diagnostic performance of clinical models.
| S-EPMC8047629 | biostudies-literature