Dataset Information

Risk Prediction Models for Melanoma: A Systematic Review on the Heterogeneity in Model Development and Validation.

ABSTRACT: The rising incidence of cutaneous melanoma over the past few decades has prompted substantial efforts to develop risk prediction models identifying people at high risk of developing melanoma to facilitate targeted screening programs. We review these models, regarding study characteristics, differences in risk factor selection and assessment, evaluation, and validation methods. Our systematic literature search revealed 40 studies comprising 46 different risk prediction models eligible for the review. Altogether, 35 different risk factors were part of the models with nevi being the most common one (n = 35, 78%); little consistency in other risk factors was observed. Results of an internal validation were reported for less than half of the studies (n = 18, 45%), and only 6 performed external validation. In terms of model performance, 29 studies assessed the discriminative ability of their models; other performance measures, e.g., regarding calibration or clinical usefulness, were rarely reported. Due to the substantial heterogeneity in risk factor selection and assessment as well as methodologic aspects of model development, direct comparisons between models are hardly possible. Uniform methodologic standards for the development and validation of risk prediction models for melanoma and reporting standards for the accompanying publications are necessary and need to be obligatory for that reason.

SUBMITTER: Kaiser I

PROVIDER: S-EPMC7662952 | biostudies-literature | 2020 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Risk Prediction Models for Melanoma: A Systematic Review on the Heterogeneity in Model Development and Validation.

Kaiser Isabelle I Pfahlberg Annette B AB Uter Wolfgang W Heppt Markus V MV Veierød Marit B MB Gefeller Olaf O

International journal of environmental research and public health 20201028 21

The rising incidence of cutaneous melanoma over the past few decades has prompted substantial efforts to develop risk prediction models identifying people at high risk of developing melanoma to facilitate targeted screening programs. We review these models, regarding study characteristics, differences in risk factor selection and assessment, evaluation, and validation methods. Our systematic literature search revealed 40 studies comprising 46 different risk prediction models eligible for the rev ...[more]

PMID: 33126677

Similar Datasets

Project description:OBJECTIVE:To provide focused evaluation of predictive modeling of electronic medical record (EMR) data to predict 30 day hospital readmission. DESIGN:Systematic review. DATA SOURCE:Ovid Medline, Ovid Embase, CINAHL, Web of Science, and Scopus from January 2015 to January 2019. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:All studies of predictive models for 28 day or 30 day hospital readmission that used EMR data. OUTCOME MEASURES:Characteristics of included studies, methods of prediction, predictive features, and performance of predictive models. RESULTS:Of 4442 citations reviewed, 41 studies met the inclusion criteria. Seventeen models predicted risk of readmission for all patients and 24 developed predictions for patient specific populations, with 13 of those being developed for patients with heart conditions. Except for two studies from the UK and Israel, all were from the US. The total sample size for each model ranged between 349 and 1?195?640. Twenty five models used a split sample validation technique. Seventeen of 41 studies reported C statistics of 0.75 or greater. Fifteen models used calibration techniques to further refine the model. Using EMR data enabled final predictive models to use a wide variety of clinical measures such as laboratory results and vital signs; however, use of socioeconomic features or functional status was rare. Using natural language processing, three models were able to extract relevant psychosocial features, which substantially improved their predictions. Twenty six studies used logistic or Cox regression models, and the rest used machine learning methods. No statistically significant difference (difference 0.03, 95% confidence interval -0.0 to 0.07) was found between average C statistics of models developed using regression methods (0.71, 0.68 to 0.73) and machine learning (0.74, 0.71 to 0.77). CONCLUSIONS:On average, prediction models using EMR data have better predictive performance than those using administrative data. However, this improvement remains modest. Most of the studies examined lacked inclusion of socioeconomic features, failed to calibrate the models, neglected to conduct rigorous diagnostic testing, and did not discuss clinical impact.

Project description:BackgroundHospitals are subject to federal financial penalties for excessive 30-day hospital readmissions for acute myocardial infarction (AMI). Prospectively identifying patients hospitalized with AMI at high risk for readmission could help prevent 30-day readmissions by enabling targeted interventions. However, the performance of AMI-specific readmission risk prediction models is unknown.Methods and resultsWe systematically searched the published literature through March 2017 for studies of risk prediction models for 30-day hospital readmission among adults with AMI. We identified 11 studies of 18 unique risk prediction models across diverse settings primarily in the United States, of which 16 models were specific to AMI. The median overall observed all-cause 30-day readmission rate across studies was 16.3% (range, 10.6%-21.0%). Six models were based on administrative data; 4 on electronic health record data; 3 on clinical hospital data; and 5 on cardiac registry data. Models included 7 to 37 predictors, of which demographics, comorbidities, and utilization metrics were the most frequently included domains. Most models, including the Centers for Medicare and Medicaid Services AMI administrative model, had modest discrimination (median C statistic, 0.65; range, 0.53-0.79). Of the 16 reported AMI-specific models, only 8 models were assessed in a validation cohort, limiting generalizability. Observed risk-stratified readmission rates ranged from 3.0% among the lowest-risk individuals to 43.0% among the highest-risk individuals, suggesting good risk stratification across all models.ConclusionsCurrent AMI-specific readmission risk prediction models have modest predictive ability and uncertain generalizability given methodological limitations. No existing models provide actionable information in real time to enable early identification and risk-stratification of patients with AMI before hospital discharge, a functionality needed to optimize the potential effectiveness of readmission reduction interventions.

Project description:BackgroundMelanoma risk prediction models could be useful for matching preventive interventions to patients' risk.ObjectivesTo develop and validate a model for incident first-primary cutaneous melanoma using clinically assessed risk factors.MethodsWe used unconditional logistic regression with backward selection from the Australian Melanoma Family Study (461 cases and 329 controls) in which age, sex and city of recruitment were kept in each step, and we externally validated it using the Leeds Melanoma Case-Control Study (960 cases and 513 controls). Candidate predictors included clinically assessed whole-body naevi and solar lentigines, and self-assessed pigmentation phenotype, sun exposure, family history and history of keratinocyte cancer. We evaluated the predictive strength and discrimination of the model risk factors using odds per age- and sex-adjusted SD (OPERA) and the area under curve (AUC), and calibration using the Hosmer-Lemeshow test.ResultsThe final model included the number of naevi ≥ 2 mm in diameter on the whole body, solar lentigines on the upper back (a six-level scale), hair colour at age 18 years and personal history of keratinocyte cancer. Naevi was the strongest risk factor; the OPERA was 3·51 [95% confidence interval (CI) 2·71-4·54] in the Australian study and 2·56 (95% CI 2·23-2·95) in the Leeds study. The AUC was 0·79 (95% CI 0·76-0·83) in the Australian study and 0·73 (95% CI 0·70-0·75) in the Leeds study. The Hosmer-Lemeshow test P-value was 0·30 in the Australian study and < 0·001 in the Leeds study.ConclusionsThis model had good discrimination and could be used by clinicians to stratify patients by melanoma risk for the targeting of preventive interventions. What's already known about this topic? Melanoma risk prediction models may be useful in prevention by tailoring interventions to personalized risk levels. For reasons of feasibility, time and cost many melanoma prediction models use self-assessed risk factors. However, individuals tend to underestimate their naevus numbers. What does this study add? We present a melanoma risk prediction model, which includes clinically-assessed whole-body naevi and solar lentigines, and self-assessed risk factors including pigmentation phenotype and history of keratinocyte cancer. This model performs well on discrimination, the model's ability to distinguish between individuals with and without melanoma, and may assist clinicians to stratify patients by melanoma risk for targeted preventive interventions.

Project description:ObjectivesTo look at the available literature on validated prediction models for contrast induced nephropathy and describe their characteristics.DesignSystematic review.Data sourcesMedline, Embase, and CINAHL (cumulative index to nursing and allied health literature) databases.Review methodsDatabases searched from inception to 2015, and the retrieved reference lists hand searched. Dual reviews were conducted to identify studies published in the English language of prediction models tested with patients that included derivation and validation cohorts. Data were extracted on baseline patient characteristics, procedural characteristics, modelling methods, metrics of model performance, risk of bias, and clinical usefulness. Eligible studies evaluated characteristics of predictive models that identified patients at risk of contrast induced nephropathy among adults undergoing a diagnostic or interventional procedure using conventional radiocontrast media (media used for computed tomography or angiography, and not gadolinium based contrast).Results16 studies were identified, describing 12 prediction models. Substantial interstudy heterogeneity was identified, as a result of different clinical settings, cointerventions, and the timing of creatinine measurement to define contrast induced nephropathy. Ten models were validated internally and six were validated externally. Discrimination varied in studies that were validated internally (C statistic 0.61-0.95) and externally (0.57-0.86). Only one study presented reclassification indices. The majority of higher performing models included measures of pre-existing chronic kidney disease, age, diabetes, heart failure or impaired ejection fraction, and hypotension or shock. No prediction model evaluated its effect on clinical decision making or patient outcomes.ConclusionsMost predictive models for contrast induced nephropathy in clinical use have modest ability, and are only relevant to patients receiving contrast for coronary angiography. Further research is needed to develop models that can better inform patient centred decision making, as well as improve the use of prevention strategies for contrast induced nephropathy.

Project description:ContextEarly detection of kidney cancer improves survival; however, low prevalence means that population-wide screening may be inefficient. Stratification of the population into risk categories could allow for the introduction of a screening programme tailored to individuals.ObjectiveThis review will identify and compare published models that predict the risk of developing kidney cancer in the general population.Evidence acquisitionA search identified primary research reporting or validating models predicting the risk of kidney cancer in Medline and EMBASE. After screening identified studies for inclusion, we extracted data onto a standardised form. The risk models were classified using the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines and evaluated using the PROBAST assessment tool.Evidence synthesisThe search identified 15 281 articles. Sixty-two satisfied the inclusion criteria; performance measures were provided for 11 models. Some models predicted the risk of prevalent undiagnosed disease and others future incident disease. Six of the models had been validated, two using external populations. The most commonly included risk factors were age, smoking status, and body mass index. Most of the models had acceptable-to-good discrimination (area under the receiver-operating curve >0.7) in development and validation. Many models also had high specificity; however, several had low sensitivity. The highest performance was seen for the models using only biomarkers to detect kidney cancer; however, these were developed and validated in small case-control studies.ConclusionsWe identified a small number of risk models that could be used to stratify the population according to the risk of kidney cancer. Most exhibit reasonable discrimination, but a few have been validated externally in population-based studies.Patient summaryIn this review, we looked at mathematical models predicting the likelihood of an individual developing kidney cancer. We found several suitable models, using a range of risk factors (such as age and smoking) to predict the risk for individuals. Most of the models identified require further testing in the general population to confirm their usefulness.

Project description:Because existing risk prediction models for lung cancer were developed in white populations, they may not be appropriate for predicting risk among African-Americans. Therefore, a need exists to construct and validate a risk prediction model for lung cancer that is specific to African-Americans. We analyzed data from 491 African-Americans with lung cancer and 497 matched African-American controls to identify specific risks and incorporate them into a multivariable risk model for lung cancer and estimate the 5-year absolute risk of lung cancer. We performed internal and external validations of the risk model using data on additional cases and controls from the same ongoing multiracial/ethnic lung cancer case-control study from which the model-building data were obtained as well as data from two different lung cancer studies in metropolitan Detroit, respectively. We also compared our African-American model with our previously developed risk prediction model for whites. The final risk model included smoking-related variables [smoking status, pack-years smoked, age at smoking cessation (former smokers), and number of years since smoking cessation (former smokers)], self-reported physician diagnoses of chronic obstructive pulmonary disease or hay fever, and exposures to asbestos or wood dusts. Our risk prediction model for African-Americans exhibited good discrimination [75% (95% confidence interval, 0.67-0.82)] for our internal data and moderate discrimination [63% (95% confidence interval, 0.57-0.69)] for the external data group, which is an improvement over the Spitz model for white subjects. Existing lung cancer prediction models may not be appropriate for predicting risk for African-Americans because (a) they were developed using white populations, (b) level of risk is different for risk factors that African-American share with whites, and (c) unique group-specific risk factors exist for African-Americans. This study developed and validated a risk prediction model for lung cancer that is specific to African-Americans and thus more precise in predicting their risks. These findings highlight the importance of conducting further ethnic-specific analyses of disease risk.

Dataset Information

Risk Prediction Models for Melanoma: A Systematic Review on the Heterogeneity in Model Development and Validation.

Publications

Risk Prediction Models for Melanoma: A Systematic Review on the Heterogeneity in Model Development and Validation.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets