Dataset Information

Validation of prediction models based on lasso regression with multiply imputed data.

ABSTRACT: In prognostic studies, the lasso technique is attractive since it improves the quality of predictions by shrinking regression coefficients, compared to predictions based on a model fitted via unpenalized maximum likelihood. Since some coefficients are set to zero, parsimony is achieved as well. It is unclear whether the performance of a model fitted using the lasso still shows some optimism. Bootstrap methods have been advocated to quantify optimism and generalize model performance to new subjects. It is unclear how resampling should be performed in the presence of multiply imputed data.The data were based on a cohort of Chronic Obstructive Pulmonary Disease patients. We constructed models to predict Chronic Respiratory Questionnaire dyspnea 6 months ahead. Optimism of the lasso model was investigated by comparing 4 approaches of handling multiply imputed data in the bootstrap procedure, using the study data and simulated data sets. In the first 3 approaches, data sets that had been completed via multiple imputation (MI) were resampled, while the fourth approach resampled the incomplete data set and then performed MI.The discriminative model performance of the lasso was optimistic. There was suboptimal calibration due to over-shrinkage. The estimate of optimism was sensitive to the choice of handling imputed data in the bootstrap resampling procedure. Resampling the completed data sets underestimates optimism, especially if, within a bootstrap step, selected individuals differ over the imputed data sets. Incorporating the MI procedure in the validation yields estimates of optimism that are closer to the true value, albeit slightly too larger.Performance of prognostic models constructed using the lasso technique can be optimistic as well. Results of the internal validation are sensitive to how bootstrap resampling is performed.

SUBMITTER: Musoro JZ

PROVIDER: S-EPMC4209042 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validation of prediction models based on lasso regression with multiply imputed data.

Musoro Jammbe Z JZ Zwinderman Aeilko H AH Puhan Milo A MA ter Riet Gerben G Geskus Ronald B RB

BMC medical research methodology 20141016

<h4>Background</h4>In prognostic studies, the lasso technique is attractive since it improves the quality of predictions by shrinking regression coefficients, compared to predictions based on a model fitted via unpenalized maximum likelihood. Since some coefficients are set to zero, parsimony is achieved as well. It is unclear whether the performance of a model fitted using the lasso still shows some optimism. Bootstrap methods have been advocated to quantify optimism and generalize model perfor ...[more]

PMID: 25323009

Similar Datasets

Project description:BackgroundAssessing disparities in injury is crucial for injury prevention and for evaluating injury prevention strategies, but efforts have been hampered by missing data. This study aimed to show the utility and reliability of the injury surveillance system as a trustworthy resource for examining disparities by generating multiple imputed companion datasets.MethodsWe employed data from the National Electronic Injury Surveillance System-All Injury Program (NEISS-AIP) for the period 2014-2018. A comprehensive simulation study was conducted to identify the appropriate strategy for addressing missing data limitations in NEISS-AIP. To evaluate the imputation performance more quantitatively, a new method based on Brier Skill Score (BSS) was developed to assess the accuracy of predictions by different approaches. We selected the multiple imputations by fully conditional specification (FCS MI) to generate the imputed companion data to NEISS-AIP 2014-2018. We further assessed health disparities systematically in nonfatal assault injuries treated in U.S. hospital emergency departments (EDs) by race and ethnicity, location of injury and sex.ResultsWe found for the first time that significantly higher age-adjusted nonfatal assault injury rates for ED visits per 100,000 population occurred among non-Hispanic Black persons (1306.8, 95% Confidence Interval [CI]: 660.1 - 1953.5), in public settings (286.3, 95% CI: 183.2 - 389.4) and for males (603.5, 95% CI: 409.4 - 797.5). We also observed similar trends in age-adjusted rates (AARs) by different subgroups for non-Hispanic Black persons, injuries occurring in public settings, and for males: AARs of nonfatal assault injury increased significantly from 2014 through 2017, then declined significantly in 2018.ConclusionsNonfatal assault injury imposes significant health care costs and productivity losses for millions of people each year. This study is the first to specifically look at health disparities in nonfatal assault injuries using multiply imputed companion data. Understanding how disparities differ by various groups may lead to the development of more effective initiatives to prevent such injury.

Dataset Information

Validation of prediction models based on lasso regression with multiply imputed data.

Publications

Validation of prediction models based on lasso regression with multiply imputed data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets