Dataset Information

A Novel Method for Identifying a Parsimonious and Accurate Predictive Model for Multiple Clinical Outcomes.

ABSTRACT:

Background and objective

Most methods for developing clinical prognostic models focus on identifying parsimonious and accurate models to predict a single outcome; however, patients and providers often want to predict multiple outcomes simultaneously. As an example, for older adults one is often interested in predicting nursing home admission as well as mortality. We propose and evaluate a novel predictor-selection computing method for multiple outcomes and provide the code for its implementation.

Methods

Our proposed algorithm selected the best subset of common predictors based on the minimum average normalized Bayesian Information Criterion (BIC) across outcomes: the Best Average BIC (baBIC) method. We compared the predictive accuracy (Harrell's C-statistic) and parsimony (number of predictors) of the model obtained using the baBIC method with: 1) a subset of common predictors obtained from the union of optimal models for each outcome (Union method), 2) a subset obtained from the intersection of optimal models for each outcome (Intersection method), and 3) a model with no variable selection (Full method). We used a case-study data from the Health and Retirement Study (HRS) to demonstrate our method and conducted a simulation study to investigate performance.

Results

In the case-study data and simulations, the average Harrell's C-statistics across outcomes of the models obtained with the baBIC and Union methods were comparable. Despite the similar discrimination, the baBIC method produced more parsimonious models than the Union method. In contrast, the models selected with the Intersection method were the most parsimonious, but with worst predictive accuracy, and the opposite was true in the Full method. In the simulations, the baBIC method performed well by identifying many of the predictors selected in the baBIC model of the case-study data most of the time and excluding those not selected in the majority of the simulations.

Conclusions

Our method identified a common subset of variables to predict multiple clinical outcomes with superior balance between parsimony and predictive accuracy to current methods.

SUBMITTER: Diaz-Ramirez LG

PROVIDER: S-EPMC8098121 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:The COVID-19 pandemic is impressively challenging the healthcare system. Several prognostic models have been validated but few of them are implemented in daily practice. The objective of the study was to validate a machine-learning risk prediction model using easy-to-obtain parameters to help to identify patients with COVID-19 who are at higher risk of death. The training cohort included all patients admitted to Fondazione Policlinico Gemelli with COVID-19 from March 5, 2020, to November 5, 2020. Afterward, the model was tested on all patients admitted to the same hospital with COVID-19 from November 6, 2020, to February 5, 2021. The primary outcome was in-hospital case-fatality risk. The out-of-sample performance of the model was estimated from the training set in terms of Area under the Receiving Operator Curve (AUROC) and classification matrix statistics by averaging the results of fivefold cross validation repeated 3-times and comparing the results with those obtained on the test set. An explanation analysis of the model, based on the SHapley Additive exPlanations (SHAP), is also presented. To assess the subsequent time evolution, the change in paO2/FiO2 (P/F) at 48 h after the baseline measurement was plotted against its baseline value. Among the 921 patients included in the training cohort, 120 died (13%). Variables selected for the model were age, platelet count, SpO2, blood urea nitrogen (BUN), hemoglobin, C-reactive protein, neutrophil count, and sodium. The results of the fivefold cross-validation repeated 3-times gave AUROC of 0.87, and statistics of the classification matrix to the Youden index as follows: sensitivity 0.840, specificity 0.774, negative predictive value 0.971. Then, the model was tested on a new population (n = 1463) in which the case-fatality rate was 22.6%. The test model showed AUROC 0.818, sensitivity 0.813, specificity 0.650, negative predictive value 0.922. Considering the first quartile of the predicted risk score (low-risk score group), the case-fatality rate was 1.6%, 17.8% in the second and third quartile (high-risk score group) and 53.5% in the fourth quartile (very high-risk score group). The three risk score groups showed good discrimination for the P/F value at admission, and a positive correlation was found for the low-risk class to P/F at 48 h after admission (adjusted R-squared = 0.48). We developed a predictive model of death for people with SARS-CoV-2 infection by including only easy-to-obtain variables (abnormal blood count, BUN, C-reactive protein, sodium and lower SpO2). It demonstrated good accuracy and high power of discrimination. The simplicity of the model makes the risk prediction applicable for patients in the Emergency Department, or during hospitalization. Although it is reasonable to assume that the model is also applicable in not-hospitalized persons, only appropriate studies can assess the accuracy of the model also for persons at home.

Project description:The diagnosis of malignant pleural effusions may be challenging when cytological examination of aspirated pleural fluid is equivocal or noncontributory. The purpose of this study was to identify protein candidate biomarkers differentially expressed in the pleural fluid of patients with mesothelioma, lung adenocarcinoma, lymphoma, and tuberculosis (TB).A multiplex protein biochip comprising 120 biomarkers was used to determine the pleural fluid protein profile of 29 mesotheliomas, 29 lung adenocarcinomas, 12 lymphomas, and 35 tuberculosis. The relative abundance of these predetermined biomarkers among groups served to establish the differential diagnosis of: malignant versus benign (TB) effusions, lung adenocarcinoma versus mesothelioma, and lymphoma versus TB. The selected putative markers were validated using widely available commercial techniques in an independent sample of 102 patients.Significant differences were found in the protein expressions of metalloproteinase-9 (MMP-9), cathepsin-B, C-reactive protein, and chondroitin sulfate between malignant and TB effusions. When integrated into a scoring model, these proteins yielded 85% sensitivity, 100% specificity, and an area under the curve (AUC) of 0.98 for labeling malignancy in the verification sample. For lung adenocarcinoma-mesothelioma discrimination, combining CA19-9, CA15-3, and kallikrein-12 had maximal discriminatory capacity (65% sensitivity, 100% specificity, AUC 0.94); figures which also refer to the validation set. Last, cathepsin-B in isolation was only moderately useful (sensitivity 89%, specificity 62%, AUC 0.75) in separating lymphomatous and TB effusions. However, this last differentiation improved significantly when cathepsin-B was used with respect to the patient's age (sensitivity 72%, specificity 100%, AUC 0.94).In conclusion, panels of 4 (i.e., MMP-9, cathepsin-B, C-reactive protein, chondroitin sulfate), or 3 (i.e., CA19-9, CA15-3, kallikrein-12) different protein biomarkers on pleural fluid samples are highly discriminative for signaling a malignant versus tuberculous effusion, or lung adenocarcinoma versus mesothelioma, respectively. Cathepsin-B could also be helpful in establishing the presence of a lymphomatous effusion versus that of TB, if the patient's age is simultaneously taken into consideration.

Project description:IntroductionAlthough the majority of singleton births after in vitro fertilization (IVF) are uncomplicated, studies have suggested that IVF pregnancies may be independently associated with low birth weight (LBW), preterm birth (PTB), and perinatal mortality. These outcomes complicate multiple gestations as expected, but have also been reported in singletons. A multiple embryo implantation model allows for assessment of the early in utero environment, and therefore, assessment of any maternal constraints on developing fetuses. We question whether adverse perinatal outcomes associated with assisted reproductive techniques (ART) occur as a result of maternal physiologic adaptations.Patients and methodsThis is a retrospective, single center study of ART cycles, specifically intracytoplasmic sperm injection (ICSI) cycles during a 16-year period. For each positive pregnancy test 9-11 days after embryo transfer, an ultrasonogram was performed at 7 weeks of gestation to record the number of implanted fetal poles with cardiac activity. Controlled ovarian stimulation (COS), hCG trigger, oocyte retrieval and sperm injection were performed as per our standard protocols. First trimester implantation sites that resulted in live births were defined as "true" to distinguish them from those that spontaneously reduced called "virtual." Birth outcomes analyzed included birth weight and gestational age at delivery.ResultsA total of 17,415 cycles were analyzed. The average maternal age was 36.9 (±5.0) years. An overall fertilization rate of 73.4% generated approximately 48,708 good quality cleavage-stage embryos. In most patients (92.8%), an average of 3 embryos were transferred. The clinical pregnancy rate was 39.2% (n = 6,281). The overall occurrence of multiple gestations was 38.2% (n = 2,608) consisting of 2,038 twin, 511 triplet, and 59 quadruplet pregnancies. Of these multiple gestations, 18.6% of twin, 54.2% of triplet and 76.3% of quadruplet gestations spontaneously reduced. Failure of the implanted embryo to progress was not related to maternal age. Singleton newborns resulting from multiple implantation sites had lower birth weights (P<0.01) and shorter gestational ages (P<0.01) than those from a single implanted embryo. The number of embryos transferred did not affect the gestational length of singleton newborns. Although the birth weights of singletons from multiple implantation sites (virtual singletons) were lower than true singletons, the birth weight of virtual singletons were comparable to the birth weights of true twin, triplet, and quadruplet live births. Multiple logistic regression revealed that virtual singletons were an independent risk factor for PTB (odds ratio: 4.55, 95% CI 2.23-9.29) and LBW (odds ratio: 3.61, 95% CI 1.78-7.32), even after controlling for the number of oocytes, stimulation protocol type, sperm source, total gonadotropins administered, age, embryo quality, and day of embryo transfer.ConclusionsOur study highlights that embryonic implantation sites during early gestation set the growth profile of each embryo, dictating later growth patterns. Specifically, spontaneous reduction of an embryo after multiple embryo implantations can confer greater perinatal risk in the form of LBW and PTB to the surviving fetus. Our findings suggest that maternal constraints or physiologic adaptations maybe one of the mechanisms mediating adverse perinatal outcomes when multiple embryo implantation occurs.