A numerical strategy to evaluate performance of predictive scores via a copula-based approach.
Ontology highlight
ABSTRACT:
SUBMITTER: Zhang Y
PROVIDER: S-EPMC7478334 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Generating synthetic baseline populations is a fundamental step of agent-based modeling and simulation, which is growing fast in a wide range of socio-economic areas including transportation planning research. Traditionally, in many commercial and non-commercial microsimulation systems, the iterative proportional fitting (IPF) procedure has been used for creating the joint distribution of individuals when combining a reference joint distribution with target marginal distributions. Although IPF is simple, computationally efficient, and rigorously founded, it is unclear whether IPF well preserves the dependence structure of the reference joint table sufficiently when fitting it to target margins. In this paper, a novel method is proposed based on the copula concept in order to provide an alternative approach to the problem that IPF resolves. The dependency characteristic measures were computed and the results from the proposed method and IPF were compared. In most test cases, the proposed method outperformed IPF in preserving the dependence structure of the reference joint distribution.
| S-EPMC4973930 | biostudies-literature
Project description:In this work we are interested in identifying clusters of "positional equivalent" actors, i.e. actors who play a similar role in a system. In particular, we analyze weighted bipartite networks that describes the relationships between actors on one side and features or traits on the other, together with the intensity level to which actors show their features. We develop a methodological approach that takes into account the underlying multivariate dependence among groups of actors. The idea is that positions in a network could be defined on the basis of the similar intensity levels that the actors exhibit in expressing some features, instead of just considering relationships that actors hold with each others. Moreover, we propose a new clustering procedure that exploits the potentiality of copula functions, a mathematical instrument for the modelization of the stochastic dependence structure. Our clustering algorithm can be applied both to binary and real-valued matrices. We validate it with simulations and applications to real-world data.
| S-EPMC4193785 | biostudies-literature
Project description:Humans can learn simple new tasks very quickly. This ability suggests that people can reuse previously learned procedural knowledge when it applies to a new context. We have proposed a modeling approach based on this idea and used it to create a model of the attentional blink (AB). The main idea of the skill-based approach is that models are not created from scratch but, instead, built up from reusable pieces of procedural knowledge (skills). This approach not only provides an explanation for the fast learning of simple tasks but also shows much promise to improve certain aspects of cognitive modeling (e.g., robustness and generalizability). We performed two experiments, in order to collect empirical support for the model's prediction that the AB will disappear when the two targets are consolidated as a single chunk. Firstly, we performed an unsuccessful replication of a study reporting that the AB disappears when participants are instructed to remember the targets as a syllable. However, a subsequent experiment using easily combinable stimuli supported the model's prediction and showed a strongly reduced AB in a large group of participants. This result suggests that it is possible to avoid the AB with the right consolidation strategy. The skill-based approach allowed relating this finding to a general cognitive process, thereby demonstrating that incorporating this approach can be very helpful to generalize the findings of cognitive models, which otherwise tends to be rather difficult.
| S-EPMC8782399 | biostudies-literature
Project description:Background. Propensity score usage seems to be growing in popularity leading researchers to question the possible role of propensity scores in prediction modeling, despite the lack of a theoretical rationale. It is suspected that such requests are due to the lack of differentiation regarding the goals of predictive modeling versus causal inference modeling. Therefore, the purpose of this study is to formally examine the effect of propensity scores on predictive performance. Our hypothesis is that a multivariable regression model that adjusts for all covariates will perform as well as or better than those models utilizing propensity scores with respect to model discrimination and calibration. Methods. The most commonly encountered statistical scenarios for medical prediction (logistic and proportional hazards regression) were used to investigate this research question. Random cross-validation was performed 500 times to correct for optimism. The multivariable regression models adjusting for all covariates were compared with models that included adjustment for or weighting with the propensity scores. The methods were compared based on three predictive performance measures: (1) concordance indices; (2) Brier scores; and (3) calibration curves. Results. Multivariable models adjusting for all covariates had the highest average concordance index, the lowest average Brier score, and the best calibration. Propensity score adjustment and inverse probability weighting models without adjustment for all covariates performed worse than full models and failed to improve predictive performance with full covariate adjustment. Conclusion. Propensity score techniques did not improve prediction performance measures beyond multivariable adjustment. Propensity scores are not recommended if the analytical goal is pure prediction modeling.
| S-EPMC3740143 | biostudies-literature
Project description:Modeling sensitivity to drugs based on genetic characterizations is a significant challenge in the area of systems medicine. Ensemble based approaches such as Random Forests have been shown to perform well in both individual sensitivity prediction studies and team science based prediction challenges. However, Random Forests generate a deterministic predictive model for each drug based on the genetic characterization of the cell lines and ignores the relationship between different drug sensitivities during model generation. This application motivates the need for generation of multivariate ensemble learning techniques that can increase prediction accuracy and improve variable importance ranking by incorporating the relationships between different output responses. In this article, we propose a novel cost criterion that captures the dissimilarity in the output response structure between the training data and node samples as the difference in the two empirical copulas. We illustrate that copulas are suitable for capturing the multivariate structure of output responses independent of the marginal distributions and the copula based multivariate random forest framework can provide higher accuracy prediction and improved variable selection. The proposed framework has been validated on genomics of drug sensitivity for cancer and cancer cell line encyclopedia database.
| S-EPMC4684346 | biostudies-literature
Project description:Quantitative kinetic parameters derived from dynamic contrast-enhanced (DCE) data are dependent on signal measurement quality and choice of pharmacokinetic model. However, the fundamental optimization analysis method is equally important and its impact on pharmacokinetic parameters has been mostly overlooked. We examine the effects of those choices on accuracy and performance of parameter estimation using both computer processing unit and graphical processing unit (GPU) numerical optimization implementations and evaluate the improvements offered by a novel optimization approach. A test framework was developed where experimentally derived population-average arterial input function and randomly sampled parameter sets {Ktrans , Kep , Vb , ?} were used to generate known tissue curves. Five numerical optimization algorithms were evaluated: sequential quadratic programming, downhill simplex (Nelder-Mead), pattern search, simulated annealing, and differential evolution. This was combined with various objective function implementation details: delay approximation, discretization and varying sampling rates. Then, impact of noise and CPU/GPU implementation was tested for speed and accuracy. Finally, the optimal method was compared to conventional implementation as applied to clinical DCE computed tomography. Nelder-Mead, differential evolution and sequential quadratic programming produced good results on clean and noisy input data outperforming simulated annealing and pattern search in terms of speed and accuracy in the respective order of 10-8%, 10-7%, and ×10-6%). A novel approach for DCE numerical optimization (infinite impulse response with fractional delay approximation) was implemented on GPU for speed increase of at least 2 orders of magnitude. Applied to clinical data, the magnitude of overall parameter error was <10%.
| S-EPMC6403032 | biostudies-literature
Project description:In this paper, we explore mutual information based stock networks to build regular vine copula structure on high frequency log returns of stocks and use it for the estimation of Value at Risk (VaR) of a portfolio of stocks. Our model is a data driven model that learns from a high frequency time series data of log returns of top 50 stocks listed on the National Stock Exchange (NSE) in India for the year 2014. The Ljung-Box test revealed the presence of Autocorrelation as well as Heteroscedasticity in the underlying time series data. Analysing the goodness of fit of a number of variants of the GARCH model on each working day of the year 2014, that is, 229 days in all, it was observed that ARMA(1,1)-EGARCH(1,1) demonstrated the best fit. The joint probability distribution of the portfolio is computed by constructed an R-Vine copula structure on the data with the mutual information guided minimum spanning tree as the key building block. The joint PDF is then fed into the Monte-Carlo simulation procedure to compute the VaR. If we replace the mutual information by the Kendall's Tau in the construction of the R-Vine copula structure, the resulting VaR estimations were found to be inferior suggesting the presence of non-linear relationships among stock returns.
| S-EPMC8211166 | biostudies-literature
Project description:Identifying genetic variants associated with complex disease in high-dimensional data is a challenging problem, and complicated etiologies such as gene-gene interactions are often ignored in analyses. The data-mining method Random Forests (RF) can handle high-dimensions; however, in high-dimensional data, RF is not an effective filter for identifying risk factors associated with the disease trait via complex genetic models such as gene-gene interactions without strong marginal components. Here we propose an extension called Weighted Random Forests (wRF), which incorporates tree-level weights to emphasize more accurate trees in prediction and calculation of variable importance. We demonstrate through simulation and application to data from a genetic study of addiction that wRF can outperform RF in high-dimensional data, although the improvements are modest and limited to situations with effect sizes that are larger than what is realistic in genetics of complex disease. Thus, the current implementation of wRF is unlikely to improve detection of relevant predictors in high-dimensional genetic data, but may be applicable in other situations where larger effect sizes are anticipated.
| S-EPMC3912194 | biostudies-literature
Project description:Sorting performance can be evaluated with regard to Purity, Yield and/or Recovery of the sorted fraction. Purity is a check on the quality of the sample and the sort decisions made by the instrument. Recovery and Yield definitions vary with some authors regarding both as how efficient the instrument is at sorting the target particles from the original sample, others distinguishing Recovery from Yield, where the former is used to describe the accuracy of the instrument's sort count. Yield and Recovery are often neglected, mostly due to difficulties in their measurement. Purity of the sort product is often cited alone but is not sufficient to evaluate sorting performance. All of these three performance metrics require re-sampling of the sorted fraction. But, unlike Purity, calculating Yield and/or Recovery calls for the absolute counting of particles in the sorted fraction, which may not be feasible, particularly when dealing with rare populations and precious samples. In addition, the counting process itself involves large errors. Here we describe a new metric for evaluating instrument sort Recovery, defined as the number of particles sorted relative to the number of original particles to be sorted. This calculation requires only measuring the ratios of target and non-target populations in the original pre-sort sample and in the waste stream or center stream catch (CSC), avoiding re-sampling the sorted fraction and absolute counting. We called this new metric Rmax, since it corresponds to the maximum expected Recovery for a particular set of instrument parameters. Rmax is ideal to evaluate and troubleshoot the optimum drop-charge delay of the sorter, or any instrument related failures that will affect sort performance. It can be used as a daily quality control check but can be particularly useful to assess instrument performance before single-cell sorting experiments. Because we do not perturb the sort fraction we can calculate Rmax during the sort process, being especially valuable to check instrument performance during rare population sorts.
| S-EPMC4503806 | biostudies-other
Project description:Per- and polyfluoroalkyl substances (PFAS) are a very large (thousands of chemicals) category; these substances have important industrial and consumer product applications. PFAS are highly persistent in the environment, and some are known to pose human health hazard. Regulatory agencies worldwide are considering restrictions and outright bans of PFAS; however, little data exists to make informed decisions. Therefore, a prioritization strategy is urgently needed for evaluation of potential hazards of PFAS. Structure-based grouping could expedite selection of PFAS for testing; still, the hypothesis that structure-effect relationships exist requires confirmation. We tested 26 structurally diverse PFAS from 8 groups in two human cell types from organs that are thought to be targets for PFAS. We used human induced pluripotent stem cell-derived hepatocytes and cardiomyocytes and tested concentration-response effects on both cell function and gene expression. Few phenotypic effects were observed in hepatocytes, but negative chronotropy was observed for 8 of 26 PFAS. Substance- and cell-dependent transcriptomic changes were more pronounced; however, little evidence of group-specific effects was observed. In hepatocytes, we found up-regulation of stress-related and extracellular matrix organization pathways, and down-regulation of fat metabolism. In cardiomyocytes, contractility-related pathways were most affected. Using these data, we derived phenotypic and transcriptomic point of departure estimates and compared them to predicted PFAS exposures. The conservative estimates for bioactivity and exposure were used to derive margin-of-exposure (MOE) for each PFAS. We found that most (23 of 26) PFAS had MOE>1. Overall, our data suggests that chemical structure-based grouping of PFAS may not be an appropriate strategy to predict their biological effects. This means that testing of the individual PFAS would be needed for confident decision-making. Our proposed strategy of using two human cell types and considering both phenotypic and transcriptomic effects, combined with dose-response analysis, may be used for prioritization of PFAS.
2024-02-27 | GSE244109 | GEO