Browse
Submit Data
Databases
API
Help

Dataset Information

11 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Training replicable predictors in multiple studies.

ABSTRACT: This article considers replicability of the performance of predictors across studies. We suggest a general approach to investigating this issue, based on ensembles of prediction models trained on different studies. We quantify how the common practice of training on a single study accounts in part for the observed challenges in replicability of prediction performance. We also investigate whether ensembles of predictors trained on multiple studies can be combined, using unique criteria, to design robust ensemble learners trained upfront to incorporate replicability into different contexts and populations.

SUBMITTER: Patil P

PROVIDER: S-EPMC5856504 | biostudies-literature | 2018 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Training replicable predictors in multiple studies.

Patil Prasad P Parmigiani Giovanni G

Proceedings of the National Academy of Sciences of the United States of America 20180312 11

This article considers replicability of the performance of predictors across studies. We suggest a general approach to investigating this issue, based on ensembles of prediction models trained on different studies. We quantify how the common practice of training on a single study accounts in part for the observed challenges in replicability of prediction performance. We also investigate whether ensembles of predictors trained on multiple studies can be combined, using unique criteria, to design ...[more]

PMID: 29531060

Similar Datasets

Extracting replicable associations across multiple studies: Empirical Bayes algorithms for controlling the false discovery rate.

Project description:In almost every field in genomics, large-scale biomedical datasets are used to report associations. Extracting associations that recur across multiple studies while controlling the false discovery rate is a fundamental challenge. Here, we propose a new method to allow joint analysis of multiple studies. Given a set of p-values obtained from each study, the goal is to identify associations that recur in at least k > 1 studies while controlling the false discovery rate. We propose several new algorithms that differ in how the study dependencies are modeled, and compare them and extant methods under various simulated scenarios. The top algorithm, SCREEN (Scalable Cluster-based REplicability ENhancement), is our new algorithm that works in three stages: (1) clustering an estimated correlation network of the studies, (2) learning replicability (e.g., of genes) within clusters, and (3) merging the results across the clusters. When we applied SCREEN to two real datasets it greatly outperformed the results obtained via standard meta-analysis. First, on a collection of 29 case-control gene expression cancer studies, we detected a large set of consistently up-regulated genes related to proliferation and cell cycle regulation. These genes are both consistently up-regulated across many cancer studies, and are well connected in known gene networks. Second, on a recent pan-cancer study that examined the expression profiles of patients with and without mutations in the HLA complex, we detected a large active module of up-regulated genes that are both related to immune responses and are well connected in known gene networks. This module covers thrice more genes as compared to the original study at a similar false discovery rate, demonstrating the high power of SCREEN. An implementation of SCREEN is available in the supplement.

| S-EPMC5576761 | biostudies-literature

Depression symptom dimensions as predictors of antidepressant treatment outcome: replicable evidence for interest-activity symptoms.

Project description:Symptom dimensions have not yet been comprehensively tested as predictors of the substantial heterogeneity in outcomes of antidepressant treatment in major depressive disorder.We tested nine symptom dimensions derived from a previously published factor analysis of depression rating scales as predictors of outcome in 811 adults with moderate to severe depression treated with flexibly dosed escitalopram or nortriptyline in Genome-based Therapeutic Drugs for Depression (GENDEP). The effects of symptom dimensions were tested in mixed-effect regression models that controlled for overall initial depression severity, age, sex and recruitment centre. Significant results were tested for replicability in 3637 adult out-patients with non-psychotic major depression treated with citalopram in level I of Sequenced Treatment Alternatives to Relieve Depression (STAR*D).The interest-activity symptom dimension (reflecting low interest, reduced activity, indecisiveness and lack of enjoyment) at baseline strongly predicted poor treatment outcome in GENDEP, irrespective of overall depression severity, antidepressant type and outcome measure used. The prediction of poor treatment outcome by the interest-activity dimension was robustly replicated in STAR*D, independent of a comprehensive list of baseline covariates.Loss of interest, diminished activity and inability to make decisions predict poor outcome of antidepressant treatment even after adjustment for overall depression severity and other clinical covariates. The prominence of such symptoms may require additional treatment strategies and should be accounted for in future investigations of antidepressant response.

| S-EPMC3787526 | biostudies-literature

Co-enrollment of critically ill patients into multiple studies: patterns, predictors and consequences.

Project description:Research on co-enrollment practices and their impact are limited in the ICU setting. The objectives of this study were: 1) to describe patterns and predictors of co-enrollment of patients in a thromboprophylaxis trial, and 2) to examine the consequences of co-enrollment on clinical and trial outcomes.In an observational analysis of an international thromboprophylaxis trial in 67 ICUs, we examined the co-enrollment of critically ill medical-surgical patients into more than one study, and examined the clinical and trial outcomes among co-enrolled and non-co-enrolled patients.Among 3,746 patients enrolled in PROTECT (Prophylaxis for ThromboEmbolism in Critical Care Trial), 713 (19.0%) were co-enrolled in at least one other study (53.6% in a randomized trial, 37.0% in an observational study and 9.4% in both). Six factors independently associated with co-enrollment (all P < 0.001) were illness severity (odds ratio (OR) 1.35, 95% confidence interval (CI) 1.19 to 1.53 for each 10-point Acute Physiology and Chronic Health Evaluation (APACHE) II score increase), substitute decision-makers providing consent, rather than patients (OR 3.31, 2.03 to 5.41), experience of persons inviting consent (OR 2.67, 1.74 to 4.11 for persons with > 10 years' experience compared to persons with none), center size (all ORs > 10 for ICUs with > 15 beds), affiliation with trials groups (OR 5.59, 3.49 to 8.95), and main trial rather than pilot phase (all ORs > 8 for recruitment year beyond the pilot). Co-enrollment did not influence clinical or trial outcomes or risk of adverse events.Co-enrollment was strongly associated with features of the patients, research personnel, setting and study. Co-enrollment had no impact on trial results, and appeared safe, acceptable and feasible. Transparent reporting, scholarly discourse, ethical analysis and further research are needed on the complex topic of co-enrollment during critical illness.

| S-EPMC4056073 | biostudies-literature

Predictors of training-related improvement in visuomotor performance in patients with multiple sclerosis: A behavioural and MRI study.

Project description:BackgroundThe development of tailored recovery-oriented strategies in multiple sclerosis requires early identification of an individual's potential for functional recovery.ObjectiveTo identify predictors of visuomotor performance improvements, a proxy of functional recovery, using a predictive statistical model that combines demographic, clinical and magnetic resonance imaging (MRI) data.MethodsRight-handed multiple sclerosis patients underwent baseline disability assessment and MRI of the brain structure, function and vascular health. They subsequently undertook 4 weeks of right upper limb visuomotor practice. Changes in performance with practice were our outcome measure. We identified predictors of improvement in a training set of patients using lasso regression; we calculated the best performing model in a validation set and applied this model to a test set.ResultsPatients improved their visuomotor performance with practice. Younger age, better visuomotor abilities, less severe disease burden and concurrent use of preventive treatments predicted improvements. Neuroimaging localised outcome-relevant sensory motor regions, the microstructure and activity of which correlated with performance improvements.ConclusionInitial characteristics, including age, disease duration, visuo-spatial abilities, hand dexterity, self-evaluated disease impact and the presence of disease-modifying treatments, can predict functional recovery in individual patients, potentially improving their clinical management and stratification in clinical trials. MRI is a correlate of outcome, potentially supporting individual prognosis.

| S-EPMC8151554 | biostudies-literature

Predictors of neurofeedback training outcome: A systematic review.

Project description:Neurofeedback (NF), a training tool aimed at enhancing neural self-regulation, has been suggested as a complementary treatment option for neuropsychiatric disorders. Despite its potential as a neurobiological intervention directly targeting neural alterations underlying clinical symptoms, the efficacy of NF for the treatment of mental disorders has been questioned recently by negative findings obtained in randomized controlled trials (e.g., Cortese et al., 2016). A possible reason for insufficient group effects of NF trainings vs. placebo could be related to the high rate of participants who fail to self-regulate brain activity by NF ("non-learners"). Another reason could be the application of standardized NF protocols not adjusted to individual differences in pathophysiology. Against this background, we have summarized information on factors determining training and treatment success to provide a basis for the development of individualized training protocols and/or clinical indications. The present systematic review included 25 reports investigating predictors for the outcome of NF trainings in healthy individuals as well as patients affected by mental disorders or epilepsy. We selected these studies based on searches in EBSCOhost using combinations of the keywords "neurofeedback" and "predictor/predictors". As "NF training" we defined all NF applications with at least two sessions. The best available evidence exists for neurophysiological baseline parameters. Among them, the target parameters of the respective training seem to be of particular importance. However, particularities of the different experimental designs and outcome criteria restrict the interpretability of some of the information we extracted. Therefore, further research is needed to gain more profound knowledge about predictors of NF outcome.

| S-EPMC7327249 | biostudies-literature

Reproducible and replicable comparisons using SummarizedBenchmark.

Project description:Summary:Benchmark studies are widely used to compare and evaluate tools developed for answering various biological questions. Despite the popularity of these comparisons, the implementation is often ad hoc, with little consistency across studies. To address this problem, we developed SummarizedBenchmark, an R package and framework for organizing and structuring benchmark comparisons. SummarizedBenchmark defines a general grammar for benchmarking and allows for easier setup and execution of benchmark comparisons, while improving the reproducibility and replicability of such comparisons. We demonstrate the wide applicability of our framework using four examples from different applications. Availability and implementation:SummarizedBenchmark is an R package available through Bioconductor (http://bioconductor.org/packages/SummarizedBenchmark). Supplementary information:Supplementary data are available at Bioinformatics online.

| S-EPMC6298041 | biostudies-literature

Sample size requirements for training high-dimensional risk predictors.

Project description:A common objective of biomarker studies is to develop a predictor of patient survival outcome. Determining the number of samples required to train a predictor from survival data is important for designing such studies. Existing sample size methods for training studies use parametric models for the high-dimensional data and cannot handle a right-censored dependent variable. We present a new training sample size method that is non-parametric with respect to the high-dimensional vectors, and is developed for a right-censored response. The method can be applied to any prediction algorithm that satisfies a set of conditions. The sample size is chosen so that the expected performance of the predictor is within a user-defined tolerance of optimal. The central method is based on a pilot dataset. To quantify uncertainty, a method to construct a confidence interval for the tolerance is developed. Adequacy of the size of the pilot dataset is discussed. An alternative model-based version of our method for estimating the tolerance when no adequate pilot dataset is available is presented. The model-based method requires a covariance matrix be specified, but we show that the identity covariance matrix provides adequate sample size when the user specifies three key quantities. Application of the sample size method to two microarray datasets is discussed.

| S-EPMC3770001 | biostudies-literature

Methylation studies in multiple human tissues N66

Project description:An Infinium microarray platform (GPL28271, HorvathMammalMethylChip40) was used to generate DNA methylation data from human blood samples. n=92 brain samples (n=46 samples from cerebellum, n=46 from cerebral cortex).

2023-08-05 | GSE184218 | GEO

Methylation studies in multiple human tissues N47

Project description:An Infinium microarray platform (GPL28271, HorvathMammalMethylChip40) was used to generate DNA methylation data from human blood samples. n=95 blood samples.

2023-08-05 | GSE184215 | GEO

Methylation studies in multiple human tissues N36

Project description:An Infinium microarray platform (GPL28271, HorvathMammalMethylChip40) was used to generate DNA methylation data from human tissue samples. n=96 tissues. Skin (n=72), keratinocyte (n=20), fibroblast (1), epidermis (2), dermis (1)

2023-08-05 | GSE184213 | GEO

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data