Browse
Submit Data
Databases
API
Help

Dataset Information

30 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Quantify and control reproducibility in high-throughput experiments.

ABSTRACT: Ensuring reproducibility of results in high-throughput experiments is crucial for biomedical research. Here, we propose a set of computational methods, INTRIGUE, to evaluate and control reproducibility in high-throughput settings. Our approaches are built on a new definition of reproducibility that emphasizes directional consistency when experimental units are assessed with signed effect size estimates. The proposed methods are designed to (1) assess the overall reproducible quality of multiple studies and (2) evaluate reproducibility at the individual experimental unit levels. We demonstrate the proposed methods in detecting unobserved batch effects via simulations. We further illustrate the versatility of the proposed methods in transcriptome-wide association studies: in addition to reproducible quality control, they are also suited to investigating genuine biological heterogeneity. Finally, we discuss the potential extensions of the proposed methods in other vital areas of reproducible research (for example, publication bias and conceptual replications).

SUBMITTER: Zhao Y

PROVIDER: S-EPMC8240032 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Quantitative reproducibility analysis for identifying reproducible targets from high-throughput experiments.

Project description:High-throughput assays are widely used in biological research to select potential targets. One single high-throughput experiment can efficiently study a large number of candidates simultaneously, but is subject to substantial variability. Therefore it is scientifically important to performance quantitative reproducibility analysis to identify reproducible targets with consistent and significant signals across replicate experiments. A few methods exist, but all have limitations.In this paper, we propose a new method for identifying reproducible targets. Considering a Bayesian hierarchical model, we show that the test statistics from replicate experiments follow a mixture of multivariate Gaussian distributions, with the one component with zero-mean representing the irreproducible targets.A target is thus classified as reproducible or irreproducible based on its posterior probability belonging to the reproducible components. We study the performance of our proposed method using simulations and a real data example.The proposed method is shown to have favorable performance in identifying reproducible targets compared to other methods.

| S-EPMC5553769 | biostudies-other

Assessing reproducibility of high-throughput experiments in the case of missing data.

Project description:High-throughput experiments are an essential part of modern biological and biomedical research. The outcomes of high-throughput biological experiments often have a lot of missing observations due to signals below detection levels. For example, most single-cell RNA-seq (scRNA-seq) protocols experience high levels of dropout due to the small amount of starting material, leading to a majority of reported expression levels being zero. Though missing data contain information about reproducibility, they are often excluded in the reproducibility assessment, potentially generating misleading assessments. In this article, we develop a regression model to assess how the reproducibility of high-throughput experiments is affected by the choices of operational factors (eg, platform or sequencing depth) when a large number of measurements are missing. Using a latent variable approach, we extend correspondence curve regression, a recently proposed method for assessing the effects of operational factors to reproducibility, to incorporate missing values. Using simulations, we show that our method is more accurate in detecting differences in reproducibility than existing measures of reproducibility. We illustrate the usefulness of our method using a single-cell RNA-seq dataset collected on HCT116 cells. We compare the reproducibility of different library preparation platforms and study the effect of sequencing depth on reproducibility, thereby determining the cost-effective sequencing depth that is required to achieve sufficient reproducibility.

| S-EPMC9039958 | biostudies-literature

A regression framework for assessing covariate effects on the reproducibility of high-throughput experiments.

Project description:The outcome of high-throughput biological experiments is affected by many operational factors in the experimental and data-analytical procedures. Understanding how these factors affect the reproducibility of the outcome is critical for establishing workflows that produce replicable discoveries. In this article, we propose a regression framework, based on a novel cumulative link model, to assess the covariate effects of operational factors on the reproducibility of findings from high-throughput experiments. In contrast to existing graphical approaches, our method allows one to succinctly characterize the simultaneous and independent effects of covariates on reproducibility and to compare reproducibility while controlling for potential confounding variables. We also establish a connection between our model and certain Archimedean copula models. This connection not only offers our regression framework an interpretation in copula models, but also provides guidance on choosing the functional forms of the regression. Furthermore, it also opens a new way to interpret and utilize these copulas in the context of reproducibility. Using simulations, we show that our method produces calibrated type I error and is more powerful in detecting difference in reproducibility than existing measures of agreement. We illustrate the usefulness of our method using a ChIP-seq study and a microarray study.

| S-EPMC5975112 | biostudies-literature

HiTC: exploration of high-throughput 'C' experiments.

Project description:The R/Bioconductor package HiTC facilitates the exploration of high-throughput 3C-based data. It allows users to import and export 'C' data, to transform, normalize, annotate and visualize interaction maps. The package operates within the Bioconductor framework and thus offers new opportunities for future development in this field.The R package HiTC is available from the Bioconductor website. A detailed vignette provides additional documentation and help for using the package.

| S-EPMC3476334 | biostudies-other

Discovering collectively informative descriptors from high-throughput experiments.

Project description:BackgroundImprovements in high-throughput technology and its increasing use have led to the generation of many highly complex datasets that often address similar biological questions. Combining information from these studies can increase the reliability and generalizability of results and also yield new insights that guide future research.ResultsThis paper describes a novel algorithm called BLANKET for symmetric analysis of two experiments that assess informativeness of descriptors. The experiments are required to be related only in that their descriptor sets intersect substantially and their definitions of case and control are consistent. From resulting lists of n descriptors ranked by informativeness, BLANKET determines shortlists of descriptors from each experiment, generally of different lengths p and q. For any pair of shortlists, four numbers are evident: the number of descriptors appearing in both shortlists, in exactly one shortlist, or in neither shortlist. From the associated contingency table, BLANKET computes Right Fisher Exact Test (RFET) values used as scores over a plane of possible pairs of shortlist lengths 12. BLANKET then chooses a pair or pairs with RFET score less than a threshold; the threshold depends upon n and shortlist length limits and represents a quality of intersection achieved by less than 5% of random lists.ConclusionsResearchers seek within a universe of descriptors some minimal subset that collectively and efficiently predicts experimental outcomes. Ideally, any smaller subset should be insufficient for reliable prediction and any larger subset should have little additional accuracy. As a method, BLANKET is easy to conceptualize and presents only moderate computational complexity. Many existing databases could be mined using BLANKET to suggest optimal sets of predictive descriptors.

| S-EPMC2813853 | biostudies-literature

Quantify permeability using on-a-chip models in high-throughput applications.

Project description:Traditionally, to quantify permeability of a biological barrier, the initial slope is used, based on the assumption of sink condition (concentration of the donor is constant, and the receiver increases less than 10%). With on-a-chip barrier models, this assumption fails in cell-free or leaky conditions, which requires the use of the exact solution. To encounter a time delay from performing the assay and acquiring the data, we present a protocol with the exact equation modified to incorporate a time offset.

| S-EPMC10037215 | biostudies-literature

Cosputtered composition-spread reproducibility established by high-throughput x-ray fluorescence.

Project description:We describe the characterization of sputtered yttria-zirconia composition spread thin films by x-ray fluorescence (XRF). We also discuss our automated analysis of the XRF data, which was collected in a high throughput experiment at the Cornell High Energy Synchrotron Source. The results indicate that both the composition reproducibility of the library deposition and the composition measurements have a precision of better than 1 atomic percent.

| S-EPMC4043122 | biostudies-other

Independent filtering increases detection power for high-throughput experiments.

Project description:With high-dimensional data, variable-by-variable statistical testing is often used to select variables whose behavior differs across conditions. Such an approach requires adjustment for multiple testing, which can result in low statistical power. A two-stage approach that first filters variables by a criterion independent of the test statistic, and then only tests variables which pass the filter, can provide higher power. We show that use of some filter/test statistics pairs presented in the literature may, however, lead to loss of type I error control. We describe other pairs which avoid this problem. In an application to microarray data, we found that gene-by-gene filtering by overall variance followed by a t-test increased the number of discoveries by 50%. We also show that this particular statistic pair induces a lower bound on fold-change among the set of discoveries. Independent filtering-using filter/test pairs that are independent under the null hypothesis but correlated under the alternative-is a general approach that can substantially increase the efficiency of experiments.

| S-EPMC2906865 | biostudies-literature

Improving reproducibility by using high-throughput observational studies with empirical calibration.

Project description:Concerns over reproducibility in science extend to research using existing healthcare data; many observational studies investigating the same topic produce conflicting results, even when using the same data. To address this problem, we propose a paradigm shift. The current paradigm centres on generating one estimate at a time using a unique study design with unknown reliability and publishing (or not) one estimate at a time. The new paradigm advocates for high-throughput observational studies using consistent and standardized methods, allowing evaluation, calibration and unbiased dissemination to generate a more reliable and complete evidence base. We demonstrate this new paradigm by comparing all depression treatments for a set of outcomes, producing 17 718 hazard ratios, each using methodology on par with current best practice. We furthermore include control hypotheses to evaluate and calibrate our evidence generation process. Results show good transitivity and consistency between databases, and agree with four out of the five findings from clinical trials. The distribution of effect size estimates reported in the literature reveals an absence of small or null effects, with a sharp cut-off at p = 0.05. No such phenomena were observed in our results, suggesting more complete and more reliable evidence.This article is part of a discussion meeting issue 'The growing ubiquity of algorithms in society: implications, impacts and innovations'.

| S-EPMC6107542 | biostudies-literature

High-throughput isotopic analysis of RNA microarrays to quantify microbial resource use.

Project description:Most microorganisms remain uncultivated, and typically their ecological roles must be inferred from diversity and genomic studies. To directly measure functional roles of uncultivated microbes, we developed Chip-stable isotope probing (SIP), a high-sensitivity, high-throughput SIP method performed on a phylogenetic microarray (chip). This approach consists of microbial community incubations with isotopically labeled substrates, hybridization of the extracted community rRNA to a microarray and measurement of isotope incorporation--and therefore substrate use--by secondary ion mass spectrometer imaging (NanoSIMS). Laboratory experiments demonstrated that Chip-SIP can detect isotopic enrichment of 0.5 atom % (13)C and 0.1 atom % (15)N, thus permitting experiments with short incubation times and low substrate concentrations. We applied Chip-SIP analysis to a natural estuarine community and quantified amino acid, nucleic acid or fatty acid incorporation by 81 distinct microbial taxa, thus demonstrating that resource partitioning occurs with relatively simple organic substrates. The Chip-SIP approach expands the repertoire of stable isotope-enabled methods available to microbial ecologists and provides a means to test genomics-generated hypotheses about biogeochemical function in any natural environment.

| S-EPMC3358021 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data