Browse
Submit Data
Databases
API
Help

Dataset Information

15 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Experiments from unfinished Registered Reports in the Reproducibility Project: Cancer Biology.

ABSTRACT: As part of the Reproducibility Project: Cancer Biology, we published Registered Reports that described how we intended to replicate selected experiments from 29 high-impact preclinical cancer biology papers published between 2010 and 2012. Replication experiments were completed and Replication Studies reporting the results were submitted for 18 papers, of which 17 were accepted and published by eLife with the rejected paper posted as a preprint. Here, we report the status and outcomes obtained for the remaining 11 papers. Four papers initiated experimental work but were stopped without any experimental outcomes. Two papers resulted in incomplete outcomes due to unanticipated challenges when conducting the experiments. For the remaining five papers only some of the experiments were completed with the other experiments incomplete due to mundane technical or unanticipated methodological challenges. The experiments from these papers, along with the other experiments attempted as part of the Reproducibility Project: Cancer Biology, provides evidence about the challenges of repeating preclinical cancer biology experiments and the replicability of the completed experiments.

SUBMITTER: Errington TM

PROVIDER: S-EPMC8651290 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

Forensic science needs registered reports.

Project description:The registered report (RR) format is rapidly being adopted by scientific researchers and journals. RRs flip the peer review process, with reviewers evaluating proposed methods, rather than the data and findings. Editors then accept or reject articles largely based on the pre-data collection review. Accordingly, RRs reduce the incentive for researchers to exaggerate their findings, and they make any data-driven changes to the methods and analysis more conspicuous. They also reduce publication bias, ensuring studies with null or otherwise unfavorable results are published. RRs are being used in many fields to improve research practices and increase confidence in study findings. The authors suggest RRs ought to be the default way in which validation studies are conducted and reported in the forensic sciences. They produce more reliable findings, advance criminal justice values, and will lead to several efficiencies in the research process.

| S-EPMC7219172 | biostudies-literature

Registered reports: an early example and analysis.

Project description:The recent 'replication crisis' in psychology has focused attention on ways of increasing methodological rigor within the behavioral sciences. Part of this work has involved promoting 'Registered Reports', wherein journals peer review papers prior to data collection and publication. Although this approach is usually seen as a relatively recent development, we note that a prototype of this publishing model was initiated in the mid-1970s by parapsychologist Martin Johnson in the European Journal of Parapsychology (EJP). A retrospective and observational comparison of Registered and non-Registered Reports published in the EJP during a seventeen-year period provides circumstantial evidence to suggest that the approach helped to reduce questionable research practices. This paper aims both to bring Johnson's pioneering work to a wider audience, and to investigate the positive role that Registered Reports may play in helping to promote higher methodological and statistical standards.

| S-EPMC6339469 | biostudies-literature

Maximum Rank Reproducibility: A Nonparametric Approach to Assessing Reproducibility in Replicate Experiments.

Project description:The identification of reproducible signals from the results of replicate high-throughput experiments is an important part of modern biological research. Often little is known about the dependence structure and the marginal distribution of the data, motivating the development of a nonparametric approach to assess reproducibility. The procedure, which we call the maximum rank reproducibility (MaRR) procedure, uses a maximum rank statistic to parse reproducible signals from noise without making assumptions about the distribution of reproducible signals. Because it uses the rank scale this procedure can be easily applied to a variety of data types. One application is to assess the reproducibility of RNA-seq technology using data produced by the sequencing quality control (SEQC) consortium, which coordinated a multi-laboratory effort to assess reproducibility across three RNA-seq platforms. Our results on simulations and SEQC data show that the MaRR procedure effectively controls false discovery rates, has desirable power properties, and compares well to existing methods. Supplementary materials for this article are available online.

| S-EPMC6596287 | biostudies-literature

State-of-the-Art Data Management: Improving the Reproducibility, Consistency, and Traceability of Structural Biology and in Vitro Biochemical Experiments.

Project description:Efficient and comprehensive data management is an indispensable component of modern scientific research and requires effective tools for all but the most trivial experiments. The LabDB system developed and used in our laboratory was originally designed to track the progress of a structure determination pipeline in several large National Institutes of Health (NIH) projects. While initially designed for structural biology experiments, its modular nature makes it easily applied in laboratories of various sizes in many experimental fields. Over many years, LabDB has transformed into a sophisticated system integrating a range of biochemical, biophysical, and crystallographic experimental data, which harvests data both directly from laboratory instruments and through human input via a web interface. The core module of the system handles many types of universal laboratory management data, such as laboratory personnel, chemical inventories, storage locations, and custom stock solutions. LabDB also tracks various biochemical experiments, including spectrophotometric and fluorescent assays, thermal shift assays, isothermal titration calorimetry experiments, and more. LabDB has been used to manage data for experiments that resulted in over 1200 deposits to the Protein Data Bank (PDB); the system is currently used by the Center for Structural Genomics of Infectious Diseases (CSGID) and several large laboratories. This chapter also provides examples of data mining analyses and warnings about incomplete and inconsistent experimental data. These features, together with its capabilities for detailed tracking, analysis, and auditing of experimental data, make the described system uniquely suited to inspect potential sources of irreproducibility in life sciences research.

| S-EPMC8019398 | biostudies-literature

Reproducibility is a Process, Not an Achievement: The Replicability of IR Reproducibility Experiments

Project description:This paper espouses a view of reproducibility in the computational sciences as a process and not just a point-in-time “achievement”. As a concrete case study, we revisit the Open-Source IR Reproducibility Challenge from 2015 and attempt to replicate those experiments: four years later, are those computational artifacts still functional? Perhaps not surprisingly, we are not able to replicate most of the retrieval runs encapsulated by those artifacts in a modern computational environment. We outline the various idiosyncratic reasons why, distilled into a series of “lessons learned” to help form an emerging set of best practices for the long-term sustainability of reproducibility efforts.

| S-EPMC7148033 | biostudies-literature

A qualitative analysis of stakeholder experiences with Registered Reports Funding Partnerships.

Project description:Background: Registered Reports (RRs) could be a way to increase the quality of scientific research and literature, such as by reducing publication bias and increasing the rigour of study designs. These potential benefits have led to Registered Report funding partnerships (RRFPs or partnerships for short) between research funders and academic journals who collaborate to encourage researchers to publish RRs. In this study we investigated the research question: "What are the experiences of the stakeholders (authors, reviewers, journal editors, funders) in the various partnership models?". Our companion paper addresses a related, but separate, research question. Methods: We conducted a thematic analysis of 32 semi-structured interviews with stakeholders (funders, editors, authors, reviewers, matchmakers) from six partnerships. Results: Interviewees had highly variable perceptions and experiences, reflecting the complex and nuanced impacts of partnerships. We identified 6 themes: "Importance of communication with authors and reviewers", "Influence on study design", "Appropriateness of partners", "Potential to reduce publication bias", "Impact on reviewer workload", and "Insufficient evidence". Conclusions: This was the first investigation into these novel initiatives. We hope that our findings can benefit and shape current and future partnerships.

| S-EPMC8672223 | biostudies-literature

Nanomotor tracking experiments at the edge of reproducibility.

Project description:The emerging field of self-propelling micro/nanorobots is teeming with a wide variety of novel micro/nanostructures, which are tested here for self-propulsion in a liquid environment. As the size of these microscopic movers diminishes into the fully nanosized region, the ballistic paths of an active micromotor become a random walk of colloidal particles. To test such colloidal samples for self-propulsion, the commonly adopted "golden rule" is to refer to the mean squared displacement (MSD) function of the measured particle tracks. The practical significance of the result strongly depends on the amount of collected particle data and the sampling rate of the particle track. Because micro/nanomotor preparation methods are mostly low-yield, the amount of used experimental data in published results is often on the edge of reproducibility. To address the situation, we perform MSD analysis on an experimental as well as simulated dataset. These data are used to explore the effects of MSD analysis on limited data and several situations where the lack of data can lead to insignificant results.

| S-EPMC6744498 | biostudies-literature

Quantify and control reproducibility in high-throughput experiments.

Project description:Ensuring reproducibility of results in high-throughput experiments is crucial for biomedical research. Here, we propose a set of computational methods, INTRIGUE, to evaluate and control reproducibility in high-throughput settings. Our approaches are built on a new definition of reproducibility that emphasizes directional consistency when experimental units are assessed with signed effect size estimates. The proposed methods are designed to (1) assess the overall reproducible quality of multiple studies and (2) evaluate reproducibility at the individual experimental unit levels. We demonstrate the proposed methods in detecting unobserved batch effects via simulations. We further illustrate the versatility of the proposed methods in transcriptome-wide association studies: in addition to reproducible quality control, they are also suited to investigating genuine biological heterogeneity. Finally, we discuss the potential extensions of the proposed methods in other vital areas of reproducible research (for example, publication bias and conceptual replications).

| S-EPMC8240032 | biostudies-literature

A Bayesian Perspective on the Reproducibility Project: Psychology.

Project description:We revisit the results of the recent Reproducibility Project: Psychology by the Open Science Collaboration. We compute Bayes factors-a quantity that can be used to express comparative evidence for an hypothesis but also for the null hypothesis-for a large subset (N = 72) of the original papers and their corresponding replication attempts. In our computation, we take into account the likely scenario that publication bias had distorted the originally published results. Overall, 75% of studies gave qualitatively similar results in terms of the amount of evidence provided. However, the evidence was often weak (i.e., Bayes factor < 10). The majority of the studies (64%) did not provide strong evidence for either the null or the alternative hypothesis in either the original or the replication, and no replication attempts provided strong evidence in favor of the null. In all cases where the original paper provided strong evidence but the replication did not (15%), the sample size in the replication was smaller than the original. Where the replication provided strong evidence but the original did not (10%), the replication sample size was larger. We conclude that the apparent failure of the Reproducibility Project to replicate many target effects can be adequately explained by overestimation of effect sizes (or overestimation of evidence against the null hypothesis) due to small sample sizes and publication bias in the psychological literature. We further conclude that traditional sample sizes are insufficient and that a more widespread adoption of Bayesian methods is desirable.

| S-EPMC4769355 | biostudies-literature

SuperPlots: Communicating reproducibility and variability in cell biology.

Project description:P values and error bars help readers infer whether a reported difference would likely recur, with the sample size n used for statistical tests representing biological replicates, independent measurements of the population from separate experiments. We provide examples and practical tutorials for creating figures that communicate both the cell-level variability and the experimental reproducibility.

| S-EPMC7265319 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data