SuperPlots: Communicating reproducibility and variability in cell biology.
Ontology highlight
ABSTRACT:
SUBMITTER: Lord SJ
PROVIDER: S-EPMC7265319 | biostudies-literature | 2020 Jun
REPOSITORIES: biostudies-literature
Publications
SuperPlots: Communicating reproducibility and variability in cell biology.
Lord Samuel J SJ Velle Katrina B KB Mullins R Dyche RD Fritz-Laylin Lillian K LK
The Journal of cell biology 20200601 6
P values and error bars help readers infer whether a reported difference would likely recur, with the sample size n used for statistical tests representing biological replicates, independent measurements of the population from separate experiments. We provide examples and practical tutorials for creating figures that communicate both the cell-level variability and the experimental reproducibility. ...[more]
Similar Datasets
Project description:Biological engineers often find it useful to communicate using diagrams. These diagrams can include information both about the structure of the nucleic acid sequences they are engineering and about the functional relationships between features of these sequences and/or other molecular species. A number of conventions and practices have begun to emerge within synthetic biology for creating such diagrams, and the Synthetic Biology Open Language Visual (SBOL Visual) has been developed as a standard to organize, systematize, and extend such conventions in order to produce a coherent visual language. Here, we describe SBOL Visual version 2, which expands previous diagram standards to include new functional interactions, categories of molecular species, support for families of glyph variants, and the ability to indicate modular structure and mappings between elements of a system. SBOL Visual 2 also clarifies a number of requirements and best practices, significantly expands the collection of glyphs available to describe genetic features, and can be readily applied using a wide variety of software tools, both general and bespoke.
| S-EPMC8023477 | biostudies-literature
Project description:Parameters describing dynamic cerebral autoregulation (DCA) have limited reproducibility. In an international, multi-center study, we evaluated the influence of multiple analytical methods on the reproducibility of DCA. Fourteen participating centers analyzed repeated measurements from 75 healthy subjects, consisting of 5 min of spontaneous fluctuations in blood pressure and cerebral blood flow velocity signals, based on their usual methods of analysis. DCA methods were grouped into three broad categories, depending on output types: (1) transfer function analysis (TFA); (2) autoregulation index (ARI); and (3) correlation coefficient. Only TFA gain in the low frequency (LF) band showed good reproducibility in approximately half of the estimates of gain, defined as an intraclass correlation coefficient (ICC) of >0.6. None of the other DCA metrics had good reproducibility. For TFA-like and ARI-like methods, ICCs were lower than values obtained with surrogate data (p < 0.05). For TFA-like methods, ICCs were lower for the very LF band (gain 0.38 ± 0.057, phase 0.17 ± 0.13) than for LF band (gain 0.59 ± 0.078, phase 0.39 ± 0.11, p ≤ 0.001 for both gain and phase). For ARI-like methods, the mean ICC was 0.30 ± 0.12 and for the correlation methods 0.24 ± 0.23. Based on comparisons with ICC estimates obtained from surrogate data, we conclude that physiological variability or non-stationarity is likely to be the main reason for the poor reproducibility of DCA parameters.
| S-EPMC6634255 | biostudies-literature
Project description:How easy is it to reproduce the results found in a typical computational biology paper? Either through experience or intuition the reader will already know that the answer is with difficulty or not at all. In this paper we attempt to quantify this difficulty by reproducing a previously published paper for different classes of users (ranging from users with little expertise to domain experts) and suggest ways in which the situation might be improved. Quantification is achieved by estimating the time required to reproduce each of the steps in the method described in the original paper and make them part of an explicit workflow that reproduces the original results. Reproducing the method took several months of effort, and required using new versions and new software that posed challenges to reconstructing and validating the results. The quantification leads to "reproducibility maps" that reveal that novice researchers would only be able to reproduce a few of the steps in the method, and that only expert researchers with advance knowledge of the domain would be able to reproduce the method in its entirety. The workflow itself is published as an online resource together with supporting software and data. The paper concludes with a brief discussion of the complexities of requiring reproducibility in terms of cost versus benefit, and a desiderata with our observations and guidelines for improving reproducibility. This has implications not only in reproducing the work of others from published papers, but reproducing work from one's own laboratory.
| S-EPMC3842296 | biostudies-literature
Project description:As part of the Reproducibility Project: Cancer Biology, we published Registered Reports that described how we intended to replicate selected experiments from 29 high-impact preclinical cancer biology papers published between 2010 and 2012. Replication experiments were completed and Replication Studies reporting the results were submitted for 18 papers, of which 17 were accepted and published by eLife with the rejected paper posted as a preprint. Here, we report the status and outcomes obtained for the remaining 11 papers. Four papers initiated experimental work but were stopped without any experimental outcomes. Two papers resulted in incomplete outcomes due to unanticipated challenges when conducting the experiments. For the remaining five papers only some of the experiments were completed with the other experiments incomplete due to mundane technical or unanticipated methodological challenges. The experiments from these papers, along with the other experiments attempted as part of the Reproducibility Project: Cancer Biology, provides evidence about the challenges of repeating preclinical cancer biology experiments and the replicability of the completed experiments.
| S-EPMC8651290 | biostudies-literature
Project description:Objective: Measurements of heart rate variability (HRV) during sleep have become increasingly popular as sleep could provide an optimal state for HRV assessments. While sleep stages have been reported to affect HRV, the effect of sleep stages on the variance of HRV parameters were hardly investigated. We aimed to assess the variance of HRV parameters during the different sleep stages. Further, we tested the accuracy of an algorithm using HRV to identify a 5-min segment within an episode of slow wave sleep (SWS, deep sleep). Methods: Polysomnographic (PSG) sleep recordings of 3 nights of 15 healthy young males were analyzed. Sleep was scored according to conventional criteria. HRV parameters of consecutive 5-min segments were analyzed within the different sleep stages. The total variance of HRV parameters was partitioned into between-subjects variance, between-nights variance, and between-segments variance and compared between the different sleep stages. Intra-class correlation coefficients of all HRV parameters were calculated for all sleep stages. To identify an SWS segment based on HRV, Pearson correlation coefficients of consecutive R-R intervals (rRR) of moving 5-min windows (20-s steps). The linear trend was removed from the rRR time series and the first segment with rRR values 0.1 units below the mean rRR for at least 10 min was identified. A 5-min segment was placed in the middle of such an identified segment and the corresponding sleep stage was used to assess the accuracy of the algorithm. Results: Good reproducibility within and across nights was found for heart rate in all sleep stages and for high frequency (HF) power in SWS. Reproducibility of low frequency (LF) power and of LF/HF was poor in all sleep stages. Of all the 5-min segments selected based on HRV data, 87% were accurately located within SWS. Conclusions: SWS, a stable state that, in contrast to waking, is unaffected by internal and external factors, is a reproducible state that allows reliable determination of heart rate, and HF power, and can satisfactorily be detected based on R-R intervals, without the need of full PSG. Sleep may not be an optimal condition to assess LF power and LF/HF power ratio.
| S-EPMC5767731 | biostudies-literature
Project description:Identifying image features that are robust with respect to segmentation variability is a tough challenge in radiomics. So far, this problem has mainly been tackled in test-retest analyses. In this work we analyse radiomics feature reproducibility in two phases: first with manual segmentations provided by four expert readers and second with probabilistic automated segmentations using a recently developed neural network (PHiseg). We test feature reproducibility on three publicly available datasets of lung, kidney and liver lesions. We find consistent results both over manual and automated segmentations in all three datasets and show that there are subsets of radiomic features which are robust against segmentation variability and other radiomic features which are prone to poor reproducibility under differing segmentations. By providing a detailed analysis of robustness of the most common radiomics features across several datasets, we envision that more reliable and reproducible radiomic models can be built in the future based on this work.
| S-EPMC7391354 | biostudies-literature
Project description:Genome editing tools have already revolutionized biomedical research and are also expected to have an important impact in the clinic. However, their extensive use in research has revealed much unpredictability, both off and on target, in the outcome of their application. We discuss the challenges associated with this unpredictability, both for research and in the clinic. For the former, an extensive validation of the model is essential. For the latter, potential unpredicted activity does not preclude the use of these tools but requires that molecular evidence to underpin the relevant risk:benefit evaluation is available. Safe and successful clinical application will also depend on the mode of delivery and the cellular context.
| S-EPMC7264426 | biostudies-literature
Project description:Most biomarkers for the diagnosis and prognosis of multiple sclerosis (MS) are not yet clinically available due to low reproducibility. Here, we show that times of day affect gene expression profiles in MS patients. Thus sample collection has to be standardized based on the time-points biomarkers are expressed to increase reproducibility. In this study, we examined transcriptome profiles in whole blood of MS patients collected at two different time-points of the day, around 2 pm as the daytime and 9 pm as the nighttime. We observed the significantly changed gene expression profiles in the relapsing patients at nighttime compared to their daytime samples and the remitting patients at nighttime. Among all DEGs, we focused on differentially expressed genes (DEGs) related to immune responses because of association with clinical outcomes of MS. 68 immune responses-associated DEGs significantly changed their expression in the relapsing patients at nighttime but not in their daytime samples and remitting patients at nighttime. Therefore, this study shows that: 1) gene expresion patterns in relapsing patients are changed compared to remitting patients; 2) altered gene expression patterns in relpasing patients are detected at nighttim; 3) times of day to collect samples from MS patients should be standardized based on the gene expression patterns across a day.
2021-11-03 | GSE150337 | GEO
Project description:BACKGROUND:There is growing interest in the use of multiplexed aptamer-based assays for large-scale proteomic studies. However, the analytic, short- and long-term variation of the measured proteins is largely uncharacterized. METHODS:We quantified 4001 plasma protein analytes from 42 participants in the Atherosclerosis Risk in Communities (ARIC) Study in split samples and at multiple visits using a multiplexed modified aptamer assay. We calculated the CV, Spearman correlation, and intraclass correlation (ICC) between split samples and evaluated the short-term (4-9 weeks) and long-term (approximately 20 years) variability using paired t-tests with log-transformed protein concentrations and Bonferroni-corrected significance thresholds. We performed principal component (PC) analysis of protein analyte concentrations and evaluated their associations with age, sex, race, and estimated glomerular filtration rate (eGFR). RESULTS:The mean baseline age was 57 years at the first visit, 43% of participants were male and 57% were white. Among 3693 protein analytes that passed quality control, half (n = 1846) had CVs < 5.0%, Spearman correlations > 0.89, and ICCs > 0.96 among the split samples. Over the short term, only 1 analyte had a statistically significant difference between the 2 time points, whereas, over approximately 20 years, 866 analytes (23.4%) had statistically significant differences (P < 1.4 × 10-5, 681 increased, 185 decreased). PC1 had high correlations with age (-0.73) and eGFR (0.60). PC2 had moderate correlation with male sex (0.18) and white race (0.31). CONCLUSIONS:Multiplexed modified aptamer technology can assay thousands of proteins with excellent precision. Our results support the potential for large-scale studies of the plasma proteome over the lifespan.
| S-EPMC6814271 | biostudies-literature
Project description:While ApoE expression by myeloid cells is recognized to control inflammation, whether such benefits can be communicated via extracellular vesicles including exosomes is not known. Through the study of exosomes produced by macrophages derived from the bone marrow of Wildtype (WT-BMDM-exo) and ApoE deficient (EKO-BMDM-exo) mice, we uncover a critical role of ApoE in regulating cell signaling properties. We treated splenic CD4+ T cells with EKO-BMDM-exo, WT-BMDM-exo, or PBS and performed gene expression profiling analysis. We found that CD4+ T cells treated with EKO-BMDM-exo show elevations in genes involved in the T-cell receptor complex (Cd3e, Cd247, & Cd4) and Il2rg, a key component of cytokine receptors on CD4+ T cells. Taken together, our data unveil a novel property of macrophage ApoE in controlling adaptive immunity by their secreted exosomes.
2023-08-23 | GSE237731 | GEO