NPARS-A Novel Approach to Address Accuracy and Reproducibility in Genomic Data Science.
Ontology highlight
ABSTRACT:
SUBMITTER: Ma L
PROVIDER: S-EPMC8503682 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:A reproducibility crisis is a situation where many scientific studies cannot be reproduced. Inappropriate practices of science, such as HARKing, p-hacking, and selective reporting of positive results, have been suggested as causes of irreproducibility. In this editorial, I propose that a lack of raw data or data fabrication is another possible cause of irreproducibility.As an Editor-in-Chief of Molecular Brain, I have handled 180 manuscripts since early 2017 and have made 41 editorial decisions categorized as "Revise before review," requesting that the authors provide raw data. Surprisingly, among those 41 manuscripts, 21 were withdrawn without providing raw data, indicating that requiring raw data drove away more than half of the manuscripts. I rejected 19 out of the remaining 20 manuscripts because of insufficient raw data. Thus, more than 97% of the 41 manuscripts did not present the raw data supporting their results when requested by an editor, suggesting a possibility that the raw data did not exist from the beginning, at least in some portions of these cases.Considering that any scientific study should be based on raw data, and that data storage space should no longer be a challenge, journals, in principle, should try to have their authors publicize raw data in a public database or journal site upon the publication of the paper to increase reproducibility of the published results and to increase public trust in science.
| S-EPMC7033918 | biostudies-literature
Project description:As members of a democratic society, individuals face complex decisions about whether to support climate change mitigation, vaccinations, genetically modified food, nanotechnology, geoengineering, and so on. To inform people's decisions and public debate, scientific experts at government agencies, nongovernmental organizations, and other organizations aim to provide understandable and scientifically accurate communication materials. Such communications aim to improve people's understanding of the decision-relevant issues, and if needed, promote behavior change. Unfortunately, existing communications sometimes fail when scientific experts lack information about what people need to know to make more informed decisions or what wording people use to describe relevant concepts. We provide an introduction for scientific experts about how to use mental models research with intended audience members to inform their communication efforts. Specifically, we describe how to conduct interviews to characterize people's decision-relevant beliefs or mental models of the topic under consideration, identify gaps and misconceptions in their knowledge, and reveal their preferred wording. We also describe methods for designing follow-up surveys with larger samples to examine the prevalence of beliefs as well as the relationships of beliefs with behaviors. Finally, we discuss how findings from these interviews and surveys can be used to design communications that effectively address gaps and misconceptions in people's mental models in wording that they understand. We present applications to different scientific domains, showing that this approach leads to communications that improve recipients' understanding and ability to make informed decisions.
| S-EPMC3752171 | biostudies-other
Project description:The Cancer Genome Atlas (TCGA) identified 4 groups of endometrial carcinomas based on an integrated genomic characterization: POLE ultramutated (POLE), microsatellite instability-high, copy number-low (CN-L), and copy number-high (CN-H). In that study, CN-H comprised all of the serous carcinoma cases and 25% of all International Federation of Gynecology and Obstetrics (FIGO) Grade 3 endometrioid carcinoma cases. In this study, 2 expert gynecologic pathologists undertook a morphologic reassessment of the FIGO Grade 3 endometrioid carcinoma subset of the TCGA study cohort, including an analysis for evidence of serous differentiation. Interobserver variability ?values are reported for the histologic evaluation of all 4 genomic clusters, and diagnostic discrepancies are discussed. Overall, there were 55 agreements, 6 disagreements, and 14 deferrals. Of the 75 cases analyzed, 6 cases had a consensus morphologic diagnosis of serous carcinoma, but only 2 of these cases had a serous carcinoma genotype, whereas the remaining 4 cases were genotypically endometrioid carcinoma. For the CN-H group, 2 of 15 cases were serous carcinoma by morphology and genotype, whereas at least 1 pathologist interpreted the remaining 13 cases as endometrioid carcinoma. The interobserver agreement rate was highest in the CN-L group (90%; ?=0.9), compared with the other genomic groups (POLE: 62%, ?=0.55; microsatellite instability-high: 78%, ?=0.74; and CN-H: 53%, ?=0.48). Our review confirms that most high-grade endometrial carcinomas diagnosed by TCGA as FIGO Grade 3 endometrioid carcinoma are indeed endometrioid carcinomas by morphology and genotype, and that the reproducibility of histologic diagnosis between pathologists varies between the TCGA-integrated genomic clusters.
| S-EPMC4934379 | biostudies-literature
Project description:Investigators from a large consortium of scientists recently performed a multi-year study in which they replicated 100 psychology experiments. Although statistically significant results were reported in 97% of the original studies, statistical significance was achieved in only 36% of the replicated studies. This article presents a reanalysis of these data based on a formal statistical model that accounts for publication bias by treating outcomes from unpublished studies as missing data, while simultaneously estimating the distribution of effect sizes for those studies that tested nonnull effects. The resulting model suggests that more than 90% of tests performed in eligible psychology experiments tested negligible effects, and that publication biases based on p-values caused the observed rates of nonreproducibility. The results of this reanalysis provide a compelling argument for both increasing the threshold required for declaring scientific discoveries and for adopting statistical summaries of evidence that account for the high proportion of tested hypotheses that are false. Supplementary materials for this article are available online.
| S-EPMC5976261 | biostudies-literature
Project description:The science surrounding vitamin D presents both challenges and opportunities. Although many uncertainties are associated with the understandings concerning vitamin D, including its physiological function, the effects of excessive intake, and its role in health, it is at the same time a major interest in the research and health communities. The approach to evaluating and interpreting the available evidence about vitamin D should be founded on the quality of the data and on the conclusions that take into account the totality of the evidence. In addition, these activities can be used to identify critical data gaps and to help structure future research. The Office of Dietary Supplements (ODS) at the National Institutes of Health has as part of its mission the goal of supporting research and dialogues for topics with uncertain data, including vitamin D. This review considers vitamin D in the context of systematically addressing the uncertainty and in identifying research needs through the filter of the work of ODS. The focus includes the role of systematic reviews, activities that encompass considerations of the totality of the evidence, and collaborative activities to clarify unknowns or to fix methodological problems, as well as a case study using the relationship between cancer and vitamin D.
| S-EPMC5748758 | biostudies-other
Project description:Effective communication is a requisite skill for scientists. However, formalized training in this area is often unavailable for members of the scientific community. As one approach to combat this problem, the American Society for Biochemistry and Molecular Biology (ASBMB) developed The Art of Science Communication, an eight-week-long online course that provides facilitated instruction on how to communicate science in an oral format. The course is offered three times a year, and as of December 2017, nearly 200 individuals from all career stages have taken part in it. The course completion rate is currently 60%, a rate three to five times as high as the average for similar Massive Open Online Courses (MOOCs). Participants have indicated that taking the course has improved their ability to communicate about their research, and that the skills and lessons learned have benefited them professionally. Moving forward, we are examining approaches that will help us improve the course and expand its reach throughout the scientific community. This article details the development of the course and examines the role and potential of such training within the larger scientific community.
| S-EPMC5969408 | biostudies-literature
Project description:BackgroundTreating cancer depends in part on identifying the mutations driving each patient's disease. Many clinical laboratories are adopting high-throughput sequencing for assaying patients' tumours, applying targeted panels to formalin-fixed paraffin-embedded tumour tissues to detect clinically-relevant mutations. While there have been some benchmarking and best practices studies of this scenario, much variant calling work focuses on whole-genome or whole-exome studies, with fresh or fresh-frozen tissue. Thus, definitive guidance on best choices for sequencing platforms, sequencing strategies, and variant calling for clinical variant detection is still being developed.MethodsBecause ground truth for clinical specimens is rarely known, we used the well-characterized Coriell cell lines GM12878 and GM12877 to generate data. We prepared samples to mimic as closely as possible clinical biopsies, including formalin fixation and paraffin embedding. We evaluated two well-known targeted sequencing panels, Illumina's TruSight 170 hybrid-capture panel and the amplification-based Oncomine Focus panel. Sequencing was performed on an Illumina NextSeq500 and an Ion Torrent PGM respectively. We performed multiple replicates of each assay, to test reproducibility. Finally, we applied four different freely-available somatic single-nucleotide variant (SNV) callers to the data, along with the vendor-recommended callers for each sequencing platform.ResultsWe did not observe major differences in variant calling success within the regions that each panel covers, but there were substantial differences between callers. All had high sensitivity for true SNVs, but numerous and non-overlapping false positives. Overriding certain default parameters to make them consistent between callers substantially reduced discrepancies, but still resulted in high false positive rates. Intersecting results from multiple replicates or from different variant callers eliminated most false positives, while maintaining sensitivity.ConclusionsReproducibility and accuracy of targeted clinical sequencing results depend less on sequencing platform and panel than on variability between replicates and downstream bioinformatics. Differences in variant callers' default parameters are a greater influence on algorithm disagreement than other differences between the algorithms. Contrary to typical clinical practice, we recommend employing multiple variant calling pipelines and/or analyzing replicate samples, as this greatly decreases false positive calls.
| S-EPMC7560075 | biostudies-literature
Project description:Currently data-dependent acquisition (DDA) is the method of choice for mass spectrometry-based proteomics discovery experiments, but data-independent acquisition (DIA) is steadily becoming more important. One of the most important requirements to perform a DIA analysis is the availability of suitable spectral libraries for peptide identification and quantification. Several studies were performed addressing the evaluation of spectral library performance for protein identification in DIA measurements. But so far only few experiments estimate the effect of these libraries on the quantitative level.In this work we created a gold standard spike-in sample set with known contents and ratios of proteins in a complex protein matrix that allowed a detailed comparison of DIA quantification data obtained with different spectral library approaches. We used in-house generated sample-specific spectral libraries created using varying sample preparation approaches and repeated DDA measurement. In addition, two different search engines were tested for protein identification from DDA data and subsequent library generation. In total, eight different spectral libraries were generated, and the quantification results compared with a library free method, as well as a default DDA analysis. Not only the number of identifications on peptide and protein level in the spectral libraries and the corresponding DIA analysis results was inspected, but also the number of expected and identified differentially abundant protein groups and their ratios.We found, that while libraries of prefractionated samples were generally larger, there was no significant increase in DIA identifications compared with repetitive non-fractionated measurements. Furthermore, we show that the accuracy of the quantification is strongly dependent on the applied spectral library and whether the quantification is based on peptide or protein level. Overall, the reproducibility and accuracy of DIA quantification is superior to DDA in all applied approaches.Data has been deposited to the ProteomeXchange repository with identifiers PXD012986, PXD012987, PXD012988 and PXD014956.
| S-EPMC6944235 | biostudies-literature
Project description:We live in an increasingly data-driven world, where high-throughput sequencing and mass spectrometry platforms are transforming biology into an information science. This has shifted major challenges in biological research from data generation and processing to interpretation and knowledge translation. However, postsecondary training in bioinformatics, or more generally data science for life scientists, lags behind current demand. In particular, development of accessible, undergraduate data science curricula has the potential to improve research and learning outcomes as well as better prepare students in the life sciences to thrive in public and private sector careers. Here, we describe the Experiential Data science for Undergraduate Cross-Disciplinary Education (EDUCE) initiative, which aims to progressively build data science competency across several years of integrated practice. Through EDUCE, students complete data science modules integrated into required and elective courses augmented with coordinated cocurricular activities. The EDUCE initiative draws on a community of practice consisting of teaching assistants (TAs), postdocs, instructors, and research faculty from multiple disciplines to overcome several reported barriers to data science for life scientists, including instructor capacity, student prior knowledge, and relevance to discipline-specific problems. Preliminary survey results indicate that even a single module improves student self-reported interest and/or experience in bioinformatics and computer science. Thus, EDUCE provides a flexible and extensible active learning framework for integration of data science curriculum into undergraduate courses and programs across the life sciences.
| S-EPMC7906378 | biostudies-literature
Project description:BACKGROUND:Research has generated valuable knowledge in identifying, understanding, and intervening to address inequities in the delivery of healthcare, yet these inequities persist. The best available interventions, programs and policies designed to address inequities in healthcare are not being adopted in routine practice settings. Implementation science can help address this gap by studying the factors, processes, and strategies at multiple levels of a system of care that influence the uptake, use, and the sustainability of these programs for vulnerable populations. We propose that an equity lens can help integrate the fields of implementation science and research that focuses on inequities in healthcare delivery. MAIN TEXT:Using Proctor et al.' (12) framework as a case study, we reframed five elements of implementation science to study inequities in healthcare. These elements include: 1) focus on reach from the very beginning; 2) design and select interventions for vulnerable populations and low-resource communities with implementation in mind; 3) implement what works and develop implementation strategies that can help reduce inequities in care; 4) develop the science of adaptations; and 5) use an equity lens for implementation outcomes. CONCLUSIONS:The goal of this paper is to continue the dialogue on how to critically infuse an equity approach in implementation studies to proactively address healthcare inequities in historically underserved populations. Our examples provide ways to operationalize how we can blend implementation science and healthcare inequities research.
| S-EPMC7069050 | biostudies-literature