Dataset Information

Developing a modern data workflow for regularly updated data.

ABSTRACT: Over the past decade, biology has undergone a data revolution in how researchers collect data and the amount of data being collected. An emerging challenge that has received limited attention in biology is managing, working with, and providing access to data under continual active collection. Regularly updated data present unique challenges in quality assurance and control, data publication, archiving, and reproducibility. We developed a workflow for a long-term ecological study that addresses many of the challenges associated with managing this type of data. We do this by leveraging existing tools to 1) perform quality assurance and control; 2) import, restructure, version, and archive data; 3) rapidly publish new data in ways that ensure appropriate credit to all contributors; and 4) automate most steps in the data pipeline to reduce the time and effort required by researchers. The workflow leverages tools from software development, including version control and continuous integration, to create a modern data management system that automates the pipeline.

SUBMITTER: Yenni GM

PROVIDER: S-EPMC6368360 | biostudies-literature | 2019 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Developing a modern data workflow for regularly updated data.

Yenni Glenda M GM Christensen Erica M EM Bledsoe Ellen K EK Supp Sarah R SR Diaz Renata M RM White Ethan P EP Ernest S K Morgan SKM

PLoS biology 20190129 1

Over the past decade, biology has undergone a data revolution in how researchers collect data and the amount of data being collected. An emerging challenge that has received limited attention in biology is managing, working with, and providing access to data under continual active collection. Regularly updated data present unique challenges in quality assurance and control, data publication, archiving, and reproducibility. We developed a workflow for a long-term ecological study that addresses m ...[more]

PMID: 30695030

Similar Datasets

Project description:At the Berlin INQUA Congress (1995) a working group, European Late Pleistocene Isotopic Stages 2 & 3: Humans, Their Ecology & Cultural Adaptations, was established under the direction of J. Renault-Miskovsky (Institut de Paléontologie humaine, Paris). One of the objectives was building a database of the human occupation of Europe during this period. The database has been enlarged and now includes Lower, Middle and Upper Palaeolithic sites connecting them to their environmental conditions and the available chronometric dating. From version 14 on, only sites with chronometric data were included. In this database we have collected the available radiometric data from literature and from other more restricted databases. We try to incorporate newly published chronometric dates, collected from all kind of available publications. Only dates older than 9500 uncalibrated BP, correlated with a "cultural" level obtained by scientific excavations of European (Asian Russian Federation included) Palaeolithic sites, have been included. The dates are complemented with information related to cultural remains, stratigraphic, sedimentologic and palaeontologic information within a Microsoft Access database. For colleagues mainly interested in a list of all chronometric dates an Microsoft Excel list (with no details) is available (Tab. 1). A file, containing all sites with known coordinates, that can be opened for immediate use in Google Earth is available as a *.kmz file. It will give the possibility to introduce (by file open) in Google Earth the whole site list in "My Places". The database, version 27 (first version was available in 2002), contains now 13,202 site forms, (most of them with their geographical coordinates), comprising 17,022 radiometric data: Conv. 14C and AMS 14C (13,144 items), TL (678 items), OSL (1050 items), ESR, Th/U and AAR (2150 items) from the Lower, Middle and Upper Palaeolithic. All 14C dates are conventional dates BP. This improved version 27 replaces the older version 26.

Project description:Background: Molecular breeding is an essential tool for accelerating genetic gain in crop improvement towards meeting the need to feed an ever-growing world population. Establishing low-cost, flexible genotyping platforms in small, public and regional laboratories can stimulate the application of molecular breeding in developing countries. These laboratories can serve plant breeding projects requiring low- to medium-density markers for marker-assisted selection (MAS) and quality control (QC) activities. Methods: We performed two QC and MAS experiments consisting of 637 maize lines, using an optimised genotyping workflow involving an in-house competitive allele-specific PCR (KASP) genotyping system with an optimised sample collection, preparation, and DNA extraction and quantitation process. A smaller volume of leaf-disc size plant samples was collected directly in 96-well plates for DNA extraction, using a slightly modified CTAB-based DArT DNA extraction protocol. DNA quality and quantity analyses were performed using a microplate reader, and the KASP genotyping and data analysis was performed in our laboratory. Results: Applying the optimized genotyping workflow expedited the QC and MAS experiments from over five weeks (when outsourcing) to two weeks and eliminated the shipping cost. Using a set of 28 KASP single nucleotide polymorphisms (SNPs) validated for maize, the QC experiment revealed the genetic identity of four maize varieties taken from five seed sources. Another set of 10 KASP SNPs was sufficient in verifying the parentage of 390 F 1 lines. The KASP-based MAS was successfully applied to a maize pro-vitamin A (PVA) breeding program and for introgressing the aflatoxin resistance gene into elite tropical maize lines. Conclusion: This improved workflow has helped accelerate maize improvement activities of IITA's Maize Improvement Program and facilitated DNA fingerprinting for tracking improved crop varieties. National Agricultural Research Systems (NARS) in developing countries can adopt this workflow to fast-track molecular marker-based genotyping for crop improvement.

Dataset Information

Developing a modern data workflow for regularly updated data.

Publications

Developing a modern data workflow for regularly updated data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets