Dataset Information

A big data approach to metagenomics for all-food-sequencing.

ABSTRACT: BACKGROUND:All-Food-Sequencing (AFS) is an untargeted metagenomic sequencing method that allows for the detection and quantification of food ingredients including animals, plants, and microbiota. While this approach avoids some of the shortcomings of targeted PCR-based methods, it requires the comparison of sequence reads to large collections of reference genomes. The steadily increasing amount of available reference genomes establishes the need for efficient big data approaches. RESULTS:We introduce an alignment-free k-mer based method for detection and quantification of species composition in food and other complex biological matters. It is orders-of-magnitude faster than our previous alignment-based AFS pipeline. In comparison to the established tools CLARK, Kraken2, and Kraken2+Bracken it is superior in terms of false-positive rate and quantification accuracy. Furthermore, the usage of an efficient database partitioning scheme allows for the processing of massive collections of reference genomes with reduced memory requirements on a workstation (AFS-MetaCache) or on a Spark-based compute cluster (MetaCacheSpark). CONCLUSIONS:We present a fast yet accurate screening method for whole genome shotgun sequencing-based biosurveillance applications such as food testing. By relying on a big data approach it can scale efficiently towards large-scale collections of complex eukaryotic and bacterial reference genomes. AFS-MetaCache and MetaCacheSpark are suitable tools for broad-scale metagenomic screening applications. They are available at https://muellan.github.io/metacache/afs.html (C++ version for a workstation) and https://github.com/jmabuin/MetaCacheSpark (Spark version for big data clusters).

SUBMITTER: Kobus R

PROVIDER: S-EPMC7069206 | biostudies-literature | 2020 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A big data approach to metagenomics for all-food-sequencing.

Kobus Robin R Abuín José M JM Müller André A Hellmann Sören Lukas SL Pichel Juan C JC Pena Tomás F TF Hildebrandt Andreas A Hankeln Thomas T Schmidt Bertil B

BMC bioinformatics 20200312 1

<h4>Background</h4>All-Food-Sequencing (AFS) is an untargeted metagenomic sequencing method that allows for the detection and quantification of food ingredients including animals, plants, and microbiota. While this approach avoids some of the shortcomings of targeted PCR-based methods, it requires the comparison of sequence reads to large collections of reference genomes. The steadily increasing amount of available reference genomes establishes the need for efficient big data approaches.<h4>Resu ...[more]

PMID: 32164527

Similar Datasets

Project description:The progress of next-generation sequencing has lead to the availability of massive data sets used by a wide range of applications in biology and medicine. This has sparked significant interest in using modern Big Data technologies to process this large amount of information in distributed memory clusters of commodity hardware. Several approaches based on solutions such as Apache Hadoop or Apache Spark, have been proposed. These solutions allow developers to focus on the problem while the need to deal with low level details, such as data distribution schemes or communication patterns among processing nodes, can be ignored. However, performance and scalability are also of high importance when dealing with increasing problems sizes, making in this way the usage of High Performance Computing (HPC) technologies such as the message passing interface (MPI) a promising alternative. Recently, MetaCacheSpark, an Apache Spark based software for detection and quantification of species composition in food samples has been proposed. This tool can be used to analyze high throughput sequencing data sets of metagenomic DNA and allows for dealing with large-scale collections of complex eukaryotic and bacterial reference genome. In this work, we propose MetaCache-MPI, a fast and memory efficient solution for computing clusters which is based on MPI instead of Apache Spark. In order to evaluate its performance a comparison is performed between the original single CPU version of MetaCache, the Spark version and the MPI version we are introducing. Results show that for 32 processes, MetaCache-MPI is 1.65× faster while consuming 48.12% of the RAM memory used by Spark for building a metagenomics database. For querying this database, also with 32 processes, the MPI version is 3.11× faster, while using 55.56% of the memory used by Spark. We conclude that the new MetaCache-MPI version is faster in both building and querying the database and uses less RAM memory, when compared with MetaCacheSpark, while keeping the accuracy of the original implementation.

Project description:Food is an important determinant of health, featuring prominently in the Sustainable Development Goals. The term "big data" is seldom used in relation to food, partly because food data are scattered across different sectors. The increasing availability of food-related data presents an opportunity to glean new insights on food and food systems. These insights may enhance the quality of products and services and improve decision-making on optimizing food availability, all to the end of producing better health. Yet, knowledge gaps remain about the unique opportunities and challenges linked to big data on food and their use in decision-making. This scoping review explored the available literature linking food with big data and decision-making, using the following research question: What is the current literature on data about food, and how are these data used in decision-making? We searched PubMed until 29 February 2020 and Embase, Web of Sciences, and the Cochrane Database of Systematic Reviews until 8 March 2020. We included studies written in English and conducted narrative analyses to identify relevant themes from included studies. Sixteen studies fulfilled our eligibility criteria, including big data analyses, modelling studies, and reviews. These studies described the added value of using big data and how evidence from big data had or can be used for decision-making, as well as challenges and opportunities for such use. The majority of the included studies examined the link between food and big data, while hypothesizing of how these insights could inform decision-making, including policies, interventions, programs, and financing. There were only two examples wherein big data on food informed decision-making directly. The review highlights several false dichotomies in how the subject is approached in the literature and the importance of context, both between and within countries, in shaping the availability and types of data that can be used as meaningful evidence to inform decision-making. This review shows the paucity of research around the intersection of food, big data, and decision-making, as well as the potential in using big data on food systems to the end of informing decisions to improve the health of populations. Future research and decision-making around health systems can benefit from examining the full spectrum of perspectives on the subject. Future research and decision-making around health systems can also employ the steadfast embrace of technology, which will potentially reduce disparities in big data availability, to the end of improving the health of populations.

Dataset Information

A big data approach to metagenomics for all-food-sequencing.

Publications

A big data approach to metagenomics for all-food-sequencing.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets