Unknown

Dataset Information

0

NCBI's Virus Discovery Hackathon: Engaging Research Communities to Identify Cloud Infrastructure Requirements.


ABSTRACT: A wealth of viral data sits untapped in publicly available metagenomic data sets when it might be extracted to create a usable index for the virological research community. We hypothesized that work of this complexity and scale could be done in a hackathon setting. Ten teams comprised of over 40 participants from six countries, assembled to create a crowd-sourced set of analysis and processing pipelines for a complex biological data set in a three-day event on the San Diego State University campus starting 9 January 2019. Prior to the hackathon, 141,676 metagenomic data sets from the National Center for Biotechnology Information (NCBI) Sequence Read Archive (SRA) were pre-assembled into contiguous assemblies (contigs) by NCBI staff. During the hackathon, a subset consisting of 2953 SRA data sets (approximately 55 million contigs) was selected, which were further filtered for a minimal length of 1 kb. This resulted in 4.2 million (Mio) contigs, which were aligned using BLAST against all known virus genomes, phylogenetically clustered and assigned metadata. Out of the 4.2 Mio contigs, 360,000 contigs were labeled with domains and an additional subset containing 4400 contigs was screened for virus or virus-like genes. The work yielded valuable insights into both SRA data and the cloud infrastructure required to support such efforts, revealing analysis bottlenecks and possible workarounds thereof. Mainly: (i) Conservative assemblies of SRA data improves initial analysis steps; (ii) existing bioinformatic software with weak multithreading/multicore support can be elevated by wrapper scripts to use all cores within a computing node; (iii) redesigning existing bioinformatic algorithms for a cloud infrastructure to facilitate its use for a wider audience; and (iv) a cloud infrastructure allows a diverse group of researchers to collaborate effectively. The scientific findings will be extended during a follow-up event. Here, we present the applied workflows, initial results, and lessons learned from the hackathon.

SUBMITTER: Connor R 

PROVIDER: S-EPMC6771016 | biostudies-literature | 2019 Sep

REPOSITORIES: biostudies-literature

altmetric image

Publications

NCBI's Virus Discovery Hackathon: Engaging Research Communities to Identify Cloud Infrastructure Requirements.

Connor Ryan R   Brister Rodney R   Buchmann Jan P JP   Deboutte Ward W   Edwards Rob R   Martí-Carreras Joan J   Tisza Mike M   Zalunin Vadim V   Andrade-Martínez Juan J   Cantu Adrian A   D'Amour Michael M   Efremov Alexandre A   Fleischmann Lydia L   Forero-Junco Laura L   Garmaeva Sanzhima S   Giluso Melissa M   Glickman Cody C   Henderson Margaret M   Kellman Benjamin B   Kristensen David D   Leubsdorf Carl C   Levi Kyle K   Levi Shane S   Pakala Suman S   Peddu Vikas V   Ponsero Alise A   Ribeiro Eldred E   Roy Farrah F   Rutter Lindsay L   Saha Surya S   Shakya Migun M   Shean Ryan R   Miller Matthew M   Tully Benjamin B   Turkington Christopher C   Youens-Clark Ken K   Vanmechelen Bert B   Busby Ben B  

Genes 20190916 9


A wealth of viral data sits untapped in publicly available metagenomic data sets when it might be extracted to create a usable index for the virological research community. We hypothesized that work of this complexity and scale could be done in a hackathon setting. Ten teams comprised of over 40 participants from six countries, assembled to create a crowd-sourced set of analysis and processing pipelines for a complex biological data set in a three-day event on the San Diego State University camp  ...[more]

Similar Datasets

| S-EPMC5072118 | biostudies-literature
| S-EPMC8218276 | biostudies-literature
| S-EPMC5835193 | biostudies-literature
| S-EPMC5547731 | biostudies-other
| S-EPMC7764237 | biostudies-literature
| S-EPMC8656837 | biostudies-literature
| S-EPMC8296813 | biostudies-literature
| S-EPMC7072042 | biostudies-literature
| S-EPMC6531853 | biostudies-literature
| S-EPMC3078622 | biostudies-literature