Clubber: removing the bioinformatics bottleneck in big data analyses.
Ontology highlight
ABSTRACT:
SUBMITTER: Miller M
PROVIDER: S-EPMC5929469 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:This review examines two important aspects that are central to modern big data bioinformatics analysis - software scalability and validity. We argue that not only are the issues of scalability and validation common to all big data bioinformatics analyses, they can be tackled by conceptually related methodological approaches, namely divide-and-conquer (scalability) and multiple executions (validation). Scalability is defined as the ability for a program to scale based on workload. It has always been an important consideration when developing bioinformatics algorithms and programs. Nonetheless the surge of volume and variety of biological and biomedical data has posed new challenges. We discuss how modern cloud computing and big data programming frameworks such as MapReduce and Spark are being used to effectively implement divide-and-conquer in a distributed computing environment. Validation of software is another important issue in big data bioinformatics that is often ignored. Software validation is the process of determining whether the program under test fulfils the task for which it was designed. Determining the correctness of the computational output of big data bioinformatics software is especially difficult due to the large input space and complex algorithms involved. We discuss how state-of-the-art software testing techniques that are based on the idea of multiple executions, such as metamorphic testing, can be used to implement an effective bioinformatics quality assurance strategy. We hope this review will raise awareness of these critical issues in bioinformatics.
| S-EPMC5537105 | biostudies-other
Project description:Lyme disease is caused by the bacteria borrelia burgdorferi and is spread primarily through the bite of a tick. There is considerable uncertainty in the medical community regarding the best approach to treating patients with Lyme disease who do not respond fully to short-term antibiotic therapy. These patients have persistent Lyme disease symptoms resulting from lack of treatment, under-treatment, or lack of response to their antibiotic treatment protocol. In the past, treatment trials have used small restrictive samples and relied on average treatment effects as their measure of success and produced conflicting results. To provide individualized care, clinicians need information that reflects their patient population. Today, we have the ability to analyze large data bases, including patient registries, that reflect the broader range of patients more typically seen in clinical practice. This allows us to examine treatment variation within the sample and identify groups of patients that are most responsive to treatment. Using patient-reported outcome data from the MyLymeData online patient registry, we show that sub-group analysis techniques can unmask valuable information that is hidden if averages alone are used. In our analysis, this approach revealed treatment effectiveness for up to a third of patients with Lyme disease. This study is important because it can help open the door to more individualized patient care using patient-centered outcomes and real-world evidence.
| S-EPMC6316052 | biostudies-literature
Project description:The explosion of the data both in the biomedical research and in the healthcare systems demands urgent solutions. In particular, the research in omics sciences is moving from a hypothesis-driven to a data-driven approach. Healthcare is additionally always asking for a tighter integration with biomedical data in order to promote personalized medicine and to provide better treatments. Efficient analysis and interpretation of Big Data opens new avenues to explore molecular biology, new questions to ask about physiological and pathological states, and new ways to answer these open issues. Such analyses lead to better understanding of diseases and development of better and personalized diagnostics and therapeutics. However, such progresses are directly related to the availability of new solutions to deal with this huge amount of information. New paradigms are needed to store and access data, for its annotation and integration and finally for inferring knowledge and making it available to researchers. Bioinformatics can be viewed as the "glue" for all these processes. A clear awareness of present high performance computing (HPC) solutions in bioinformatics, Big Data analysis paradigms for computational biology, and the issues that are still open in the biomedical and healthcare fields represent the starting point to win this challenge.
| S-EPMC4165507 | biostudies-literature
Project description:We present a software tool, called cMatch, to reconstruct and identify synthetic genetic constructs from their sequences, or a set of sub-sequences-based on two practical pieces of information: their modular structure, and libraries of components. Although developed for combinatorial pathway engineering problems and addressing their quality control (QC) bottleneck, cMatch is not restricted to these applications. QC takes place post assembly, transformation and growth. It has a simple goal, to verify that the genetic material contained in a cell matches what was intended to be built - and when it is not the case, to locate the discrepancies and estimate their severity. In terms of reproducibility/reliability, the QC step is crucial. Failure at this step requires repetition of the construction and/or sequencing steps. When performed manually or semi-manually QC is an extremely time-consuming, error prone process, which scales very poorly with the number of constructs and their complexity. To make QC frictionless and more reliable, cMatch performs an operation we have called "construct-matching" and automates it. Construct-matching is more thorough than simple sequence-matching, as it matches at the functional level-and quantifies the matching at the individual component level and across the whole construct. Two algorithms (called CM_1 and CM_2) are presented. They differ according to the nature of their inputs. CM_1 is the core algorithm for construct-matching and is to be used when input sequences are long enough to cover constructs in their entirety (e.g., obtained with methods such as next generation sequencing). CM_2 is an extension designed to deal with shorter data (e.g., obtained with Sanger sequencing), and that need recombining. Both algorithms are shown to yield accurate construct-matching in a few minutes (even on hardware with limited processing power), together with a set of metrics that can be used to improve the robustness of the decision-making process. To ensure reliability and reproducibility, cMatch builds on the highly validated pairwise-matching Smith-Waterman algorithm. All the tests presented have been conducted on synthetic data for challenging, yet realistic constructs - and on real data gathered during studies on a metabolic engineering example (lycopene production).
| S-EPMC8784771 | biostudies-literature
Project description:Combined analysis of multiple, large datasets is a common objective in the health- and biosciences. Existing methods tend to require researchers to physically bring data together in one place or follow an analysis plan and share results. Developed over the last 10 years, the DataSHIELD platform is a collection of R packages that reduce the challenges of these methods. These include ethico-legal constraints which limit researchers' ability to physically bring data together and the analytical inflexibility associated with conventional approaches to sharing results. The key feature of DataSHIELD is that data from research studies stay on a server at each of the institutions that are responsible for the data. Each institution has control over who can access their data. The platform allows an analyst to pass commands to each server and the analyst receives results that do not disclose the individual-level data of any study participants. DataSHIELD uses Opal which is a data integration system used by epidemiological studies and developed by the OBiBa open source project in the domain of bioinformatics. However, until now the analysis of big data with DataSHIELD has been limited by the storage formats available in Opal and the analysis capabilities available in the DataSHIELD R packages. We present a new architecture ("resources") for DataSHIELD and Opal to allow large, complex datasets to be used at their original location, in their original format and with external computing facilities. We provide some real big data analysis examples in genomics and geospatial projects. For genomic data analyses, we also illustrate how to extend the resources concept to address specific big data infrastructures such as GA4GH or EGA, and make use of shell commands. Our new infrastructure will help researchers to perform data analyses in a privacy-protected way from existing data sharing initiatives or projects. To help researchers use this framework, we describe selected packages and present an online book (https://isglobal-brge.github.io/resource_bookdown).
| S-EPMC8034722 | biostudies-literature
Project description:MotivationIn this paper, we present an open source package with the latest release of Evolutionary-based BIClustering (EBIC), a next-generation biclustering algorithm for mining genetic data. The major contribution of this paper is adding a full support for multiple graphics processing units (GPUs) support, which makes it possible to run efficiently large genomic data mining analyses. Multiple enhancements to the first release of the algorithm include integration with R and Bioconductor, and an option to exclude missing values from the analysis.ResultsEvolutionary-based BIClustering was applied to datasets of different sizes, including a large DNA methylation dataset with 436 444 rows. For the largest dataset we observed over 6.6-fold speedup in computation time on a cluster of eight GPUs compared to running the method on a single GPU. This proves high scalability of the method.Availability and implementationThe latest version of EBIC could be downloaded from http://github.com/EpistasisLab/ebic. Installation and usage instructions are also available online.Supplementary informationSupplementary data are available at Bioinformatics online.
| S-EPMC6736067 | biostudies-literature
Project description:IntroductionCoronavirus disease 2019 (COVID-19) has led to radical changes in social distancing awareness and affected social relationships. Owing to large-scale lockdown, home quarantine and social distancing requirements, it was anticipated that sexual activities would be severely impacted. However, retrospective self-report studies showed that pornography use and autoerotism increased during the pandemic.AimThis study used big-data databases available on the Internet to investigate factors that modulated pornography use during the pandemic.MethodsDaily relative search volume (RSV) data from Google Trends for the period from 24 February 2020 to 13 July 2020 were extracted. Pornhub traffic data were extracted from the Pornhub Insights website, for the period from 24 February 2020 to 13 July 2020. The parameter was defined as 'percent change in traffic compared to an average day in 2019'. The number of daily new cases of COVID-19 was extracted from the database on Our World in Data.Outcome measuresThe normality of the data was examined using the Shapiro-Wilk test. All variables included in this study were non-normally distributed. Therefore, non-parametric tests or parametric tests with bootstrapping were adopted where appropriate.ResultsAccording to Google Trends, the RSV for 'pornography' increased after late March 2020, which is close to the date when the World Health Organization declared COVID-19 a global pandemic. The number of daily new cases of COVID-19 was positively correlated with the traffic of Pornhub, a popular pornography website, and the RSV for 'pornography'. Moderation analysis demonstrated a significant main effect of daily new cases of COVID-19 and the RSV for 'social distancing' in predicting Pornhub traffic/RSV for 'pornography'. Furthermore, the RSV for 'social distancing' significantly moderated the relationship between daily new cases and Pornhub traffic/RSV for 'pornography'. A stronger COVID-pornography use association was observed with increased social distancing awareness.ConclusionIncreased pornography consumption during the pandemic was observed, and it was associated with the severity of the pandemic. Social distancing awareness could be a key factor influencing interest in and use of pornography. Further studies on the changes in sexual desire and birth-rate control are worthwhile because long-term public health may be affected by the changes in sexual behaviour during the pandemic.
| S-EPMC8691607 | biostudies-literature
Project description:Bioinformatics has become an indispensable part of life science over the past 2 decades. However, bioinformatics education is not well integrated at the undergraduate level, especially in liberal arts colleges and regional universities in the United States. One significant obstacle pointed out by the Network for Integrating Bioinformatics into Life Sciences Education is the lack of faculty in the bioinformatics area. Most current life science professors did not acquire bioinformatics analysis skills during their own training. Consequently, a great number of undergraduate and graduate students do not get the chance to learn bioinformatics or computational biology skills within a structured curriculum during their education. To address this gap, we developed a module-based, week-long short course to train small college and regional university professors with essential bioinformatics skills. The bioinformatics modules were built to be adapted by the professor-trainees afterward and used in their own classes. All the course materials can be accessed at https://github.com/TheJacksonLaboratory/JAXBD2K-ShortCourse.
| S-EPMC6563947 | biostudies-literature
Project description:Secondary analyses of survey data collected from large probability samples of persons or establishments further scientific progress in many fields. The complex design features of these samples improve data collection efficiency, but also require analysts to account for these features when conducting analysis. Unfortunately, many secondary analysts from fields outside of statistics, biostatistics, and survey methodology do not have adequate training in this area, and as a result may apply incorrect statistical methods when analyzing these survey data sets. This in turn could lead to the publication of incorrect inferences based on the survey data that effectively negate the resources dedicated to these surveys. In this article, we build on the results of a preliminary meta-analysis of 100 peer-reviewed journal articles presenting analyses of data from a variety of national health surveys, which suggested that analytic errors may be extremely prevalent in these types of investigations. We first perform a meta-analysis of a stratified random sample of 145 additional research products analyzing survey data from the Scientists and Engineers Statistical Data System (SESTAT), which describes features of the U.S. Science and Engineering workforce, and examine trends in the prevalence of analytic error across the decades used to stratify the sample. We once again find that analytic errors appear to be quite prevalent in these studies. Next, we present several example analyses of real SESTAT data, and demonstrate that a failure to perform these analyses correctly can result in substantially biased estimates with standard errors that do not adequately reflect complex sample design features. Collectively, the results of this investigation suggest that reviewers of this type of research need to pay much closer attention to the analytic methods employed by researchers attempting to publish or present secondary analyses of survey data.
| S-EPMC4927119 | biostudies-other
Project description:The term "big data" refers broadly to large volumes of data, often gathered from several sources, that are then analyzed, for example, for predictive analytics. Combining and mining genetic data from varied sources including clinical genetic testing, for example, electronic health records, what might be termed as "recreational" genetic testing such as ancestry testing, as well as research studies, provide one type of "big data." Challenges and cautions in analyzing big data include recognizing the lack of systematic collection of the source data, the variety of assay technologies used, the potential variation in classification and interpretation of genetic variants. While advanced technologies such as microarrays and, more recently, next-generation sequencing, that enable testing an individual's DNA for thousands of genes and variants simultaneously are briefly discussed, attention is focused more closely on challenges to analysis of the massive data generated by these genomic technologies. The main theme of this review is to evaluate challenges associated with big data in general and specifically to bring the sophisticated technology of genetic/genomic testing down to the individual level, keeping in mind the human aspect of the data source and considering where the impact of the data will be translated and applied. Considerations in this "humanizing" process include providing adequate counseling and consent for genetic testing in all settings, as well as understanding the strengths and limitations of assays and their interpretation.
| S-EPMC7082327 | biostudies-literature