Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

Comparative investigation of parallel spatial interpolation algorithms for building large-scale digital elevation models.

ABSTRACT: The building of large-scale Digital Elevation Models (DEMs) using various interpolation algorithms is one of the key issues in geographic information science. Different choices of interpolation algorithms may trigger significant differences in interpolation accuracy and computational efficiency, and a proper interpolation algorithm needs to be carefully used based on the specific characteristics of the scene of interpolation. In this paper, we comparatively investigate the performance of parallel Radial Basis Function (RBF)-based, Moving Least Square (MLS)-based, and Shepard's interpolation algorithms for building DEMs by evaluating the influence of terrain type, raw data density, and distribution patterns on the interpolation accuracy and computational efficiency. The drawn conclusions may help select a suitable interpolation algorithm in a specific scene to build large-scale DEMs.

SUBMITTER: Tu J

PROVIDER: S-EPMC7924418 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

Comparative investigation of parallel spatial interpolation algorithms for building large-scale digital elevation models.

Tu Jingzhi J Yang Guoxiang G Qi Pian P Ding Zengyu Z Mei Gang G

PeerJ. Computer science 20200302

The building of large-scale Digital Elevation Models (DEMs) using various interpolation algorithms is one of the key issues in geographic information science. Different choices of interpolation algorithms may trigger significant differences in interpolation accuracy and computational efficiency, and a proper interpolation algorithm needs to be carefully used based on the specific characteristics of the scene of interpolation. In this paper, we comparatively investigate the performance of paralle ...[more]

PMID: 33816915

Similar Datasets

Parallel algorithms for large-scale biological sequence alignment on Xeon-Phi based clusters.

Project description:BackgroundComputing alignments between two or more sequences are common operations frequently performed in computational molecular biology. The continuing growth of biological sequence databases establishes the need for their efficient parallel implementation on modern accelerators.ResultsThis paper presents new approaches to high performance biological sequence database scanning with the Smith-Waterman algorithm and the first stage of progressive multiple sequence alignment based on the ClustalW heuristic on a Xeon Phi-based compute cluster. Our approach uses a three-level parallelization scheme to take full advantage of the compute power available on this type of architecture; i.e. cluster-level data parallelism, thread-level coarse-grained parallelism, and vector-level fine-grained parallelism. Furthermore, we re-organize the sequence datasets and use Xeon Phi shuffle operations to improve I/O efficiency.ConclusionsEvaluations show that our method achieves a peak overall performance up to 220 GCUPS for scanning real protein sequence databanks on a single node consisting of two Intel E5-2620 CPUs and two Intel Xeon Phi 7110P cards. It also exhibits good scalability in terms of sequence length and size, and number of compute nodes for both database scanning and multiple sequence alignment. Furthermore, the achieved performance is highly competitive in comparison to optimized Xeon Phi and GPU implementations. Our implementation is available at https://github.com/turbo0628/LSDBS-mpi .

| S-EPMC4959381 | biostudies-literature

Large-scale digital forensic investigation for Windows registry on Apache Spark.

Project description:In this study, we investigate large-scale digital forensic investigation on Apache Spark using a Windows registry. Because the Windows registry depends on the system on which it operates, the existing forensic methods on the Windows registry have been targeted on the Windows registry in a single system. However, it is a critical issue to analyze large-scale registry data collected from several Windows systems because it allows us to detect suspiciously changed data by comparing the Windows registry in multiple systems. To this end, we devise distributed algorithms to analyze large-scale registry data collected from multiple Windows systems on the Apache Spark framework. First, we define three main scenarios in which we classify the existing registry forensic studies into them. Second, we propose an algorithm to load the Windows registry into the Hadoop distributed file system (HDFS) for subsequent forensics. Third, we propose a distributed algorithm for each defined forensic scenario using Apache Spark operations. Through extensive experiments using eight nodes in an actual distributed environment, we demonstrate that the proposed method can perform forensics efficiently on large-scale registry data. Specifically, we perform forensics on 1.52 GB of Windows registry data collected from four computers and show that the proposed algorithms can reduce the processing time by up to approximately 3.31 times, as we increase the number of CPUs from 1 to 8 and the number of worker nodes from 2 to 8. Because the distributed algorithms on Apache Spark require the inherent network and MapReduce overheads, this improvement of the processing performance verifies the efficiency and scalability of the proposed algorithms.

| S-EPMC9728846 | biostudies-literature

Parallel STEPS: Large Scale Stochastic Spatial Reaction-Diffusion Simulation with High Performance Computers.

Project description:Stochastic, spatial reaction-diffusion simulations have been widely used in systems biology and computational neuroscience. However, the increasing scale and complexity of models and morphologies have exceeded the capacity of any serial implementation. This led to the development of parallel solutions that benefit from the boost in performance of modern supercomputers. In this paper, we describe an MPI-based, parallel operator-splitting implementation for stochastic spatial reaction-diffusion simulations with irregular tetrahedral meshes. The performance of our implementation is first examined and analyzed with simulations of a simple model. We then demonstrate its application to real-world research by simulating the reaction-diffusion components of a published calcium burst model in both Purkinje neuron sub-branch and full dendrite morphologies. Simulation results indicate that our implementation is capable of achieving super-linear speedup for balanced loading simulations with reasonable molecule density and mesh quality. In the best scenario, a parallel simulation with 2,000 processes runs more than 3,600 times faster than its serial SSA counterpart, and achieves more than 20-fold speedup relative to parallel simulation with 100 processes. In a more realistic scenario with dynamic calcium influx and data recording, the parallel simulation with 1,000 processes and no load balancing is still 500 times faster than the conventional serial SSA simulation.

| S-EPMC5301017 | biostudies-literature

Spatial structure, parameter nonlinearity, and intelligent algorithms in constructing pedotransfer functions from large-scale soil legacy data.

Project description:Pedotransfer function (PTF) approach is a convenient way for estimating difficult-to-measure soil properties from basic soil data. Typically, PTFs are developed using a large number of samples collected from small (regional) areas for training and testing a predictive model. National soil legacy databases offer an opportunity to provide soil data for developing PTFs although legacy data are sparsely distributed covering large areas. Here, we examined the Indian soil legacy (ISL) database to select a comprehensive training dataset for estimating cation exchange capacity (CEC) as a test case in the PTF approach. Geostatistical and correlation analyses showed that legacy data entail diverse spatial and correlation structure needed in building robust PTFs. Through non-linear correlation measures and intelligent predictive algorithms, we developed a methodology to extract an efficient training dataset from the ISL data for estimating CEC with high prediction accuracy. The selected training data had comparable spatial variation and nonlinearity in parameters for training and test datasets. Thus, we identified specific indicators for constructing robust PTFs from legacy data. Our results open a new avenue to use large volume of existing soil legacy data for developing region-specific PTFs without the need for collecting new soil data.

| S-EPMC7490350 | biostudies-literature

Building digital innovation capacity at a large academic medical center.

Project description:Academic medical centers (AMCs) today prioritize digital innovation. In efforts to develop and disseminate the best technology for their institutions, challenges arise in organizational structure, cross-disciplinary collaboration, and creative and agile problem solving that are essential for successful implementation. To address these challenges, the Digital DesignLab was created at NYU Langone Health to provide structured processes for assessing and supporting the capacity for innovative digital development in our research and clinical community. Digital DesignLab is an enterprise level, multidisciplinary, digital development team that guides faculty and student innovators through a digital development "pipeline", which consists of intake, discovery, bootcamp, development. It also provides a framework for digital health innovation and dissemination at the institution. This paper describes the Digital DesignLab's creation and processes, and highlights key lessons learned to support digital health innovation at AMCs.

| S-EPMC6550180 | biostudies-literature

Ultrahigh Dimensional Variable Selection for Interpolation of Point Referenced Spatial Data: A Digital Soil Mapping Case Study.

Project description:Modern soil mapping is characterised by the need to interpolate point referenced (geostatistical) observations and the availability of large numbers of environmental characteristics for consideration as covariates to aid this interpolation. Modelling tasks of this nature also occur in other fields such as biogeography and environmental science. This analysis employs the Least Angle Regression (LAR) algorithm for fitting Least Absolute Shrinkage and Selection Operator (LASSO) penalized Multiple Linear Regressions models. This analysis demonstrates the efficiency of the LAR algorithm at selecting covariates to aid the interpolation of geostatistical soil carbon observations. Where an exhaustive search of the models that could be constructed from 800 potential covariate terms and 60 observations would be prohibitively demanding, LASSO variable selection is accomplished with trivial computational investment.

| S-EPMC5014409 | biostudies-literature

Large scale parallel identification of papillomaviruses miRNAs

Project description:Large scale parallel identification of virus-encoded microRNAs in divergent Papillomaviruses

| PRJEB25054 | ENA

Genome-scale investigation of olfactory system spatial heterogeneity.

Project description:The early olfactory system is organized in parallel, with numerous, specialized subsystems established by the modular and topographic projections of sensory inputs. While these anatomical sub-systems are in many cases demarcated by well-known marker genes, we stand to learn considerably more about their possible functional specializations from comprehensive, genome-scale descriptions of their molecular anatomy. Here, we leverage the resources of the Allen Brain Atlas (ABA)-a spatially registered compendium of gene expression for the mouse brain-to investigate the early olfactory system's genomic anatomy. We cluster thousands of genes across thousands of voxels in the ABA to derive several novel parcellations of the olfactory system, and concomitantly discover novel sets of enriched, subregion-specific genes that can serve as a starting point for future inquiry.

| S-EPMC5443560 | biostudies-literature

Puzzle Imaging: Using Large-Scale Dimensionality Reduction Algorithms for Localization.

Project description:Current high-resolution imaging techniques require an intact sample that preserves spatial relationships. We here present a novel approach, "puzzle imaging," that allows imaging a spatially scrambled sample. This technique takes many spatially disordered samples, and then pieces them back together using local properties embedded within the sample. We show that puzzle imaging can efficiently produce high-resolution images using dimensionality reduction algorithms. We demonstrate the theoretical capabilities of puzzle imaging in three biological scenarios, showing that (1) relatively precise 3-dimensional brain imaging is possible; (2) the physical structure of a neural network can often be recovered based only on the neural connectivity matrix; and (3) a chemical map could be reproduced using bacteria with chemosensitive DNA and conjugative transfer. The ability to reconstruct scrambled images promises to enable imaging based on DNA sequencing of homogenized tissue samples.

| S-EPMC4507868 | biostudies-literature

Parallel clustering algorithm for large-scale biological data sets.

Project description:BackgroundsRecent explosion of biological data brings a great challenge for the traditional clustering algorithms. With increasing scale of data sets, much larger memory and longer runtime are required for the cluster identification problems. The affinity propagation algorithm outperforms many other classical clustering algorithms and is widely applied into the biological researches. However, the time and space complexity become a great bottleneck when handling the large-scale data sets. Moreover, the similarity matrix, whose constructing procedure takes long runtime, is required before running the affinity propagation algorithm, since the algorithm clusters data sets based on the similarities between data pairs.MethodsTwo types of parallel architectures are proposed in this paper to accelerate the similarity matrix constructing procedure and the affinity propagation algorithm. The memory-shared architecture is used to construct the similarity matrix, and the distributed system is taken for the affinity propagation algorithm, because of its large memory size and great computing capacity. An appropriate way of data partition and reduction is designed in our method, in order to minimize the global communication cost among processes.ResultA speedup of 100 is gained with 128 cores. The runtime is reduced from serval hours to a few seconds, which indicates that parallel algorithm is capable of handling large-scale data sets effectively. The parallel affinity propagation also achieves a good performance when clustering large-scale gene data (microarray) and detecting families in large protein superfamilies.

| S-EPMC3976248 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data