Unknown

Dataset Information

0

Load-Balancing Parallel Relational Algebra


ABSTRACT: Relational algebra (RA) comprises a basis of important operations, sufficient to power state-of-the-art reasoning engines for Datalog and related logic-programming languages. Parallel RA implementations can thus play a significant role in extracting parallelism inherent in a wide variety of analytic problems. In general, bottom-up logical inference can be implemented as fixed-point iteration over RA kernels; relations dynamically accumulate new tuples of information according to a set of rules until no new tuples can be discovered from previously inferred tuples and relevant rules (RA kernels). While this strategy has been quite successful in single-node contexts, it poses unique challenges when distributed over many-node, networked clusters—especially regarding how the work-load is balanced across available compute resources. In this paper, we identify two fundamental kinds of load imbalance and present a strategy to address each. We investigate both spatial load imbalance—imbalance across each relation (across compute nodes)—and temporal load imbalance–imbalance in tuples produced across fixed-point iterations. For spatial balancing, we implement refinement and consolidation procedures. For temporal balancing, we implement a technique that permits the residual workload from a busy iteration to roll over to a new iteration. In sum, these techniques permit fully dynamic load-balancing of relational algebra that is robust to changes across time.

SUBMITTER: Sadayappan P 

PROVIDER: S-EPMC7295339 | biostudies-literature | 2020 May

REPOSITORIES: biostudies-literature

Similar Datasets

| S-EPMC7826171 | biostudies-literature
| S-EPMC7523073 | biostudies-literature
| S-EPMC4469845 | biostudies-other
| S-EPMC7170225 | biostudies-literature
| S-EPMC6249390 | biostudies-literature
| S-EPMC3683287 | biostudies-literature
| S-EPMC8444070 | biostudies-literature
| S-EPMC4805712 | biostudies-other
| S-EPMC9879725 | biostudies-literature
| S-EPMC5461685 | biostudies-literature