Random access with a distributed Bitmap Join Index for Star Joins.
Ontology highlight
ABSTRACT:
SUBMITTER: Brito JJ
PROVIDER: S-EPMC7030998 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:We consider running-time optimization for band-joins in a distributed system, e.g., the cloud. To balance load across worker machines, input has to be partitioned, which causes duplication. We explore how to resolve this tension between maximum load per worker and input duplication for band-joins between two relations. Previous work suffered from high optimization cost or considered partitionings that were too restricted (resulting in suboptimal join performance). Our main insight is that recursive partitioning of the join-attribute space with the appropriate split scoring measure can achieve both low optimization cost and low join cost. It is the first approach that is not only effective for one-dimensional band-joins but also for joins on multiple attributes. Experiments indicate that our method is able to find partitionings that are within 10% of the lower bound for both maximum load per worker and input duplication for a broad range of settings, significantly improving over previous work.
| S-EPMC7872589 | biostudies-literature
Project description:In big data area a significant challenge about string similarity join is to find all similar pairs more efficiently. In this paper, we propose a parallel processing framework for efficient string similarity join. First, the input is split into some disjoint small subsets according to the joint frequency distribution and the interval distribution of strings. Then the filter-verification strategy is adopted in the computation of string similarity for each subset so that the number of candidate pairs is reduced before an effective pruning strategy is used to improve the performance. Finally, the operation of string join is executed in parallel. Para-Join algorithm based on the multi-threading technique is proposed to implement the framework in a multi-core system while Pada-Join algorithm based on Spark platform is proposed to implement the framework in a cluster system. We prove that Para-Join and Pada-Join cannot only avoid reduplicate computation but also ensure the completeness of the result. Experimental results show that Para-Join can achieve high efficiency and significantly outperform than state-of-the-art approaches, meanwhile, Pada-Join can work on large datasets.
| S-EPMC5344375 | biostudies-other
Project description:We introduce a random-access parallel (RAP) imaging modality that uses a novel design inspired by a Newtonian telescope to image multiple spatially separated samples without moving parts or robotics. This scheme enables near-simultaneous image capture of multiple petri dishes and random-access imaging with sub-millisecond switching times at the full resolution of the camera. This enables the RAP system to capture long-duration records from different samples in parallel, which is not possible using conventional automated microscopes. The system is demonstrated by continuously imaging multiple cardiac monolayer and Caenorhabditis elegans preparations.
| S-EPMC7843131 | biostudies-literature
Project description:Random distributed feedback fibre lasers belong to the class of random lasers, where the feedback is provided by amplified Rayleigh scattering on sub-micron refractive index inhomogenities randomly distributed over the fibre length. Despite the elastic nature of Rayleigh scattering, the feedback mechanism has been insofar deemed incoherent, which corresponds to the commonly observed smooth generation spectra. Here, using a real-time spectral measurement technique based on a scanning Fabry-Pérot interferometer, we observe long-living narrowband components in the random fibre laser's spectrum. Statistical analysis of the ∼104 single-scan spectra reveals a preferential interspacing for the components and their anticorrelation in intensities. Furthermore, using mutual information analysis, we confirm the existence of nonlinear correlations between different parts of the random fibre laser spectra. The existence of such narrowband spectral components, together with their observed correlations, establishes a long-missing parallel between the fields of random fibre lasers and conventional random lasers.
| S-EPMC5454388 | biostudies-literature
Project description:Magnetic random access memory schemes employing magnetoelectric coupling to write binary information promise outstanding energy efficiency. We propose and demonstrate a purely antiferromagnetic magnetoelectric random access memory (AF-MERAM) that offers a remarkable 50-fold reduction of the writing threshold compared with ferromagnet-based counterparts, is robust against magnetic disturbances and exhibits no ferromagnetic hysteresis losses. Using the magnetoelectric antiferromagnet Cr2O3, we demonstrate reliable isothermal switching via gate voltage pulses and all-electric readout at room temperature. As no ferromagnetic component is present in the system, the writing magnetic field does not need to be pulsed for readout, allowing permanent magnets to be used. Based on our prototypes, we construct a comprehensive model of the magnetoelectric selection mechanisms in thin films of magnetoelectric antiferromagnets, revealing misfit induced ferrimagnetism as an important factor. Beyond memory applications, the AF-MERAM concept introduces a general all-electric interface for antiferromagnets and should find wide applicability in antiferromagnetic spintronics.
| S-EPMC5216083 | biostudies-other
Project description: Not available
| S-EPMC8373264 | biostudies-literature
Project description:We describe the first DNA-based storage architecture that enables random access to data blocks and rewriting of information stored at arbitrary locations within the blocks. The newly developed architecture overcomes drawbacks of existing read-only methods that require decoding the whole file in order to read one data fragment. Our system is based on new constrained coding techniques and accompanying DNA editing methods that ensure data reliability, specificity and sensitivity of access, and at the same time provide exceptionally high data storage capacity. As a proof of concept, we encoded parts of the Wikipedia pages of six universities in the USA, and selected and edited parts of the text written in DNA corresponding to three of these schools. The results suggest that DNA is a versatile media suitable for both ultrahigh density archival and rewritable storage applications.
| S-EPMC4585656 | biostudies-literature
Project description:Due to the rapid cost decline of synthesizing and sequencing deoxyribonucleic acid (DNA), high information density, and its durability of up to centuries, utilizing DNA as an information storage medium has received the attention of many scientists. State-of-the-art DNA storage systems exploit the high capacity of DNA and enable random access (predominantly random reads) by primers, which serve as unique identifiers for directly accessing data. However, primers come with a significant limitation regarding the maximum available number per DNA library. The number of different primers within a library is typically very small (e.g. ≈10). We propose a method to overcome this deficiency and present a general-purpose technique for addressing and directly accessing thousands to potentially millions of different data objects within the same DNA pool. Our approach utilizes a fountain code, sophisticated probe design, and microarray technologies. A key component is locality-sensitive hashing, making checks for dissimilarity among such a large number of probes and data objects feasible.
| S-EPMC8829907 | biostudies-literature
Project description:The Zagreb connection indices are the known topological descriptors of the graphs that are constructed from the connection cardinality (degree of given nodes lying at a distance 2) presented in 1972 to determine the total electron energy of the alternate hydrocarbons. For a long time, these connection indices did not receive much research attention. Ali and Trinajstić [Mol. Inform. 37, Art. No. 1800008, 2018] examined the Zagreb connection indices and found that they compared to basic Zagreb indices and that they provide a finer value for the correlation coefficient for the 13 physico-chemical characteristics of the octane isomers. This article acquires the formulae of expected values of the first Zagreb connection index of a random cyclooctatetraene chain, a random polyphenyls chain, and a random chain network with l number of octagons, hexagons, and pentagons, respectively. The article presents extreme and average values of all the above random chains concerning a set of special chains, including the meta-chain, the ortho-chain, and the para-chain.
| S-EPMC9846538 | biostudies-literature
Project description:Pneumatically-actuated soft robots have advantages over traditional rigid robots in many applications. In particular, their flexible bodies and gentle air-powered movements make them more suitable for use around humans and other objects that could be injured or damaged by traditional robots. However, existing systems for controlling soft robots currently require dedicated electromechanical hardware (usually solenoid valves) to maintain the actuation state (expanded or contracted) of each independent actuator. When combined with power, computation, and sensing components, this control hardware adds considerable cost, size, and power demands to the robot, thereby limiting the feasibility of soft robots in many important application areas. In this work, we introduce a pneumatic memory that uses air (not electricity) to set and maintain the states of large numbers of soft robotic actuators without dedicated electromechanical hardware. These pneumatic logic circuits use normally-closed microfluidic valves as transistor-like elements; this enables our circuits to support more complex computational functions than those built from normally-open valves. We demonstrate an eight-bit nonvolatile random-access pneumatic memory (RAM) that can maintain the states of multiple actuators, control both individual actuators and multiple actuators simultaneously using a pneumatic version of time division multiplexing (TDM), and set actuators to any intermediate position using a pneumatic version of analog-to-digital conversion. We perform proof-of-concept experimental testing of our pneumatic RAM by using it to control soft robotic hands playing individual notes, chords, and songs on a piano keyboard. By dramatically reducing the amount of hardware required to control multiple independent actuators in pneumatic soft robots, our pneumatic RAM can accelerate the spread of soft robotic technologies to a wide range of important application areas.
| S-EPMC8284813 | biostudies-literature