Browse
Submit Data
Databases
API
Help

Dataset Information

14 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

The Deep Learning Solutions on Lossless Compression Methods for Alleviating Data Load on IoT Nodes in Smart Cities.

ABSTRACT: Networking is crucial for smart city projects nowadays, as it offers an environment where people and things are connected. This paper presents a chronology of factors on the development of smart cities, including IoT technologies as network infrastructure. Increasing IoT nodes leads to increasing data flow, which is a potential source of failure for IoT networks. The biggest challenge of IoT networks is that the IoT may have insufficient memory to handle all transaction data within the IoT network. We aim in this paper to propose a potential compression method for reducing IoT network data traffic. Therefore, we investigate various lossless compression algorithms, such as entropy or dictionary-based algorithms, and general compression methods to determine which algorithm or method adheres to the IoT specifications. Furthermore, this study conducts compression experiments using entropy (Huffman, Adaptive Huffman) and Dictionary (LZ77, LZ78) as well as five different types of datasets of the IoT data traffic. Though the above algorithms can alleviate the IoT data traffic, adaptive Huffman gave the best compression algorithm. Therefore, in this paper, we aim to propose a conceptual compression method for IoT data traffic by improving an adaptive Huffman based on deep learning concepts using weights, pruning, and pooling in the neural network. The proposed algorithm is believed to obtain a better compression ratio. Additionally, in this paper, we also discuss the challenges of applying the proposed algorithm to IoT data compression due to the limitations of IoT memory and IoT processor, which later it can be implemented in IoT networks.

SUBMITTER: Nasif A

PROVIDER: S-EPMC8235183 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Similar Datasets

IoT-Enabled Smart Cities: Evolution and Outlook.

Project description:For the last decade the Smart City concept has been under development, fostered by the growing urbanization of the world's population and the need to handle the challenges that such a scenario raises. During this time many Smart City projects have been executed-some as proof-of-concept, but a growing number resulting in permanent, production-level deployments, improving the operation of the city and the quality of life of its citizens. Thus, Smart Cities are still a highly relevant paradigm which needs further development before it reaches its full potential and provides robust and resilient solutions. In this paper, the focus is set on the Internet of Things (IoT) as an enabling technology for the Smart City. In this sense, the paper reviews the current landscape of IoT-enabled Smart Cities, surveying relevant experiences and city initiatives that have embedded IoT within their city services and how they have generated an impact. The paper discusses the key technologies that have been developed and how they are contributing to the realization of the Smart City. Moreover, it presents some challenges that remain open ahead of us and which are the initiatives and technologies that are under development to tackle them.

| S-EPMC8271664 | biostudies-literature

A Systematic Review of IoT Solutions for Smart Farming.

Project description:The world population growth is increasing the demand for food production. Furthermore, the reduction of the workforce in rural areas and the increase in production costs are challenges for food production nowadays. Smart farming is a farm management concept that may use Internet of Things (IoT) to overcome the current challenges of food production. This work uses the preferred reporting items for systematic reviews (PRISMA) methodology to systematically review the existing literature on smart farming with IoT. The review aims to identify the main devices, platforms, network protocols, processing data technologies and the applicability of smart farming with IoT to agriculture. The review shows an evolution in the way data is processed in recent years. Traditional approaches mostly used data in a reactive manner. In more recent approaches, however, new technological developments allowed the use of data to prevent crop problems and to improve the accuracy of crop diagnosis.

| S-EPMC7436012 | biostudies-literature

Blockchain and smart contract for IoT enabled smart agriculture.

Project description:The agricultural sector is still lagging behind from all other sectors in terms of using the newest technologies. For production, the latest machines are being introduced and adopted. However, pre-harvest and post-harvest processing are still done by following traditional methodologies while tracing, storing, and publishing agricultural data. As a result, farmers are not getting deserved payment, consumers are not getting enough information before buying their product, and intermediate person/processors are increasing retail prices. Using blockchain, smart contracts, and IoT devices, we can fully automate the process while establishing absolute trust among all these parties. In this research, we explored the different aspects of using blockchain and smart contracts with the integration of IoT devices in pre-harvesting and post-harvesting segments of agriculture. We proposed a system that uses blockchain as the backbone while IoT devices collect data from the field level, and smart contracts regulate the interaction among all these contributing parties. The system implementation has been shown in diagrams and with proper explanations. Gas costs of every operation have also been attached for a better understanding of the costs. We also analyzed the system in terms of challenges and advantages. The overall impact of this research was to show the immutable, available, transparent, and robustly secure characteristics of blockchain in the field of agriculture while also emphasizing the vigorous mechanism that the collaboration of blockchain, smart contract, and IoT presents.

| S-EPMC8022535 | biostudies-literature

StormSense: A New Integrated Network of IoT Water Level Sensors in the Smart Cities of Hampton Roads, VA.

Project description:Propagation of cost-effective water level sensors powered through the Internet of Things (IoT) has expanded the available offerings of ingestible data streams at the disposal of modern smart cities. StormSense is an IoT-enabled inundation forecasting research initiative and an active participant in the Global City Teams Challenge seeking to enhance flood preparedness in the smart cities of Hampton Roads, VA for flooding resulting from storm surge, rain, and tides. In this study, we present the results of the new StormSense water level sensors to help establish the "regional resilience monitoring network" noted as a key recommendation from the Intergovernmental Pilot Project. To accomplish this, the Commonwealth Center for Recurrent Flooding Resiliency's Tidewatch tidal forecast system is being used as a starting point to integrate the extant (NOAA) and new (USGS and StormSense) water level sensors throughout the region, and demonstrate replicability of the solution across the cities of Newport News, Norfolk, and Virginia Beach within Hampton Roads, VA. StormSense's network employs a mix of ultrasonic and radar remote sensing technologies to record water levels during 2017 Hurricanes Jose and Maria. These data were used to validate the inundation predictions of a street-level hydrodynamic model (5-m resolution), while the water levels from the sensors and the model were concomitantly validated by a temporary water level sensor deployed by the USGS in the Hague, and crowd-sourced GPS maximum flooding extent observations from the Sea Level Rise app, developed in Norfolk, VA.

| S-EPMC6512834 | biostudies-other

Hadoop Oriented Smart Cities Architecture.

Project description:A smart city implies a consistent use of technology for the benefit of the community. As the city develops over time, components and subsystems such as smart grids, smart water management, smart traffic and transportation systems, smart waste management systems, smart security systems, or e-governance are added. These components ingest and generate a multitude of structured, semi-structured or unstructured data that may be processed using a variety of algorithms in batches, micro batches or in real-time. The ICT architecture must be able to handle the increased storage and processing needs. When vertical scaling is no longer a viable solution, Hadoop can offer efficient linear horizontal scaling, solving storage, processing, and data analyses problems in many ways. This enables architects and developers to choose a stack according to their needs and skill-levels. In this paper, we propose a Hadoop-based architectural stack that can provide the ICT backbone for efficiently managing a smart city. On the one hand, Hadoop, together with Spark and the plethora of NoSQL databases and accompanying Apache projects, is a mature ecosystem. This is one of the reasons why it is an attractive option for a Smart City architecture. On the other hand, it is also very dynamic; things can change very quickly, and many new frameworks, products and options continue to emerge as others decline. To construct an optimized, modern architecture, we discuss and compare various products and engines based on a process that takes into consideration how the products perform and scale, as well as the reusability of the code, innovations, features, and support and interest in online communities.

| S-EPMC5948833 | biostudies-other

Hybrid Adaptive Lossless Image Compression Based on Discrete Wavelet Transform.

Project description:A new hybrid transform for lossless image compression exploiting a discrete wavelet transform (DWT) and prediction is the main new contribution of this paper. Simple prediction is generally considered ineffective in conjunction with DWT but we applied it to subbands of DWT modified using reversible denoising and lifting steps (RDLSs) with step skipping. The new transform was constructed in an image-adaptive way using heuristics and entropy estimation. For a large and diverse test set consisting of 499 photographic and 247 non-photographic (screen content) images, we found that RDLS with step skipping allowed effectively combining DWT with prediction. Using prediction, we nearly doubled the JPEG 2000 compression ratio improvements that could be obtained using RDLS with step skipping. Because for some images it might be better to apply prediction instead of DWT, we proposed compression schemes with various tradeoffs, which are practical contributions of this study. Compared with unmodified JPEG 2000, one scheme improved the compression ratios of photographic and non-photographic images, on average, by 1.2% and 30.9%, respectively, at the cost of increasing the compression time by 2% and introducing only minimal modifications to JPEG 2000. Greater ratio improvements, exceeding 2% and 32%, respectively, are attainable at a greater cost.

| S-EPMC7517294 | biostudies-literature

Context-Aware Lossless and Lossy Compression of Radio Frequency Signals.

Project description:We propose an algorithm based on linear prediction that can perform both the lossless and near-lossless compression of RF signals. The proposed algorithm is coupled with two signal detection methods to determine the presence of relevant signals and apply varying levels of loss as needed. The first method uses spectrum sensing techniques, while the second one takes advantage of the error computed in each iteration of the Levinson-Durbin algorithm. These algorithms have been integrated as a new pre-processing stage into FAPEC, a data compressor first designed for space missions. We test the lossless algorithm using two different datasets. The first one was obtained from OPS-SAT, an ESA CubeSat, while the second one was obtained using a SDRplay RSPdx in Barcelona, Spain. The results show that our approach achieves compression ratios that are 23% better than gzip (on average) and very similar to those of FLAC, but at higher speeds. We also assess the performance of our signal detectors using the second dataset. We show that high ratios can be achieved thanks to the lossy compression of the segments without any relevant signal.

| S-EPMC10098950 | biostudies-literature

A Survey on 5G and LPWAN-IoT for Improved Smart Cities and Remote Area Applications: From the Aspect of Architecture and Security.

Project description:Addressing the recent trend of the massive demand for resources and ubiquitous use for all citizens has led to the conceptualization of technologies such as the Internet of Things (IoT) and smart cities. Ubiquitous IoT connectivity can be achieved to serve both urban and underserved remote areas such as rural communities by deploying 5G mobile networks with Low Power Wide Area Network (LPWAN). The current architectures will not offer flexible connectivity to many IoT applications due to high service demand, data exchange, emerging technologies, and security challenges. Hence, this paper explores various architectures that consider a hybrid 5G-LPWAN-IoT and Smart Cities. This includes security challenges as well as endogenous security and solutions in 5G and LPWAN-IoT. The slicing of virtual networks using software-defined network (SDN)/network function virtualization (NFV) based on the different quality of service (QoS) to satisfy different services and quality of experience (QoE) is presented. Also, a strategy that considers the implementation of 5G jointly with Weightless-N (TVWS) technologies to reduce the cell edge interference is considered. Discussions on the need for ubiquity connectivity leveraging 5G and LPWAN-IoT are presented. In addition, future research directions are presented, including a unified 5G network and LPWAN-IoT architecture that will holistically support integration with emerging technologies and endogenous security for improved/secured smart cities and remote areas IoT applications. Finally, the use of LPWAN jointly with low earth orbit (LEO) satellites for ubiquitous IoT connectivity is advocated in this paper.

| S-EPMC9412619 | biostudies-literature

NGC: lossless and lossy compression of aligned high-throughput sequencing data.

Project description:A major challenge of current high-throughput sequencing experiments is not only the generation of the sequencing data itself but also their processing, storage and transmission. The enormous size of these data motivates the development of data compression algorithms usable for the implementation of the various storage policies that are applied to the produced intermediate and final result files. In this article, we present NGC, a tool for the compression of mapped short read data stored in the wide-spread SAM format. NGC enables lossless and lossy compression and introduces the following two novel ideas: first, we present a way to reduce the number of required code words by exploiting common features of reads mapped to the same genomic positions; second, we present a highly configurable way for the quantization of per-base quality values, which takes their influence on downstream analyses into account. NGC, evaluated with several real-world data sets, saves 33-66% of disc space using lossless and up to 98% disc space using lossy compression. By applying two popular variant and genotype prediction tools to the decompressed data, we could show that the lossy compression modes preserve >99% of all called variants while outperforming comparable methods in some configurations.

| S-EPMC3592443 | biostudies-literature

LFC-UNet: learned lossless medical image fast compression with U-Net.

Project description:In the field of medicine, the rapid advancement of medical technology has significantly increased the speed of medical image generation, compelling us to seek efficient methods for image compression. Neural networks, owing to their outstanding image estimation capabilities, have provided new avenues for lossless compression. In recent years, learning-based lossless image compression methods, combining neural network predictions with residuals, have achieved performance comparable to traditional non-learning algorithms. However, existing methods have not taken into account that residuals often concentrate excessively, hindering the neural network's ability to learn accurate residual probability estimation. To address this issue, this study employs a weighted cross-entropy method to handle the imbalance in residual categories. In terms of network architecture, we introduce skip connections from U-Net to better capture image features, thereby obtaining accurate probability estimates. Furthermore, our framework boasts excellent encoding speed, as the model is able to acquire all residuals and residual probabilities in a single inference pass. The experimental results demonstrate that the proposed method achieves state-of-the-art performance on medical datasets while also offering the fastest processing speed. As illustrated by an instance using head CT data, our approach achieves a compression efficiency of 2.30 bits per pixel, with a processing time of only 0.320 seconds per image.

| S-EPMC10909184 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data