Dataset Information

HyAdamC: A New Adam-Based Hybrid Optimization Algorithm for Convolution Neural Networks.

ABSTRACT: As the performance of devices that conduct large-scale computations has been rapidly improved, various deep learning models have been successfully utilized in various applications. Particularly, convolution neural networks (CNN) have shown remarkable performance in image processing tasks such as image classification and segmentation. Accordingly, more stable and robust optimization methods are required to effectively train them. However, the traditional optimizers used in deep learning still have unsatisfactory training performance for the models with many layers and weights. Accordingly, in this paper, we propose a new Adam-based hybrid optimization method called HyAdamC for training CNNs effectively. HyAdamC uses three new velocity control functions to adjust its search strength carefully in term of initial, short, and long-term velocities. Moreover, HyAdamC utilizes an adaptive coefficient computation method to prevent that a search direction determined by the first momentum is distorted by any outlier gradients. Then, these are combined into one hybrid method. In our experiments, HyAdamC showed not only notable test accuracies but also significantly stable and robust optimization abilities when training various CNN models. Furthermore, we also found that HyAdamC could be applied into not only image classification and image segmentation tasks.

SUBMITTER: Kim KS

PROVIDER: S-EPMC8231656 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:This research paper develops a novel hybrid approach, called hybrid particle swarm optimization-teaching-learning-based optimization (hPSO-TLBO), by combining two metaheuristic algorithms to solve optimization problems. The main idea in hPSO-TLBO design is to integrate the exploitation ability of PSO with the exploration ability of TLBO. The meaning of "exploitation capabilities of PSO" is the ability of PSO to manage local search with the aim of obtaining possible better solutions near the obtained solutions and promising areas of the problem-solving space. Also, "exploration abilities of TLBO" means the ability of TLBO to manage the global search with the aim of preventing the algorithm from getting stuck in inappropriate local optima. hPSO-TLBO design methodology is such that in the first step, the teacher phase in TLBO is combined with the speed equation in PSO. Then, in the second step, the learning phase of TLBO is improved based on each student learning from a selected better student that has a better value for the objective function against the corresponding student. The algorithm is presented in detail, accompanied by a comprehensive mathematical model. A group of benchmarks is used to evaluate the effectiveness of hPSO-TLBO, covering various types such as unimodal, high-dimensional multimodal, and fixed-dimensional multimodal. In addition, CEC 2017 benchmark problems are also utilized for evaluation purposes. The optimization results clearly demonstrate that hPSO-TLBO performs remarkably well in addressing the benchmark functions. It exhibits a remarkable ability to explore and exploit the search space while maintaining a balanced approach throughout the optimization process. Furthermore, a comparative analysis is conducted to evaluate the performance of hPSO-TLBO against twelve widely recognized metaheuristic algorithms. The evaluation of the experimental findings illustrates that hPSO-TLBO consistently outperforms the competing algorithms across various benchmark functions, showcasing its superior performance. The successful deployment of hPSO-TLBO in addressing four engineering challenges highlights its effectiveness in tackling real-world applications.

Dataset Information

HyAdamC: A New Adam-Based Hybrid Optimization Algorithm for Convolution Neural Networks.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets