پرورش شبکه های عصبی پیشخور (FNN ها) توسط بهینه سازی ازدحام ذرات (PSO) هیبرید و الگوریتم جستجوی گرانشی

ABSTRACT Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm

Recently, Feedforward Neural Networks (FNNs), especially FNNs with two layers [1], have been widely used. In fact, FNNs with two layers are the most popular neural network in practical applications. Generally, an FNN with two layers is suitable for classifications of nonlinearly separable patterns [2,3] and to approximate functions [4,5]. It has been proven that two layer FNNs can approximate any continuous and discontinuous function [4]. Learning is an essential part of any neural network and becomes attractive to many researchers. For FNNs, most applications have used the standard [6] or improved [7– 9] Back-Propagation (BP) algorithm as their training method. The BP algorithm is a gradient-based algorithm which has some drawbacks, such as slow convergence [10] and the tendency to become trapped in local minima [11].

In FNNs, during the learning process, the goal is to find the best combination of connection weights and biases in order to achieve the minimum error. However, most of the time FNNs converge to points which are the best solution locally but not globally. In other words, learning algorithms lead FNNs to local minima rather than the global minimum. According to [10], the convergence of the BP algorithm is highly dependent on the initial values of weights, biases, and its parameters. These parameters include learning rate and momentum. In the literature, using novel heuristic optimization methods or evolutionary algorithms is a popular solution to enhance the problems of BP-based learning algorithms.

Various heuristic optimization methods have been used to train FNNs, such as Simulated Annealing (SA) [12,13], Genetic Algorithms (GAs) [14], Particle Swarm Optimization (PSO) algorithms [15–19], Magnetic Optimization Algorithm (MOA) [20], and Differential Evolution (DE) [21]. According to [10], some, such as SA and GA, could reduce the probability of trapping in local minima, but they still suffer from slow convergence rates. According to [16,22], PSO is one of the most efficient and practical optimization algorithms in terms of reducing both of the aforementioned drawbacks. In this paper, the efficiencies of GSA and PSOGSA are investigated for training FNNs in comparison with PSO [23]. The rest of the paper is organized as follows: Sections 2–4 present a brief introduction to the concepts of PSO, GSA, and PSOGSA, respectively. Section 5 discusses the method of applying PSO, GSA, and PSOGSA to FNNs as evolutionary training algorithms. The experimental results are demonstrated in Section 6. Finally, Section 7 concludes the paper.