Author(s): Hamid Salimi | Davar Giveki | Mohammad Ali Soltanshahi | Javad Hatami
Journal: International Journal of Artificial Intelligence & Applications
ISSN 0976-2191
Volume: 3;
Issue: 1;
Start page: 1;
Date: 2012;
VIEW PDF
DOWNLOAD PDF 
Original page
Keywords: Back Propagation (BP) algorithm | Gradient Decent (GD) | Conjugate Gradient (CG) | Modified Cuckoo Search (MCS) | Mixture of Experts (MEs)
ABSTRACT
This paper investigates a new method for improving the learning algorithm of Mixture of Experts (ME)model using a hybrid of Modified Cuckoo Search (MCS) and Conjugate Gradient (CG) as a second orderoptimization technique. The CG technique is combined with Back-Propagation (BP) algorithm to yield amuch more efficient learning algorithm for ME structure. In addition, the experts and gating networks inenhanced model are replaced by CG based Multi-Layer Perceptrons (MLPs) to provide faster and moreaccurate learning. The CG is considerably depends on initial weights of connections of Artificial NeuralNetwork (ANN), so, a metaheuristic algorithm, the so-called Modified Cuckoo Search is applied in order toselect the optimal weights. The performance of proposed method is compared with Gradient Decent BasedME (GDME) and Conjugate Gradient Based ME (CGME) in classification and regression problems. Theexperimental results show that hybrid MSC and CG based ME (MCS-CGME) has faster convergence andbetter performance in utilized benchmark data sets.
Journal: International Journal of Artificial Intelligence & Applications
ISSN 0976-2191
Volume: 3;
Issue: 1;
Start page: 1;
Date: 2012;
VIEW PDF
DOWNLOAD PDF Original pageKeywords: Back Propagation (BP) algorithm | Gradient Decent (GD) | Conjugate Gradient (CG) | Modified Cuckoo Search (MCS) | Mixture of Experts (MEs)
ABSTRACT
This paper investigates a new method for improving the learning algorithm of Mixture of Experts (ME)model using a hybrid of Modified Cuckoo Search (MCS) and Conjugate Gradient (CG) as a second orderoptimization technique. The CG technique is combined with Back-Propagation (BP) algorithm to yield amuch more efficient learning algorithm for ME structure. In addition, the experts and gating networks inenhanced model are replaced by CG based Multi-Layer Perceptrons (MLPs) to provide faster and moreaccurate learning. The CG is considerably depends on initial weights of connections of Artificial NeuralNetwork (ANN), so, a metaheuristic algorithm, the so-called Modified Cuckoo Search is applied in order toselect the optimal weights. The performance of proposed method is compared with Gradient Decent BasedME (GDME) and Conjugate Gradient Based ME (CGME) in classification and regression problems. Theexperimental results show that hybrid MSC and CG based ME (MCS-CGME) has faster convergence andbetter performance in utilized benchmark data sets.
