-

The goal of neural network engineering (NNE) is to study the advantages and disadvantages of neural networks and also providing methods to increase their performance. One of the problems in NNE is determination of optimal topology of neural networks for solving a given problem. There is no method to determine the optimal topology of multi-layer neural networks for a given problem. Usually, the designer selects a
topology for neural networks and then trains it. The selected topology remains fixed during the training period. The performance of neural network depends on its size(number of hidden layers and hidden units). Determination of the optimal topology of neural network is an intractable problem. Therefore, most of
algorithms for determination of the
topology of neural ne1twork are
approximate algorithms. These algorithms
could be classified into five main groups: pruning algorithms, constructive algorithms, hybrid algorithms, evolutionary algorithms, and learning automata based algorithms. The only learning automata(LA) based algorithms, called survival algorithm, has been proposed by Beigy and Meybodi. This algorithm uses an object migrating learning automata and error backpropagation (BP) algorithm and determines the number of hidden units of
three layers neural networks, as training proceeds. In this paper, we propose three
algorithms which are based on LA and BP. These algorithms determine a near optimal topology with low time complexity and high generalization capability for a given training set. These algorithms: have two parts: determination of number of hidden units and determination of the number of
hidden weights. One of the proposed algorithms uses the survival algorithm to determine the number of hidden units. A new algorithm based on LA is proposed to determine the number of hidden weights. This algorithm deletes weights with small effect, which leads to lower time complexity and higher generalization rate. Two other algorithms do not omit the hidden units explicitly; a hidden unit is omitted when all its input weights are deleted. Most of the reported algorithms in the literature for determination of topology of neural networks use hill-climbing method and may stuck at local minima.
The proposed algorithms use global search
method which results in increasing the probability of escaping from local minima. The proposed algorithms have been tested on several problems such as: recognition of Farsi and English digits. Simulation results show that the produced networks have good performance. The proposed algorithms are compared with Karnin pruning algorithm.