Neural network training and stochastic global optimization

This paper investigates local minima problem in neural network (NN) backpropagation supervised learning. The proposed algorithm of training makes use of stochastic optimization technique based on so-called low-discrepancy sequences. The learning process is considered as an unconstrained optimization problem and once parameter space (defined by the NN weights) and objective functions are defined, the proposed method searches for a global optimum. First, regions of attraction as candidates for local minima are obtained, and secondly, each region is searched for locating minima and subsequently finding a global minimum. The conducted algorithm is initially tested on multimodal mathematical functions and then on common benchmark problems for NN training. The results are finally discussed and compared with such obtained from backpropagation and other methods.