Supervised neural network training with a hybrid global optimization technique

Antoniya Georgieva; Ivan Jordanov

doi:10.1109/ijcnn.2006.247342

Supervised neural network training with a hybrid global optimization technique

Antoniya Georgieva^*, Ivan Jordanov

^*Corresponding author for this work

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

A novel hybrid global optimization method applied for feedforward neural networks (NN) supervised learning is investigated. The network weights are determined by minimizing the traditional mean-square error function. The optimization technique, called GLPτS is a combination of novel global optimization heuristic search based on low-discrepancy sequences of points, called LP_τ Optimization (LP_τO), a Genetic Algorithm, and a Simplex local search. The proposed method is initially tested on 10 multimodal mathematical functions of 30 and 100 dimensions. Subsequently, it is applied for training moderate size NN for function fitting and solving benchmark classification problems, such as the parity problem (XOR and 4-Parity), Iris dataset, and a medical diagnosis problem (Diabetes). The investigated technique is also tested on predicting continuous output of a mechanical system dataset (Servo). Finally, the results are analysed, discussed, and compared with others.

Original language	English
Title of host publication	The 2006 IEEE International Joint Conference on Neural Network Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	3401-3408
Number of pages	8
ISBN (Print)	0780394909, 9780780394902
DOIs	https://doi.org/10.1109/ijcnn.2006.247342
Publication status	Published - 30 Oct 2006
Event	International Joint Conference on Neural Networks 2006, IJCNN '06 - Vancouver, BC, Canada Duration: 16 Jul 2006 → 21 Jul 2006

Publication series

Name	Proceedings of IEEE International Conference on Neural Networks
Publisher	IEEE
ISSN (Print)	2161-4393
ISSN (Electronic)	2161-4407

Conference

Conference	International Joint Conference on Neural Networks 2006, IJCNN '06
Country/Territory	Canada
City	Vancouver, BC
Period	16/07/06 → 21/07/06

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

Genetic algorithms
Global optimization
Hybrid methods
Low-discrepancy sequences
Simplex search
Supervised NN learning

Access to Document

10.1109/ijcnn.2006.247342

Cite this

Georgieva, A., & Jordanov, I. (2006). Supervised neural network training with a hybrid global optimization technique. In The 2006 IEEE International Joint Conference on Neural Network Proceedings (pp. 3401-3408). (Proceedings of IEEE International Conference on Neural Networks). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ijcnn.2006.247342

@inproceedings{c4607737eddb46378eedc495b66301e7,

title = "Supervised neural network training with a hybrid global optimization technique",

abstract = "A novel hybrid global optimization method applied for feedforward neural networks (NN) supervised learning is investigated. The network weights are determined by minimizing the traditional mean-square error function. The optimization technique, called GLPτS is a combination of novel global optimization heuristic search based on low-discrepancy sequences of points, called LPτ Optimization (LPτO), a Genetic Algorithm, and a Simplex local search. The proposed method is initially tested on 10 multimodal mathematical functions of 30 and 100 dimensions. Subsequently, it is applied for training moderate size NN for function fitting and solving benchmark classification problems, such as the parity problem (XOR and 4-Parity), Iris dataset, and a medical diagnosis problem (Diabetes). The investigated technique is also tested on predicting continuous output of a mechanical system dataset (Servo). Finally, the results are analysed, discussed, and compared with others.",

keywords = "Genetic algorithms, Global optimization, Hybrid methods, Low-discrepancy sequences, Simplex search, Supervised NN learning",

author = "Antoniya Georgieva and Ivan Jordanov",

year = "2006",

month = oct,

day = "30",

doi = "10.1109/ijcnn.2006.247342",

language = "English",

isbn = "0780394909",

series = "Proceedings of IEEE International Conference on Neural Networks",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "3401--3408",

booktitle = "The 2006 IEEE International Joint Conference on Neural Network Proceedings",

address = "United States",

note = "International Joint Conference on Neural Networks 2006, IJCNN '06 ; Conference date: 16-07-2006 Through 21-07-2006",

}

Georgieva, A & Jordanov, I 2006, Supervised neural network training with a hybrid global optimization technique. in The 2006 IEEE International Joint Conference on Neural Network Proceedings. Proceedings of IEEE International Conference on Neural Networks, Institute of Electrical and Electronics Engineers Inc., pp. 3401-3408, International Joint Conference on Neural Networks 2006, IJCNN '06, Vancouver, BC, Canada, 16/07/06. https://doi.org/10.1109/ijcnn.2006.247342

Supervised neural network training with a hybrid global optimization technique. / Georgieva, Antoniya; Jordanov, Ivan.
The 2006 IEEE International Joint Conference on Neural Network Proceedings. Institute of Electrical and Electronics Engineers Inc., 2006. p. 3401-3408 (Proceedings of IEEE International Conference on Neural Networks).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Supervised neural network training with a hybrid global optimization technique

AU - Georgieva, Antoniya

AU - Jordanov, Ivan

PY - 2006/10/30

Y1 - 2006/10/30

N2 - A novel hybrid global optimization method applied for feedforward neural networks (NN) supervised learning is investigated. The network weights are determined by minimizing the traditional mean-square error function. The optimization technique, called GLPτS is a combination of novel global optimization heuristic search based on low-discrepancy sequences of points, called LPτ Optimization (LPτO), a Genetic Algorithm, and a Simplex local search. The proposed method is initially tested on 10 multimodal mathematical functions of 30 and 100 dimensions. Subsequently, it is applied for training moderate size NN for function fitting and solving benchmark classification problems, such as the parity problem (XOR and 4-Parity), Iris dataset, and a medical diagnosis problem (Diabetes). The investigated technique is also tested on predicting continuous output of a mechanical system dataset (Servo). Finally, the results are analysed, discussed, and compared with others.

AB - A novel hybrid global optimization method applied for feedforward neural networks (NN) supervised learning is investigated. The network weights are determined by minimizing the traditional mean-square error function. The optimization technique, called GLPτS is a combination of novel global optimization heuristic search based on low-discrepancy sequences of points, called LPτ Optimization (LPτO), a Genetic Algorithm, and a Simplex local search. The proposed method is initially tested on 10 multimodal mathematical functions of 30 and 100 dimensions. Subsequently, it is applied for training moderate size NN for function fitting and solving benchmark classification problems, such as the parity problem (XOR and 4-Parity), Iris dataset, and a medical diagnosis problem (Diabetes). The investigated technique is also tested on predicting continuous output of a mechanical system dataset (Servo). Finally, the results are analysed, discussed, and compared with others.

KW - Genetic algorithms

KW - Global optimization

KW - Hybrid methods

KW - Low-discrepancy sequences

KW - Simplex search

KW - Supervised NN learning

UR - https://www.scopus.com/pages/publications/40649093832

U2 - 10.1109/ijcnn.2006.247342

DO - 10.1109/ijcnn.2006.247342

M3 - Conference contribution

AN - SCOPUS:40649093832

SN - 0780394909

SN - 9780780394902

T3 - Proceedings of IEEE International Conference on Neural Networks

SP - 3401

EP - 3408

BT - The 2006 IEEE International Joint Conference on Neural Network Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - International Joint Conference on Neural Networks 2006, IJCNN '06

Y2 - 16 July 2006 through 21 July 2006

ER -

Supervised neural network training with a hybrid global optimization technique

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Fingerprint

Cite this