Gesture recognition based on depth information and convolutional neural network

Du Jiang; Gongfa Li; Guozhang Jiang; Disi Chen; Zhaojie Ju

doi:10.1109/SMC.2018.00685

Gesture recognition based on depth information and convolutional neural network

Du Jiang
, Gongfa Li
, Guozhang Jiang
, Disi Chen
, Zhaojie Ju

Wuhan University of Science and Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

385 Downloads (Pure)

Abstract

Vision-based gesture Recognition accords with natural communication habits of human and can carry out long-distance and non-contact interactions. So it has become a hot direction in human-computer interaction research whose Recognition effect largely depends on the performance of image preprocessing and Recognition algorithms. In this paper, a gesture Recognition method using color image and depth image combined is designed. For the influence of the angle on the same gesture, the skeleton algorithm is optimized based on the layer-by-layer stripping concept. The fast refinement algorithm improves the process of repeated scanning, extracts the key node information in the skeleton map of the hand, and establishes the spatial axis of the hand to determine the gesture direction. The gesture Recognition experiment was performed based on convolutional neural network. The results showed the Recognition accuracy rate was 96.01%, and the robustness and accuracy of the proposed Recognition method were verified.

Original language	English
Title of host publication	Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	4041-4046
Number of pages	6
ISBN (Electronic)	978-1-5386-6650-0
ISBN (Print)	978-1-5386-6651-7
DOIs	https://doi.org/10.1109/SMC.2018.00685
Publication status	Published - 17 Jan 2019
Event	2018 IEEE International Conference on Systems, Man and Cybernetics - Miyazaki, Japan Duration: 7 Oct 2018 → 10 Oct 2018 http://www.smc2018.org/

Publication series

Name	IEEE SMC Proceedings Series
Publisher	IEEE
ISSN (Print)	1062-922X
ISSN (Electronic)	2577-1655

Conference

Conference	2018 IEEE International Conference on Systems, Man and Cybernetics
Abbreviated title	SMC 2018
Country/Territory	Japan
City	Miyazaki
Period	7/10/18 → 10/10/18
Internet address	http://www.smc2018.org/

Keywords

Convolutional neural network
Depth information
Gesture Recognition
Hand skeleton extraction

Access to Document

10.1109/SMC.2018.00685

Gesture Recognition Based on Depth Information and Convolutional Neural Network
© © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript (Post-print), 343 KB

Cite this

Jiang, D., Li, G., Jiang, G., Chen, D., & Ju, Z. (2019). Gesture recognition based on depth information and convolutional neural network. In Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018 (pp. 4041-4046). Article 8616682 (IEEE SMC Proceedings Series). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC.2018.00685

@inproceedings{8fb08a6fb83c451aa1e37f896a5bce83,

title = "Gesture recognition based on depth information and convolutional neural network",

abstract = "Vision-based gesture Recognition accords with natural communication habits of human and can carry out long-distance and non-contact interactions. So it has become a hot direction in human-computer interaction research whose Recognition effect largely depends on the performance of image preprocessing and Recognition algorithms. In this paper, a gesture Recognition method using color image and depth image combined is designed. For the influence of the angle on the same gesture, the skeleton algorithm is optimized based on the layer-by-layer stripping concept. The fast refinement algorithm improves the process of repeated scanning, extracts the key node information in the skeleton map of the hand, and establishes the spatial axis of the hand to determine the gesture direction. The gesture Recognition experiment was performed based on convolutional neural network. The results showed the Recognition accuracy rate was 96.01\%, and the robustness and accuracy of the proposed Recognition method were verified.",

keywords = "Convolutional neural network, Depth information, Gesture Recognition, Hand skeleton extraction",

author = "Du Jiang and Gongfa Li and Guozhang Jiang and Disi Chen and Zhaojie Ju",

year = "2019",

month = jan,

day = "17",

doi = "10.1109/SMC.2018.00685",

language = "English",

isbn = "978-1-5386-6651-7",

series = "IEEE SMC Proceedings Series",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "4041--4046",

booktitle = "Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018",

address = "United States",

note = "2018 IEEE International Conference on Systems, Man and Cybernetics, SMC 2018 ; Conference date: 07-10-2018 Through 10-10-2018",

url = "http://www.smc2018.org/",

}

Jiang, D, Li, G, Jiang, G, Chen, D & Ju, Z 2019, Gesture recognition based on depth information and convolutional neural network. in Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018., 8616682, IEEE SMC Proceedings Series, Institute of Electrical and Electronics Engineers Inc., pp. 4041-4046, 2018 IEEE International Conference on Systems, Man and Cybernetics, Miyazaki, Japan, 7/10/18. https://doi.org/10.1109/SMC.2018.00685

Gesture recognition based on depth information and convolutional neural network. / Jiang, Du; Li, Gongfa; Jiang, Guozhang et al.
Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018. Institute of Electrical and Electronics Engineers Inc., 2019. p. 4041-4046 8616682 (IEEE SMC Proceedings Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Gesture recognition based on depth information and convolutional neural network

AU - Jiang, Du

AU - Li, Gongfa

AU - Jiang, Guozhang

AU - Chen, Disi

AU - Ju, Zhaojie

PY - 2019/1/17

Y1 - 2019/1/17

N2 - Vision-based gesture Recognition accords with natural communication habits of human and can carry out long-distance and non-contact interactions. So it has become a hot direction in human-computer interaction research whose Recognition effect largely depends on the performance of image preprocessing and Recognition algorithms. In this paper, a gesture Recognition method using color image and depth image combined is designed. For the influence of the angle on the same gesture, the skeleton algorithm is optimized based on the layer-by-layer stripping concept. The fast refinement algorithm improves the process of repeated scanning, extracts the key node information in the skeleton map of the hand, and establishes the spatial axis of the hand to determine the gesture direction. The gesture Recognition experiment was performed based on convolutional neural network. The results showed the Recognition accuracy rate was 96.01%, and the robustness and accuracy of the proposed Recognition method were verified.

AB - Vision-based gesture Recognition accords with natural communication habits of human and can carry out long-distance and non-contact interactions. So it has become a hot direction in human-computer interaction research whose Recognition effect largely depends on the performance of image preprocessing and Recognition algorithms. In this paper, a gesture Recognition method using color image and depth image combined is designed. For the influence of the angle on the same gesture, the skeleton algorithm is optimized based on the layer-by-layer stripping concept. The fast refinement algorithm improves the process of repeated scanning, extracts the key node information in the skeleton map of the hand, and establishes the spatial axis of the hand to determine the gesture direction. The gesture Recognition experiment was performed based on convolutional neural network. The results showed the Recognition accuracy rate was 96.01%, and the robustness and accuracy of the proposed Recognition method were verified.

KW - Convolutional neural network

KW - Depth information

KW - Gesture Recognition

KW - Hand skeleton extraction

UR - https://www.scopus.com/pages/publications/85062241558

U2 - 10.1109/SMC.2018.00685

DO - 10.1109/SMC.2018.00685

M3 - Conference contribution

AN - SCOPUS:85062241558

SN - 978-1-5386-6651-7

T3 - IEEE SMC Proceedings Series

SP - 4041

EP - 4046

BT - Proceedings - 2018 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2018

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2018 IEEE International Conference on Systems, Man and Cybernetics

Y2 - 7 October 2018 through 10 October 2018

ER -

Gesture recognition based on depth information and convolutional neural network

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this