Serial-parallel dynamic hand gesture recognition network for human-robot interaction

Yinan Zhao; Jian Zhou; Zhaojie Ju; Junkang Chen; Qing Gao

doi:10.1109/M2VIP58386.2023.10413398

Serial-parallel dynamic hand gesture recognition network for human-robot interaction

Yinan Zhao, Jian Zhou, Zhaojie Ju, Junkang Chen, Qing Gao

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

44 Downloads (Pure)

Abstract

In recent years, hand gesture recognition has played a crucial role in human-robot interaction (HRI). This paper proposes a skeleton-based serial-parallel dynamic hand gesture recognition network. A set of skeleton-based physical features is designed to model the spatial relationship of joints to construct skeletal space configurations. A slow-fast double-scale parallel network is proposed to extract the temporal dynamics of gestures. The attention mechanism is used to fuse the spatiotemporal information of the gestures, and the recognition result is obtained through the serial 1DCNN structure. In addition, the data enhancement technology based on transformation is used to improve the generalization of the network. The proposed methods are evaluated on the SHREC14 and SHREC28 datasets, which show superior performance, with an accuracy of 95.11% and 92.98%, respectively. The network is fine-tuned on the customized dataset HRIGes, and the recognition results are mapped to a fivefingered dexterous manipulator to realize real-time human-robot interaction.

Original language	English
Title of host publication	Proceedings of 29th IEEE International Conference on Mechatronics and Machine Vision in Practice
Publisher	Institute of Electrical and Electronics Engineers Inc.
Number of pages	6
ISBN (Electronic)	9798350325621
ISBN (Print)	9798350325638
DOIs	https://doi.org/10.1109/M2VIP58386.2023.10413398
Publication status	Published - 2 Feb 2024
Event	29th IEEE International Conference On Mechatronics And Machine Vision In Practice - Queenstown, New Zealand Duration: 21 Nov 2023 → 24 Nov 2023

Conference

Conference	29th IEEE International Conference On Mechatronics And Machine Vision In Practice
Country/Territory	New Zealand
City	Queenstown
Period	21/11/23 → 24/11/23

Keywords

serial-parallel network
hand gesture recognition
human-robot interaction

Access to Document

10.1109/M2VIP58386.2023.10413398

Serial-Parallel_Network
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript (Post-print), 1.89 MB

Cite this

@inproceedings{a898722c905b474081d71d0bbaeac0a3,

title = "Serial-parallel dynamic hand gesture recognition network for human-robot interaction",

abstract = "In recent years, hand gesture recognition has played a crucial role in human-robot interaction (HRI). This paper proposes a skeleton-based serial-parallel dynamic hand gesture recognition network. A set of skeleton-based physical features is designed to model the spatial relationship of joints to construct skeletal space configurations. A slow-fast double-scale parallel network is proposed to extract the temporal dynamics of gestures. The attention mechanism is used to fuse the spatiotemporal information of the gestures, and the recognition result is obtained through the serial 1DCNN structure. In addition, the data enhancement technology based on transformation is used to improve the generalization of the network. The proposed methods are evaluated on the SHREC14 and SHREC28 datasets, which show superior performance, with an accuracy of 95.11% and 92.98%, respectively. The network is fine-tuned on the customized dataset HRIGes, and the recognition results are mapped to a fivefingered dexterous manipulator to realize real-time human-robot interaction.",

keywords = "serial-parallel network, hand gesture recognition, human-robot interaction",

author = "Yinan Zhao and Jian Zhou and Zhaojie Ju and Junkang Chen and Qing Gao",

year = "2024",

month = feb,

day = "2",

doi = "10.1109/M2VIP58386.2023.10413398",

language = "English",

isbn = "9798350325638",

booktitle = "Proceedings of 29th IEEE International Conference on Mechatronics and Machine Vision in Practice",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

address = "United States",

note = "29th IEEE International Conference On Mechatronics And Machine Vision In Practice ; Conference date: 21-11-2023 Through 24-11-2023",

}

Zhao, Y, Zhou, J, Ju, Z, Chen, J & Gao, Q 2024, Serial-parallel dynamic hand gesture recognition network for human-robot interaction. in Proceedings of 29th IEEE International Conference on Mechatronics and Machine Vision in Practice. Institute of Electrical and Electronics Engineers Inc., 29th IEEE International Conference On Mechatronics And Machine Vision In Practice, Queenstown, New Zealand, 21/11/23. https://doi.org/10.1109/M2VIP58386.2023.10413398

Serial-parallel dynamic hand gesture recognition network for human-robot interaction. / Zhao, Yinan; Zhou, Jian; Ju, Zhaojie et al.
Proceedings of 29th IEEE International Conference on Mechatronics and Machine Vision in Practice. Institute of Electrical and Electronics Engineers Inc., 2024.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Serial-parallel dynamic hand gesture recognition network for human-robot interaction

AU - Zhao, Yinan

AU - Zhou, Jian

AU - Ju, Zhaojie

AU - Chen, Junkang

AU - Gao, Qing

PY - 2024/2/2

Y1 - 2024/2/2

N2 - In recent years, hand gesture recognition has played a crucial role in human-robot interaction (HRI). This paper proposes a skeleton-based serial-parallel dynamic hand gesture recognition network. A set of skeleton-based physical features is designed to model the spatial relationship of joints to construct skeletal space configurations. A slow-fast double-scale parallel network is proposed to extract the temporal dynamics of gestures. The attention mechanism is used to fuse the spatiotemporal information of the gestures, and the recognition result is obtained through the serial 1DCNN structure. In addition, the data enhancement technology based on transformation is used to improve the generalization of the network. The proposed methods are evaluated on the SHREC14 and SHREC28 datasets, which show superior performance, with an accuracy of 95.11% and 92.98%, respectively. The network is fine-tuned on the customized dataset HRIGes, and the recognition results are mapped to a fivefingered dexterous manipulator to realize real-time human-robot interaction.

AB - In recent years, hand gesture recognition has played a crucial role in human-robot interaction (HRI). This paper proposes a skeleton-based serial-parallel dynamic hand gesture recognition network. A set of skeleton-based physical features is designed to model the spatial relationship of joints to construct skeletal space configurations. A slow-fast double-scale parallel network is proposed to extract the temporal dynamics of gestures. The attention mechanism is used to fuse the spatiotemporal information of the gestures, and the recognition result is obtained through the serial 1DCNN structure. In addition, the data enhancement technology based on transformation is used to improve the generalization of the network. The proposed methods are evaluated on the SHREC14 and SHREC28 datasets, which show superior performance, with an accuracy of 95.11% and 92.98%, respectively. The network is fine-tuned on the customized dataset HRIGes, and the recognition results are mapped to a fivefingered dexterous manipulator to realize real-time human-robot interaction.

KW - serial-parallel network

KW - hand gesture recognition

KW - human-robot interaction

UR - https://www.aconf.org/conf_190091.html

U2 - 10.1109/M2VIP58386.2023.10413398

DO - 10.1109/M2VIP58386.2023.10413398

M3 - Conference contribution

SN - 9798350325638

BT - Proceedings of 29th IEEE International Conference on Mechatronics and Machine Vision in Practice

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 29th IEEE International Conference On Mechatronics And Machine Vision In Practice

Y2 - 21 November 2023 through 24 November 2023

ER -

Serial-parallel dynamic hand gesture recognition network for human-robot interaction

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this