Serial-parallel dynamic hand gesture recognition network for human-robot interaction

Yinan Zhao, Jian Zhou, Zhaojie Ju, Junkang Chen, Qing Gao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Downloads (Pure)


In recent years, hand gesture recognition has played a crucial role in human-robot interaction (HRI). This paper proposes a skeleton-based serial-parallel dynamic hand gesture recognition network. A set of skeleton-based physical features is designed to model the spatial relationship of joints to construct skeletal space configurations. A slow-fast double-scale parallel network is proposed to extract the temporal dynamics of gestures. The attention mechanism is used to fuse the spatiotemporal information of the gestures, and the recognition result is obtained through the serial 1DCNN structure. In addition, the data enhancement technology based on transformation is used to improve the generalization of the network. The proposed methods are evaluated on the SHREC14 and SHREC28 datasets, which show superior performance, with an accuracy of 95.11% and 92.98%, respectively. The network is fine-tuned on the customized dataset HRIGes, and the recognition results are mapped to a fivefingered dexterous manipulator to realize real-time human-robot interaction.
Original languageEnglish
Title of host publicationProceedings of 29th IEEE International Conference on Mechatronics and Machine Vision in Practice
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9798350325621
ISBN (Print)9798350325638
Publication statusPublished - 2 Feb 2024
Event29th IEEE International Conference On Mechatronics And Machine Vision In Practice - Queenstown, New Zealand
Duration: 21 Nov 202324 Nov 2023


Conference29th IEEE International Conference On Mechatronics And Machine Vision In Practice
Country/TerritoryNew Zealand


  • serial-parallel network
  • hand gesture recognition
  • human-robot interaction

Cite this