Skeleton-based multi-feature sharing real-time action recognition network for human-robot interaction

Zhiwen Deng, Qing Gao, Xiang Yu, Zhaojie Ju, Junkang Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

31 Downloads (Pure)

Abstract

Human-Robot Interaction (HRI) is one of the directions that deserves to be studied, and it is used in various fields (e.g., emergency rescue, telemedicine, and astronaut assistance). Action-based HRI has been proven practical in many application scenarios (e.g., industrial teleoperation and telemedicine). Among these many applications, speed and accuracy are two common optimization goals for researchers. A skeleton-based multi-feature sharing real-time action recognition network (MSR-Net) has been proposed to ensure its high accuracy and speed. Three pairs of feature inputs are proposed to make the network input more informative, which contain: joint distance-joint distance motion (JD-JDM), angle-angle motion (A-AM), slow motion coordinates-fast motion coordinates (SMC-FMC). To ensure that features are fully extracted while reducing the number of model parameters, one-dimensional convolutional neural networks (1DCNN) are used to build multi-feature sharing two-stage feature extraction networks. As a result, MSR-Net outperforms state-of-the-art models in accuracy on the JHMDB (86.8%) and SHREC datasets (96.8% on coarse class and 93.8% on fine class). Application experiments were carried out on the HRI platform to demonstrate the effectiveness of MSR-Net in HRI. The video demonstration of the experimental results of the HRI application can be accessed on https://youtu.be/NzXrngh7BbQ.
Original languageEnglish
Title of host publicationProceedings of International Conference on Advanced Robotics and Mechatronics (ICARM) 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages25-30
ISBN (Electronic)9798350300178
ISBN (Print)9798350300185
DOIs
Publication statusPublished - 25 Aug 2023
EventInternational Conference on Advanced Robotics and Mechatronics (ICARM) -
Duration: 8 Jul 202310 Jul 2023

Conference

ConferenceInternational Conference on Advanced Robotics and Mechatronics (ICARM)
Period8/07/2310/07/23

Cite this