Skeleton-based multi-feature sharing real-time action recognition network for human-robot interaction

Human-Robot Interaction (HRI) is one of the directions that deserves to be studied, and it is used in various fields (e.g., emergency rescue, telemedicine, and astronaut assistance). Action-based HRI has been proven practical in many application scenarios (e.g., industrial teleoperation and telemedicine). Among these many applications, speed and accuracy are two common optimization goals for researchers. A skeleton-based multi-feature sharing real-time action recognition network (MSR-Net) has been proposed to ensure its high accuracy and speed. Three pairs of feature inputs are proposed to make the network input more informative, which contain: joint distance-joint distance motion (JD-JDM), angle-angle motion (A-AM), slow motion coordinates-fast motion coordinates (SMC-FMC). To ensure that features are fully extracted while reducing the number of model parameters, one-dimensional convolutional neural networks (1DCNN) are used to build multi-feature sharing two-stage feature extraction networks. As a result, MSR-Net outperforms state-of-the-art models in accuracy on the JHMDB (86.8%) and SHREC datasets (96.8% on coarse class and 93.8% on fine class). Application experiments were carried out on the HRI platform to demonstrate the effectiveness of MSR-Net in HRI. The video demonstration of the experimental results of the HRI application can be accessed on https://youtu.be/NzXrngh7BbQ.