LGF-SLR: hand local-global fusion network for skeleton-based sign language recognition

For those who are deaf, sign language recognition (SLR) technology can offer a more practical and effective means of communication. A skeleton-based sign language recognition system has gained prominence and been employed in practical settings due to its advantages in generalization, computing accuracy, anti-interference, and privacy protection. It is unfortunate that this method's low recognition accuracy is frequently caused by inadequate hand-skeletal information. In order to recognize sign language, the multi-stream fusion method is also frequently employed. However, this approach frequently uses artificial weight assignment, which has been identified as having drawbacks including subjectivity and inaccuracy. This research proposes a Hand Local-Global Fusion Network for Skeleton-Based Sign Language Recognition (LGF-SLR), which deals with the aforementioned difficulties. To the greatest extent possible, the skeleton information is preserved while it is extracted and inputted into the network's global feature streams (upper body) and local feature streams (left and right hands) independently. Concurrently, the multi-stream fusion component incorporates Bayesian optimization, which automatically assigns weights to disparate prediction knots and mitigates the influence of manually assigned weights on trial outcomes. Lastly, the AUTSL sign language dataset, the WLASL2000 American sign language dataset, and the SLR500 Chinese sign language dataset are employed to evaluate the efficacy of the LGF-SLR. The findings indicate that the LGF-SLR exhibits superior accuracy compared to existing methods, with accuracy of 95.93%, 52.74%, and 98.42%, respectively. Code is available at https://github.com/MeiqiZhang7/LGF-SLR.