Deep learning of fuzzy weighted multi-resolution depth motion maps with spatial feature fusion for action recognition

Mahmoud M N Al-Faris, John Chiverton, Linda Yang, David Ndzi

Research output: Contribution to journalArticlepeer-review

174 Downloads (Pure)

Abstract

Human action recognition (HAR) is an important yet challenging task. This paper presents a novel method. First, fuzzy weight functions are used in computations of depth motion maps (DMMs). Multiple length motion information is also used. These features are referred to as fuzzy weighted multi-resolution DMMs (FWMDMMs). This formulation allows for various aspects of individual actions to be emphasized. It also helps to characterise the importance of the temporal dimension. This is important to help overcome, e.g., variations in time over which a single type of action might be performed. A deep convolutional neural network (CNN) motion model is created and trained to extract discriminative and compact features. Transfer learning is also used to extract spatial information from RGB and depth data using the AlexNet network. Different late fusion techniques are then investigated to fuse the deep motion model with the spatial network. The result is a spatial temporal HAR model. The developed approach is capable of recognising both human action and human–object interaction. Three public domain datasets are used to evaluate the proposed solution. The experimental results demonstrate the robustness of this approach compared with state-of-the art algorithms.
Original languageEnglish
Article number82
Number of pages25
JournalJournal of Imaging
Volume5
Issue number10
DOIs
Publication statusPublished - 21 Oct 2019

Keywords

  • action recognition
  • transfer learning
  • multi-resolution
  • feature fusion

Fingerprint

Dive into the research topics of 'Deep learning of fuzzy weighted multi-resolution depth motion maps with spatial feature fusion for action recognition'. Together they form a unique fingerprint.

Cite this