Computer vision based action recognition and indoor thermal control

  • Mahmoud Al-Faris

Student thesis: Doctoral Thesis


Intelligent buildings and home automation systems can comprise an immense range of technologies. Applications can include assistive living, energy management, heating, lighting and access control. Thermal comfort can be considered to be one of most important factors in many of these systems. Thermal comfort can be determined by the Predictive Mean Vote (PMV) thermal comfort index.
For this work, a new fuzzy based adaptive PMV thermal comfort system is proposed together with a series of novel human action recognition systems. Human action recognition is incorporated into the PMV thermal comfort system. The result is a novel context-aware adaptive environmental control system. It is introduced here for first time and aims to control the indoor environmental climate based on occupant activity.
However, activity recognition has many challenges making it a difficult problem to solve. A further aim of this thesis is to develop novel techniques using hand-crafted and deep learning models to characterise and encode captured data and improve human action recognition. The first action recognition system proposed here has only basic hardware requirements and is based on hand-crafted methods. A couple of innovative methods are then presented utilising deep learning approaches. In the first approach, transfer learning using a pre-trained 2D-CNN deep model and a deep motion model are adopted and developed for human action recognition. Time-variant weighted Depth Motion Maps (DMM)s are proposed to solve an action’s region of importance issue. A second approach is proposed using a pre-trained model based on a 3D-CNN. The 3D-CNN explicitly utilises different temporal dimensions of an action. Furthermore, multi-temporal resolution and multi-view information are also included. These help to create a view and time invariant action recognition system. The methods proposed in this research achieve comparable if not competitive results compared to the state-of-the-art.
Date of AwardAug 2019
Original languageEnglish
Awarding Institution
  • University of Portsmouth
SupervisorJohn Chiverton (Supervisor) & David Ndzi (Supervisor)

Cite this