Hand gesture recognition based on signals cross-correlation

Interactive gestures and body movements let us control and interact mobile devices, screens and robots. Vision-based gesture recognition systems analyze the detected infrared and visible light after converting them into some measurable signal, e.g. voltage or current. Since, infrared and visible light are electromagnetic waves (EMW) with particular wavelength between 0.4 and 1.6μm, we introduce a concept of a new kind of sensor for direct perception of EMW to see objects. We propose a novel framework for hand gesture featuring, profiling and recognizing based on signal processing and cross correlation of detected signals instead of Euclidean space analysis of image pixels by visual-based algorithms. Hand segmentation is accomplished on infrared radiation, while hand joints are categorized according to the intensity of visible light on hand edges. The meaning of a gesture is described by wave-based profiles representing the informative features of hand joints and their spatial relations over some period of time. A hand joint profile is a waveform of known shape obtained by superposition of feature waves. During the hand segmentation, we use online fuzzy clustering to categorize the infrared radiation. During the feature extraction, the clustering algorithm categorizes the grayscale light intensity on hand edges. During the training, the hand joint profiles are stored in the database as sampled sequences corresponding to the superposition of sine waves with amplitudes and frequencies derived from the obtained clusters. During the recognition phase, the current hand gesture is matched to the hand joint profiles in the database by fast signals cross-correlation. Our first implementation of the proposed framework inputs the raw data of Microsoft Kinect infrared and RGB image sensors that are wavelength dependent and produce a signal for electric current that is directly proportional to the changes in the focused flow of reflected light during hand gesturing.