Current Volume 9
Multimodal Sensor-Agnostic Gesture- Controlled Gaming Interface with Adaptive Depth Integration
Abstract
Despite the fact that HCI through gestures has gained significant popularity due to their intuitiveness in recent years, the current gesture recognition systems lack robustness since they rely on hardware support, use a single modality as input, and have few practical applications in real-world settings. In this paper, a gesture recognition algorithm that does not require any hardware support except for the camera sensor is introduced, taking into account RGB data and Microsoft Kinect depth sensors. In particular, the gesture recognizer is based on the detection of the joints of a skeleton and hand landmarks. The mathematical formula used for the proposed multimodal representation is shown below. F_t = [S_t || H_t || D_t]. These experiments are carried out using our own customized gesture dataset that includes 4,000 gestures from 4 categories from 10 different subjects. Recognition accuracy is 91.6% and latency is 34 ms in real-time operation mode, which are significantly better than the outcomes of the baselines based on individual modality.
Keywords
Gesture Recognition, Human Computer Interaction, Multimodal Fusion, LSTM, MediaPipe, Depth Sensing
Citations
IRE Journals:
Sagar Gangal, Rahul Nelogi, Sachidananda K "Multimodal Sensor-Agnostic Gesture- Controlled Gaming Interface with Adaptive Depth Integration" Iconic Research And Engineering Journals Volume 9 Issue 10 2026 Page 3184-3191 https://doi.org/10.64388/IREV9I10-1716845
IEEE:
Sagar Gangal, Rahul Nelogi, Sachidananda K
"Multimodal Sensor-Agnostic Gesture- Controlled Gaming Interface with Adaptive Depth Integration" Iconic Research And Engineering Journals, 9(10) https://doi.org/10.64388/IREV9I10-1716845
About IRE Journals
IRE Journals is an open access online journals established with an aim to publish high quality of research work in various diciplines. We have more than 6 years in publications. Over 2500+ papers already published.
Important Links
For Authors
Volume 8:
Issue 1, Issue 2, Issue 3, Issue 4, Issue 5, Issue 6, Issue 7, Issue 8, Issue 9, Issue 10, Issue 11, Issue 12Volume 7:
Issue 1, Issue 2, Issue 3, Issue 4, Issue 5, Issue 6, Issue 7, Issue 8, Issue 9, Issue 10, Issue 11, Issue 12Volume 6:
Issue 1, Issue 2, Issue 3, Issue 4, Issue 5, Issue 6, Issue 7, Issue 8, Issue 9, Issue 10, Issue 11, Issue 12Volume 5:
Issue 1, Issue 2, Issue 3, Issue 4, Issue 5, Issue 6, Issue 7, Issue 8, Issue 9, Issue 10, Issue 11, Issue 12Volume 4:
Issue 1, Issue 2, Issue 3, Issue 4, Issue 5, Issue 6, Issue 7, Issue 8, Issue 9, Issue 10, Issue 11, Issue 12Volume 3:
Issue 1, Issue 2, Issue 3, Issue 4, Issue 5, Issue 6, Issue 7, Issue 8, Issue 9, Issue 10, Issue 11, Issue 12