Home / Current Issue / Paper 1725231
Real-Time Static Hand-Gesture-to-Text Conversion Using MediaPipe Landmarks and Classical Machine Learning
Subject area: Science,Engineering and Technology · Area of research: Computer Vision
Abstract
This paper presents a low-cost, browser-based system that converts static hand gestures into text in real time using only a standard webcam. Hand landmarks are extracted with MediaPipe Hands, converted into a 42-dimensional wrist-relative, scale-normalized feature vector, and classified into 36 classes (digits 0–9 and letters a–z). A hold-to-confirm state machine with a short voting filter turns the per-frame predictions into words and sentences, and the application is deployed as a Streamlit web service using WebRTC. On a custom landmark dataset of 3,263 samples, a Random Forest classifier reaches 95.9% accuracy and 0.952 macro F1 on a stratified 20% hold-out set (majority-class baseline: 4.6%). Repeated stratified cross-validation on the training partition gives 94.2 ± 1.0% accuracy, and Random Forest, k-nearest neighbours and RBF-SVM classifiers differ by about one percentage point after tuning. Classifier-only inference takes a median of 2.8 ms per sample on a laptop CPU. The evaluation is row-level and not signer-independent; this limitation, together with confusable classes such as 0/o and 6/w, is analysed and directions for signer-independent evaluation are given.
Keywords
gesture recognition, hand landmarks, MediaPipe, random forest, real-time systems, Streamlit
References
[1] Rautaray SS, Agrawal A. Vision based hand gesture recognition for human computer interaction: A survey. Artif. Intell. Rev. 2015;43(1):1–54.
[2] Pugeault N, Bowden R. Spelling it out: real-time ASL fingerspelling recognition. In: Proc. IEEE Int. Conf. Computer Vision Workshops (ICCV Workshops). 2011:1114–1119.
[3] Zhang F, Bazarevsky V, Vakunov A, Tkachenka A, Sung G, Chang CL, Grundmann M. MediaPipe Hands: on-device real-time hand tracking. arXiv:2006.10214; 2020.
[4] Lugaresi C, Tang J, Nash H, McClanahan C, Uboweja E, Hays M, et al. MediaPipe: a framework for building perception pipelines. arXiv:1906.08172; 2019.
[5] Breiman L. Random forests. Mach. Learn. 2001;45(1):5–32.
[6] Cortes C, Vapnik V. Support-vector networks. Mach. Learn. 1995;20(3):273–297.
[7] Cover T, Hart P. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory. 1967;13(1):21–27.
[8] Pedregosa F, et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 2011;12:2825–2830.
[9] Bradski G. The OpenCV library. Dr. Dobb’s Journal of Software Tools. 2000.
How to cite this paper
@article{1725231,
author = {Prof. Viddulata Patil, Rudra Shelke, Samruddhi Kaule, Samarthi Mante, Pavan Memane},
title = {Real-Time Static Hand-Gesture-to-Text Conversion Using MediaPipe Landmarks and Classical Machine Learning},
journal = {Iconic Research And Engineering Journals},
year = {2026},
volume = {10},
number = {5},
pages = {3363-3369},
issn = {2456-8880},
url = {https://www.irejournals.com/formatedpaper/1725231.pdf},
abstract = {This paper presents a low-cost, browser-based system that converts static hand gestures into text in real time using only a standard webcam. Hand landmarks are extracted with MediaPipe Hands, converted into a 42-dimensional wrist-relative, scale-normalized feature vector, and classified into 36 classes (digits 0–9 and letters a–z). A hold-to-confirm state machine with a short voting filter turns the per-frame predictions into words and sentences, and the application is deployed as a Streamlit web service using WebRTC. On a custom landmark dataset of 3,263 samples, a Random Forest classifier reaches 95.9% accuracy and 0.952 macro F1 on a stratified 20% hold-out set (majority-class baseline: 4.6%). Repeated stratified cross-validation on the training partition gives 94.2 ± 1.0% accuracy, and Random Forest, k-nearest neighbours and RBF-SVM classifiers differ by about one percentage point after tuning. Classifier-only inference takes a median of 2.8 ms per sample on a laptop CPU. The evaluation is row-level and not signer-independent; this limitation, together with confusable classes such as 0/o and 6/w, is analysed and directions for signer-independent evaluation are given.},
keywords = {gesture recognition, hand landmarks, MediaPipe, random forest, real-time systems, Streamlit},
month = {November},
}