International Peer-Reviewed Journal•Open Access•ISSN 2456-8880
[email protected]•+91-7433024337

Home / Current Issue / Paper 1725231

1725231 Vol 10 · Issue 5 Download Paper

Real-Time Static Hand-Gesture-to-Text Conversion Using MediaPipe Landmarks and Classical Machine Learning

Prof. Viddulata Patil Rudra Shelke Samruddhi Kaule Samarthi Mante Pavan Memane

Subject area: Science,Engineering and Technology  ·  Area of research: Computer Vision

Abstract

This paper presents a low-cost, browser-based system that converts static hand gestures into text in real time using only a standard webcam. Hand landmarks are extracted with MediaPipe Hands, converted into a 42-dimensional wrist-relative, scale-normalized feature vector, and classified into 36 classes (digits 0–9 and letters a–z). A hold-to-confirm state machine with a short voting filter turns the per-frame predictions into words and sentences, and the application is deployed as a Streamlit web service using WebRTC. On a custom landmark dataset of 3,263 samples, a Random Forest classifier reaches 95.9% accuracy and 0.952 macro F1 on a stratified 20% hold-out set (majority-class baseline: 4.6%). Repeated stratified cross-validation on the training partition gives 94.2 ± 1.0% accuracy, and Random Forest, k-nearest neighbours and RBF-SVM classifiers differ by about one percentage point after tuning. Classifier-only inference takes a median of 2.8 ms per sample on a laptop CPU. The evaluation is row-level and not signer-independent; this limitation, together with confusable classes such as 0/o and 6/w, is analysed and directions for signer-independent evaluation are given.

Keywords

gesture recognition, hand landmarks, MediaPipe, random forest, real-time systems, Streamlit

References

[1] Rautaray SS, Agrawal A. Vision based hand gesture recognition for human computer interaction: A survey. Artif. Intell. Rev. 2015;43(1):1–54.

[2] Pugeault N, Bowden R. Spelling it out: real-time ASL fingerspelling recognition. In: Proc. IEEE Int. Conf. Computer Vision Workshops (ICCV Workshops). 2011:1114–1119.

[3] Zhang F, Bazarevsky V, Vakunov A, Tkachenka A, Sung G, Chang CL, Grundmann M. MediaPipe Hands: on-device real-time hand tracking. arXiv:2006.10214; 2020.

[4] Lugaresi C, Tang J, Nash H, McClanahan C, Uboweja E, Hays M, et al. MediaPipe: a framework for building perception pipelines. arXiv:1906.08172; 2019.

[5] Breiman L. Random forests. Mach. Learn. 2001;45(1):5–32.

[6] Cortes C, Vapnik V. Support-vector networks. Mach. Learn. 1995;20(3):273–297.

[7] Cover T, Hart P. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory. 1967;13(1):21–27.

[8] Pedregosa F, et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 2011;12:2825–2830.

[9] Bradski G. The OpenCV library. Dr. Dobb’s Journal of Software Tools. 2000.

How to cite this paper

Prof. Viddulata Patil, Rudra Shelke, Samruddhi Kaule, Samarthi Mante, Pavan Memane "Real-Time Static Hand-Gesture-to-Text Conversion Using MediaPipe Landmarks and Classical Machine Learning" Iconic Research And Engineering Journals Volume 10 Issue 5 2026 Page 3363-3369
Prof. Viddulata Patil, Rudra Shelke, Samruddhi Kaule, Samarthi Mante, Pavan Memane "Real-Time Static Hand-Gesture-to-Text Conversion Using MediaPipe Landmarks and Classical Machine Learning" Iconic Research And Engineering Journals, vol. 10, no. 5, Nov. 2026
Prof. Viddulata Patil, Rudra Shelke, Samruddhi Kaule, Samarthi Mante, Pavan Memane (2026). Real-Time Static Hand-Gesture-to-Text Conversion Using MediaPipe Landmarks and Classical Machine Learning. Iconic Research And Engineering Journals, 10(5).
Prof. Viddulata Patil, Rudra Shelke, Samruddhi Kaule, Samarthi Mante, Pavan Memane "Real-Time Static Hand-Gesture-to-Text Conversion Using MediaPipe Landmarks and Classical Machine Learning" Iconic Research And Engineering Journals, vol. 10, no. 5, Nov. 2026.
@article{1725231,
      author = {Prof. Viddulata Patil, Rudra Shelke, Samruddhi Kaule, Samarthi Mante, Pavan Memane},
      title = {Real-Time Static Hand-Gesture-to-Text Conversion Using MediaPipe Landmarks and Classical Machine Learning},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {10},
      number = {5},
      pages = {3363-3369},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1725231.pdf},
      abstract = {This paper presents a low-cost, browser-based system that converts static hand gestures into text in real time using only a standard webcam. Hand landmarks are extracted with MediaPipe Hands, converted into a 42-dimensional wrist-relative, scale-normalized feature vector, and classified into 36 classes (digits 0–9 and letters a–z). A hold-to-confirm state machine with a short voting filter turns the per-frame predictions into words and sentences, and the application is deployed as a Streamlit web service using WebRTC. On a custom landmark dataset of 3,263 samples, a Random Forest classifier reaches 95.9% accuracy and 0.952 macro F1 on a stratified 20% hold-out set (majority-class baseline: 4.6%). Repeated stratified cross-validation on the training partition gives 94.2 ± 1.0% accuracy, and Random Forest, k-nearest neighbours and RBF-SVM classifiers differ by about one percentage point after tuning. Classifier-only inference takes a median of 2.8 ms per sample on a laptop CPU. The evaluation is row-level and not signer-independent; this limitation, together with confusable classes such as 0/o and 6/w, is analysed and directions for signer-independent evaluation are given.},
      keywords = {gesture recognition, hand landmarks, MediaPipe, random forest, real-time systems, Streamlit},
      month = {November},
  }