Assistive Communication Framework

References

[1] Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Proc. NeurIPS, pp. 1097–1105, 2012.

[2] Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.

[3] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.

[4] Vaswani et al., “Attention Is All You Need,” Proc. NeurIPS, pp. 5998–6008, 2017.

[5] K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks,” Proc. ICLR, 2015.

[6] Szegedy et al., “Going Deeper with Convolutions,” Proc. CVPR, pp. 1–9, 2015.

[7] J. Devlin et al., “BERT: Pre-training of Deep Bidirectional Transformers,” Proc. NAACL, pp. 4171–4186, 2019.

[8] Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, 2016.

[9] T. Baltrušaitis, C. Ahuja, and L.-P. Morency, “Multimodal Machine Learning: A Survey,” IEEE TPAMI, vol. 41, no. 2, pp. 423–443, 2019.

[10] Z. Zhang et al., “Multimodal Deep Learning for Assistive Technology,” IEEE Access, vol. 8, pp. 12345–12360, 2020.

[11] O. Vinyals et al., “Show and Tell: Image Caption Generator,” Proc. CVPR, pp. 3156–3164, 2015.

[12] Bahdanau et al., “Neural Machine Translation,” Proc. ICLR, 2015.

[13] sGraves et al., “Speech Recognition with Deep RNNs,” Proc. ICASSP, pp. 6645–6649, 2013.

[14] G. Hinton et al., “Deep Neural Networks for Acoustic Modeling,” IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 82–97, 2012.

[15] He et al., “Deep Residual Learning for Image Recognition,” Proc. CVPR, pp. 770–778, 2016.

[16] Redmon et al., “YOLO: Real-Time Object Detection,” Proc. CVPR, pp. 779–788, 2016.A. Howard et al., “MobileNets: Efficient CNNs,” arXiv preprint arXiv:1704.04861, 2017.

[17] S. Ren et al., “Faster R-CNN,” Proc. NeurIPS, pp. 91–99, 2015.

[18] R. Girshick, “Fast R-CNN,” Proc. ICCV, pp. 1440–1448, 2015.

[19] H. Hermansky, “PLP Analysis of Speech,” J. Acoust. Soc. Am., vol. 87, no. 4, pp. 1738–1752, 1990.

[20] Abadi et al., “TensorFlow: Large-Scale ML System,” Proc. OSDI, pp. 265–283, 2016.

[21] Paszke et al., “PyTorch: Deep Learning Library,” Proc. NeurIPS, pp. 8026–8037, 2019.

[22] S. Koller et al., “Sign Language Recognition,”

[23] Proc. ICCV Workshops, pp. 85–91, 2015.

[24] Neverova et al., “ModDrop: Multimodal Gesture Recognition,” IEEE TPAMI, vol. 38, no. 8, pp. 1692–1706, 2016.

[25] R. W. Picard, Affective Computing, MIT Press, 1997.

@article{1716490,
      author = {M Nithya, V Priya, S Rajeswari, S Harini},
      title = {Assistive Communication Framework},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {9},
      number = {10},
      pages = {2031-2035},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1716490.pdf},
      abstract = {Assistive communication technologies have significantly improved the quality of life for individuals with speech and motor impairments. However, existing systems often rely on unimodal inputs such as text or voice, limiting their usability in real-world scenarios. This paper proposes a deep learning-based multimodal assistive communication framework that integrates visual, auditory, and textual inputs to enable robust and adaptive communication. The framework leverages convolutional neural networks (CNNs), recurrent neural networks (RNNs), and transformer-based architectures for feature extraction and fusionExperimental results demonstrate improved accuracy, responsiveness, and usability compared to traditional systems.The proposed system aims to provide an inclusive, scalable, and real- time communication solution. and few lines Furthermore, an attention-based multimodal fusion strategy is employed to effectively combine heterogeneous features and improve contextual understanding. The system is designed to operate reliably under noisy and dynamic conditions by handling incomplete or ambiguous inputs from different modalities. User-centric design considerations are incorporated to ensure accessibility, ease of use, and real-time responsiveness Experimental results demonstrate improved accuracy, responsiveness, and usability compared to traditional unimodal systems. The proposed framework shows strong potential for deployment in real-world assistive applications, providing an inclusive, scalable, and intelligent communication solution.},
      keywords = {Assistive Communication, Multimodal Learning, Deep Learning, CNN, RNN, Transformer, Human-Computer Interaction},
      month = {April},
      doi = {https://doi.org/10.64388/IREV9I10-1716490}
  }