Fine-Tuning of a LLM for Public Complaint Classification and Routing for Resolution

References

[1] T. Brown et al., “Language Models are Few-Shot Learners,” Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901, 2020.

[2] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How Transferable Are Features in Deep Neural Networks?,” Advances in Neural Information Processing Systems, vol. 27, 2014.

[3] J. Howard and S. Ruder, “Universal Language Model Fine-Tuning for Text Classification,” in Proc. ACL, 2018.

[4] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proc. NAACL-HLT, 2019.

[5] A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving Language Understanding by Generative Pre-Training,” OpenAI Technical Report, 2018.

[6] S. J. Pan and Q. Yang, “A Survey on Transfer Learning,” IEEE Transactions on Knowledge and Data Engineering, 2010.

[7] N. Houlsby et al., “Parameter-Efficient Transfer Learning for NLP,” in Proc. ICML, 2019.

[8] B. Lester, R. Al-Rfou, and N. Constant, “The Power of Scale for Parameter-Efficient Prompt Tuning,” in Proc. EMNLP, 2021.

[9] X. L. Li and P. Liang, “Prefix-Tuning: Optimizing Continuous Prompts for Generation,” in Proc. ACL, 2021.

[10] A. Vaswani et al., “Attention Is All You Need,” Advances in Neural Information Processing Systems, 2017.

[11] J. Wei et al., “Finetuned Language Models Are Zero-Shot Learners,” in Proc. ICLR, 2022.

[12] L. Ouyang et al., “Training Language Models to Follow Instructions with Human Feedback,” Advances in Neural Information Processing Systems, 2022.

[13] E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” arXiv:2106.09685, 2021.

[14] T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient Finetuning of Quantized LLMs,” Advances in Neural Information Processing Systems, 2023.

[15] J. Kirkpatrick et al., “Overcoming Catastrophic Forgetting in Neural Networks,” Proceedings of the National Academy of Sciences, 2017.

[16] Y. Kim, “Convolutional Neural Networks for Sentence Classification,” in Proc. EMNLP, 2014.

[17] S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.

[18] K. Cho et al., “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation,” in Proc. EMNLP, 2014.

[19] T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient Estimation of Word Representations in Vector Space,” arXiv:1301.3781, 2013.

[20] J. Pennington, R. Socher, and C. D. Manning, “GloVe: Global Vectors for Word Representation,” in Proc. EMNLP, 2014.

[21] L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001.

[22] T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” in Proc. KDD, 2016.

[23] C. Cortes and V. Vapnik, “Support-Vector Networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995.

[24] Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” arXiv:1907.11692, 2019.

[25] C. Raffel et al., “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,” Journal of Machine Learning Research, vol. 21, no. 140, pp. 1–67, 2020.

[26] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a Distilled Version of BERT,” arXiv:1910.01108, 2019.

[27] H. Touvron et al., “LLaMA: Open and Efficient Foundation Language Models,” arXiv:2302.13971, 2023.

[28] H. Touvron et al., “Llama 2: Open Foundation and Fine-Tuned Chat Models,” arXiv:2307.09288, 2023.

[29] A. Q. Jiang et al., “Mistral 7B,” arXiv:2310.06825, 2023.

[30] A. Q. Jiang et al., “Mixtral of Experts,” arXiv:2401.04088, 2024.

[31] Gemma Team, “Gemma: Open Models Based on Gemini Research and Technology,” arXiv:2403.08295, 2024.

[32] E. Almazrouei et al., “The Falcon Series of Open Language Models,” arXiv:2311.16867, 2023.

[33] E. Frantar, S. Ashkboos, T. Hoefler, and D. Alistarh, “GPTQ: Accurate Post-Training Quantization for Generative Pretrained Transformers,” in Proc. ICLR, 2023.

[34] Q. Zhang et al., “Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning,” in Proc. ICLR, 2023.

[35] S. Kumar, P. Mehta, and R. Verma, “Deep Learning-Based Complaint Classification Model for Grievance Redressal,” in Proc. IEEE Int. Conf. Smart Governance, 2021, pp. 33–40.

[36] S. C. Rajkumar, D. Yuvasini, Shitharth Selvarajan, and N. R. Sivakumar, “A Zero-Shot LLM Framework for Multimodal Grievance Classification, Urgency Scoring, and Abuse Detection in Civic Feedback Systems,” Scientific Reports, vol. 16, Art. no. 2299, 2026.

[37] M. Esperança, D. Freitas, P. V. Paixão, T. A. Marcos, R. A. Martins, and J. C. Ferreira, “Proactive Complaint Management in Public Sector Informatics Using AI: A Semantic Pattern Recognition Framework,” MDPI Applied Sciences, vol. 15, no. 12, p. 6673, 2025.

[38] C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval, Cambridge University Press, 2008.

[39] G. Salton and C. Buckley, “Term-Weighting Approaches in Automatic Text Retrieval,” Information Processing & Management, vol. 24, no. 5, pp. 513–523, 1988.

[40] Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” Nature, vol. 521, pp. 436–444, 2015.

[41] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323, pp. 533–536, 1986.

[42] M. Schuster and K. K. Paliwal, “Bidirectional Recurrent Neural Networks,” IEEE Trans. Signal Processing, vol. 45, no. 11, pp. 2673–2681, 1997.

[43] D. Bahdanau, K. Cho, and Y. Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate,” in Proc. ICLR, 2015.

@article{1720173,
      author = {Suraj Sharma, Dr. Amandeep, Dr. Dharmender Kumar, Ankit, Keshav Kumar},
      title = {Fine-Tuning of a LLM for Public Complaint Classification and Routing for Resolution},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {10},
      number = {1},
      pages = {2724-2735},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1720173.pdf},
      abstract = {The tremendous growth of digital governance platforms has significantly increased the number of complaints being lodged in the online Public Grievance Redressal Management System like CPGRAMS. These complaints are often processed and routed manually, which results in delays and inconsistency in categorization and administrative workload. In response to these challenges, the research proposes an intelligent public complaint classification and routing framework based on the Mistral-7B-Instruct-v0.2 large language model (LLM) with fine-tuning using the Quantized Low-Rank Adaptation (QLoRA) technique. The framework will automatically classify multilingual complaints to government service categories (Electricity, Water Supply, Roads and Transport, Health, Education, Banking, Telecom, E-Commerce) while, predicting a three-level priority (High, Medium, Low) and routing the complaint to the department based on confidence mechanism.Our methodology calls for data generation with CPGRAMS, data preprocessing, instruction prompt formatting in Mistral chat format, and parameter-efficient fine-tuning with the help of Hugging Face Transformers, TRL, PEFT, and BitsAndBytes libraries. The model was loaded in 4-bit NF4 quantized format with LoRA adapters (rank = 16, alpha = 32) injected into projection layers of the transformer, allowing tuning of the 7B model on 1 NVIDIA Tesla T4 (16 GB). During the training, only 41.9 million parameters (0.58%) were updated, which led to reduced GPU memory and computational cost while maintaining the pretrained linguistic knowledge.Our model delivers a Category Classification Accuracy of 93.7% (Macro F1 ≈ 0.85) and Complaint Routing Accuracy of 96.2% on a held-out set of  multilingual complaints. That said, the performance already outperforms classical ML, and DL, and baselines of generic transformers/LLMs reported in the literature. The suggested method was tested against other NLP pipelines and grievance-classification studies and it is shown that instruction-tuned LLMs adapted through parameter-efficient methods provide better contextual understanding of informal, multilingual citizen complaints over feature-engineered or fully fine-tuned alternatives and can be deployed on low-end single-GPU hardware which is compatible with government infrastructure.The system was deployed using a Streamlit dashboard that provides real-time predictions. Only low-confidence predictions are escalated to human officers. There is human oversight even in automation. According to the study, the combination of open-source LLMs and parameter-efficient fine-tuning will offer scalable, resource-efficient & sovereign-deployable intelligent grievance management of e-Governance applications with future scope of continual learning, multilingual extension and integration with resolution-support systems.},
      keywords = {Large Language Models, Mistral-7B, QLoRA, LoRA, Parameter-Efficient Fine-Tuning, Complaint Classification, Priority Prediction, Complaint Routing, CPGRAMS, e-Governance, Natural Language Processing.},
      month = {July},
      doi = {https://doi.org/10.64388/IREV10I1-1720173}
  }