Fine-Tuning of a LLM for Public Complaint Classification and Routing for Resolution

@article{1720173,
      author = {Suraj Sharma, Dr. Amandeep, Dr. Dharmender Kumar, Ankit, Keshav Kumar},
      title = {Fine-Tuning of a LLM for Public Complaint Classification and Routing for Resolution},
      journal = {Iconic Research And Engineering Journals},
      year = {2026},
      volume = {10},
      number = {1},
      pages = {2724-2735},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1720173.pdf},
      abstract = {The tremendous growth of digital governance platforms has significantly increased the number of complaints being lodged in the online Public Grievance Redressal Management System like CPGRAMS. These complaints are often processed and routed manually, which results in delays and inconsistency in categorization and administrative workload. In response to these challenges, the research proposes an intelligent public complaint classification and routing framework based on the Mistral-7B-Instruct-v0.2 large language model (LLM) with fine-tuning using the Quantized Low-Rank Adaptation (QLoRA) technique. The framework will automatically classify multilingual complaints to government service categories (Electricity, Water Supply, Roads and Transport, Health, Education, Banking, Telecom, E-Commerce) while, predicting a three-level priority (High, Medium, Low) and routing the complaint to the department based on confidence mechanism.Our methodology calls for data generation with CPGRAMS, data preprocessing, instruction prompt formatting in Mistral chat format, and parameter-efficient fine-tuning with the help of Hugging Face Transformers, TRL, PEFT, and BitsAndBytes libraries. The model was loaded in 4-bit NF4 quantized format with LoRA adapters (rank = 16, alpha = 32) injected into projection layers of the transformer, allowing tuning of the 7B model on 1 NVIDIA Tesla T4 (16 GB). During the training, only 41.9 million parameters (0.58%) were updated, which led to reduced GPU memory and computational cost while maintaining the pretrained linguistic knowledge.Our model delivers a Category Classification Accuracy of 93.7% (Macro F1 ≈ 0.85) and Complaint Routing Accuracy of 96.2% on a held-out set of  multilingual complaints. That said, the performance already outperforms classical ML, and DL, and baselines of generic transformers/LLMs reported in the literature. The suggested method was tested against other NLP pipelines and grievance-classification studies and it is shown that instruction-tuned LLMs adapted through parameter-efficient methods provide better contextual understanding of informal, multilingual citizen complaints over feature-engineered or fully fine-tuned alternatives and can be deployed on low-end single-GPU hardware which is compatible with government infrastructure.The system was deployed using a Streamlit dashboard that provides real-time predictions. Only low-confidence predictions are escalated to human officers. There is human oversight even in automation. According to the study, the combination of open-source LLMs and parameter-efficient fine-tuning will offer scalable, resource-efficient & sovereign-deployable intelligent grievance management of e-Governance applications with future scope of continual learning, multilingual extension and integration with resolution-support systems.},
      keywords = {Large Language Models, Mistral-7B, QLoRA, LoRA, Parameter-Efficient Fine-Tuning, Complaint Classification, Priority Prediction, Complaint Routing, CPGRAMS, e-Governance, Natural Language Processing.},
      month = {July},
      doi = {https://doi.org/10.64388/IREV10I1-1720173}
  }