Toward Responsible Deployment of On Device Language Models: A Framework for Privacy Centric Evaluation and Regulatory Alignment

References

[1] Gaurav, A., Gupta, B. B., Wu, J., & Chaurasia, P. (2024). Regulatory and Policy Considerations for Large Language Models: Frameworks, Implications, and Future Directions. In Challenges in Large Language Model Development and AI Ethics (pp. 212-238). IGI Global.

[2] Zhou, S., Zhou, Z., Wang, C., Liang, Y., Wang, L., Zhang, J., ... & Lv, C. (2024). A User-Centered Framework for Data Privacy Protection Using Large Language Models and Attention Mechanisms. Applied Sciences, 14(15), 6824.

[3] Pujari, Mangesh & Pakina, Anil Kumar. (2023). Balancing Innovation and Privacy: A Red Teaming Approach to Evaluating Phone-Based Large Language Models under AI Privacy Regulations. International Journal Science and Technology. 2. 10.56127/ijst.v2i3.1956.

[4] Woodring, J., Perez, K., & Ali-Gombe, A. (2024). Enhancing privacy policy comprehension through privacify: A user-centric approach using advanced language models. Computers & Security, 145, 103997.

[5] Shao, Y., Li, T., Shi, W., Liu, Y., & Yang, D. (2024). Privacylens: Evaluating privacy norm awareness of language models in action. arXiv preprint arXiv:2409.00138.

[6] Kibriya, H., Khan, W. Z., Siddiqa, A., & Khan, M. K. (2024). Privacy issues in large language models: a survey. Computers and Electrical Engineering, 120, 109698.

[7] Kirk, H. R., Vidgen, B., Röttger, P., & Hale, S. A. (2024). The benefits, risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence, 6(4), 383-392.

[8] Chadwick, R., Blundell, S., & Prendergast, E. (2024). Assessments of the privacy compliance in commercial large language models.

[9] Kirk, H. R., Vidgen, B., Röttger, P., & Hale, S. A. (2023). Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback. arXiv preprint arXiv:2303.05453.

[10] Radanliev, P., Santos, O., Brandon-Jones, A., & Joinson, A. (2024). Ethics and responsible AI deployment. Frontiers in Artificial Intelligence, 7, 1377011.

[11] Yang, Q. (2021). Toward responsible ai: An overview of federated learning for user-centered privacy-preserving computing. ACM Transactions on Interactive Intelligent Systems (TiiS), 11(3-4), 1-22.

[12] Shen, T., Jin, R., Huang, Y., Liu, C., Dong, W., Guo, Z., ... & Xiong, D. (2023). Large language model alignment: A survey. arXiv preprint arXiv:2309.15025.

[13] Xu, J., Li, Z., Chen, W., Wang, Q., Gao, X., Cai, Q., & Ling, Z. (2024). On-device language models: A comprehensive review. arXiv preprint arXiv:2409.00088.

[14] Miranda, M., Ruzzetti, E. S., Santilli, A., Zanzotto, F. M., Bratières, S., & Rodolà, E. (2024). Preserving privacy in large language models: A survey on current threats and solutions. arXiv preprint arXiv:2408.05212.

[15] Tarkoma, S., Morabito, R., & Sauvola, J. (2023). AI-native interconnect framework for integration of large language model technologies in 6G systems. arXiv preprint arXiv:2311.05842.

[16] Rivadeneira, J. E., Silva, J. S., Colomo-Palacios, R., Rodrigues, A., & Boavida, F. (2023). User-centric privacy preserving models for a new era of the Internet of Things. Journal of Network and Computer Applications, 217, 103695.

[17] Mylrea, M., & Robinson, N. (2023). Artificial Intelligence (AI) trust framework and maturity model: applying an entropy lens to improve security, privacy, and ethical AI. Entropy, 25(10), 1429.

[18] Chen, J., Liu, Z., Huang, X., Wu, C., Liu, Q., Jiang, G., ... & Chen, E. (2024). When large language models meet personalization: Perspectives of challenges and opportunities. World Wide Web, 27(4), 42.

[19] Malode, V. M. (2024). Benchmarking public large language model (Doctoral dissertation, Technische Hochschule Ingolstadt).

[20] Pandhare, H. V. (2024). Evaluating Large Language Models: Frameworks and Methodologies for AI/ML System Testing. International Journal of Scientific Research and Management (IJSRM), 12(09), 1467-1486.

[21] Gupta, B. B., Gaurav, A., Arya, V., Alhalabi, W., Alsalman, D., & Vijayakumar, P. (2024). Enhancing user prompt confidentiality in Large Language Models through advanced differential encryption. Computers and Electrical Engineering, 116, 109215.

[22] Williamson, S. M., & Prybutok, V. (2024). Balancing privacy and progress: a review of privacy challenges, systemic oversight, and patient perceptions in AI-driven healthcare. Applied Sciences, 14(2), 675.

[23] Pujari, T., Goel, A., & Sharma, A. (2024). Ensuring responsible AI: The role of supervised fine-tuning (SFT) in upholding integrity and privacy regulations. International Journal Science and Technology, 3(3), 46-55.

[24] Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., ... & Ge, B. (2023). Summary of chatgpt-related research and perspective towards the future of large language models. Meta-radiology, 1(2), 100017.

[25] Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024). A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, 100211.

[26] Ferrara, E. (2023). Should chatgpt be biased? challenges and risks of bias in large language models. arXiv preprint arXiv:2304.03738.

[27] Khowaja, S. A., Khuwaja, P., Dev, K., Wang, W., & Nkenyereye, L. (2024). Chatgpt needs spade (sustainability, privacy, digital divide, and ethics) evaluation: A review. Cognitive Computation, 16(5), 2528-2550.

[28] Obradovich, N., Khalsa, S. S., Khan, W. U., Suh, J., Perlis, R. H., Ajilore, O., & Paulus, M. P. (2024). Opportunities and risks of large language models in psychiatry. NPP—Digital Psychiatry and Neuroscience, 2(1), 8.

[29] Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., de Prado, M. L., Herrera-Viedma, E., & Herrera, F. (2023). Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation. Information Fusion, 99, 101896.

[30] Shen, H., Knearem, T., Ghosh, R., Alkiek, K., Krishna, K., Liu, Y., ... & Jurgens, D. (2024). Towards bidirectional human-ai alignment: A systematic review for clarifications, framework, and future directions. arXiv preprint arXiv:2406.09264.

@article{1709117,
      author = {Deepak Kejriwal, Saurabh Kansal},
      title = {Toward Responsible Deployment of On Device Language Models: A Framework for Privacy Centric Evaluation and Regulatory Alignment},
      journal = {Iconic Research And Engineering Journals},
      year = {2024},
      volume = {7},
      number = {7},
      pages = {701-714},
      issn = {2456-8880},
      url = {https://www.irejournals.com/formatedpaper/1709117.pdf},
      abstract = {The integration of large language models (LLMs) into mobile and edge devices heralds a paradigm shift in personalized, low latency AI experiences. Yet on the other side, a plethora of concerns arise regarding user privacy, regulation, and adversarial robustness. Till date, there remain certain shortcomings inherent to methodologies assessing privacy risks (e.g., red teaming, privacy probes) that could not be adequately extended to resource constrained environments, such as smartphones or embedded systems. We introduce an encompassing multilayer framework for evaluating and mitigating privacy risks of on device LLMs. This includes synthetic data injection, differential audit trails, and consent aware tracing mechanisms that go hand in hand with developments in privacy regulations such as the EU AI Act and data protection laws at the U.S. state level. Based on real-world case studies ranging from messaging to voice assistant platforms, we investigate the challenges of implementation and regulatory incoherence and technical approaches that can help bridge the chasm between theoretical compliance and actual deployment. The hybrid result of these studies emphasizes that embedding regulatory principles within LLM system architectures is the initiative toward responsible AI in real-time privacy-sensitive settings.},
      keywords = {On-device LLMs, Privacy centric AI, AI regulation, EU AI Act, Data protection, Consent aware tracing, Red teaming, Edge AI, Regulatory alignment, Privacy leakage},
      month = {January},
  }